Control device, control method, and program

Abstract

To improve convenience for an occupant (for example, a driver) of a mobile object.SOLUTION: A control device includes: a recognition section that recognizes a surrounding situation of a mobile object; and a control section that causes a display section to display a lane included in the surrounding situation recognized by the recognition section, and causes the display section to display, in a case in which a target route of the mobile object is on a side of a lane adjacent to a first lane where the mobile object is traveling, an image in which an image of an object of movement display showing movement of the mobile object to the lane side is superimposed on an image of an object of the adjacent lane. In a case in which the adjacent lane is a second lane connected to the first lane and widening ahead of a location where the mobile object is traveling in a traveling direction of the mobile object, the control section causes the display section to display an image of an object of the second lane even if the recognition section has not recognized the second lane when the display section is caused to display the image of the object of the movement display.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a control device, a control method, and a program.

Background Art

[0002] In recent years, efforts to provide a sustainable transportation system that takes various situations into account have been active. Toward this realization, research and development focused on further improving traffic safety and convenience through research and development of driving support technologies. When a determination is made to recommend a lane change based on route information, a first content for guiding the lane change is displayed, and a second content for guiding the driving operation recommended after the lane change is displayed before the display of the first content ends. A display control device has been disclosed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, information corresponding to the control of the moving body cannot be displayed on the display unit, and the convenience for the user may be low. For example, when there is no road, the road of the lane change destination cannot be displayed, the timing of displaying the lane change guidance may be delayed, or the guidance display may be displayed at a position where there is no road, which may give the user a sense of discomfort. As a result, the convenience for the user may be low.

[0005] This invention was made in consideration of these circumstances, and one of its objectives is to provide a control device, control method, and program that can improve convenience for the occupants of a moving vehicle (e.g., the driver). Ultimately, it contributes to the development of sustainable transportation systems. Specifically, it can display guidance information at an appropriate time and display it in a way that does not cause any discomfort. [Means for solving the problem]

[0006] The control device, control method, and program according to this invention employ the following configuration. (1) A control device according to one aspect of the present invention includes a recognition unit that recognizes the surrounding conditions of a moving body, and a control unit that, when the target path of the moving body is on the side of a lane adjacent to the first lane on which the moving body is traveling, displays on the display unit an image in which an image of an object representing movement of the moving body toward the lane is superimposed on an image of an object in the adjacent lane, wherein when the adjacent lane is a second lane that connects to and increases from the first lane ahead of the direction of travel of the moving body rather than the position on which the moving body is traveling, the control unit displays an image of an object in the second lane on the display unit even if the recognition unit does not recognize the second lane when it displays an image of an object representing movement of the moving body.

[0007] (2) In the embodiment of (1) above, the control unit identifies, based on road information, that the second lane is located in the direction of travel of the moving body and causes the display unit to display an image of the object in the second lane.

[0008] (3) In the embodiment of (1) above, the control unit causes the display unit to simultaneously display the image of the object of the second lane and the image of the moving display object.

[0009] (4) In any embodiment of (1) above, the control unit displays the image of the object of the second lane on the display unit, and then displays the image of the object of the moving display on the display unit.

[0010] (5) The embodiment of (1) above further comprises an auto lane change control unit that performs auto lane change control to automatically change the moving body into the second lane, wherein the control unit superimposes the image of the object in the second lane and the image of the moving display object and displays them on the display unit when the auto lane change control is being performed.

[0011] (6) In the embodiment of (1) above, even if the recognition unit cannot recognize the second lane because the moving body is located a predetermined distance before the starting point of the second lane, the control unit causes the display unit to display an image including a first image of the object of the moving body, a second image of the object of the second lane, and a third image of the object of the moving display associated with the first image and the second image, when the moving body is planning to change lanes to the second lane and notifies the occupants of the moving body of the lane change to the second lane.

[0012] (7): In the embodiment of (6) above, the control unit displays an image of the lane object recognized by the recognition unit on the display unit, and when the occupant of the moving body gives an instruction to the automatic lane change control unit to automatically change lanes, even if the second lane is not included in the lanes recognized by the recognition unit when the moving body reaches a position of a first distance from the second lane, the control unit displays an image of the object of the second lane on the display unit, based on information obtained from the navigation device indicating that the second lane exists, and then displays an image of a moving indicator object indicating that the moving body is moving into the second lane on the display unit, or displays an image of the moving indicator object on the display unit at the same time as the image of the object of the second lane is displayed.

[0013] (8) A control method according to another aspect of the present invention involves a computer performing the following steps: a process of recognizing the surrounding conditions of a moving body; a process of displaying lanes included in the surrounding conditions on a display unit; a process of displaying an image on the display unit in which, if the target path of the moving body is on the side of a lane adjacent to the first lane on which the moving body is traveling, an image of an object representing movement of the moving body toward the lane is superimposed on an image of an object in the adjacent lane; and a process of displaying an image of an object in the second lane on the display unit in which, if the adjacent lane is a second lane that connects to and increases from the first lane ahead of the direction of travel of the moving body beyond the position on which the moving body is traveling, even if the second lane is not recognized in the recognition process when displaying the image of the object representing movement on the display unit.

[0014] (9): A program according to another aspect of the present invention is a program that causes a computer to perform the following: a process of recognizing the surrounding conditions of a moving body; a process of displaying the lanes included in the surrounding conditions on a display unit; a process of displaying on the display unit an image in which an image of an object representing movement of the moving body toward the lane is superimposed on an image of an object in the adjacent lane when the target path of the moving body is on the side of the lane adjacent to the first lane on which the moving body is traveling; and a process of displaying on the display unit an image of an object in the second lane when the recognition process does not recognize the second lane when displaying the image of the object representing movement of the moving body on the display unit when the recognition process does not recognize the second lane when displaying the image of the object representing movement of the moving body on the display unit. [Effects of the Invention]

[0015] According to the embodiments of (1)-(9), user convenience can be improved. By displaying the image of the object in the second lane in accordance with the timing at which the image of the moving display object is displayed, it is possible to suppress the feeling of incongruity that occupants may experience when the image of the moving display object is displayed but the image of the object in the destination lane is not displayed.

[0016] According to the aspect of (2), based on the road information, the image of the object in the second lane can be more reliably displayed.

[0017] According to the aspect of (3), since the display of the image of the object in the moving display and the image of the object in the second lane are displayed simultaneously in conjunction with each other, it is possible to more reliably suppress giving a sense of discomfort to the occupant. For example, it is possible to suppress the display of only the image of the object in the second lane or only the image of the object in the moving display.

[0018] According to the aspect of (4), after the image of the object in the second lane is displayed, the image of the object in the moving display is displayed, so that the sense of discomfort for the occupant is further reduced.

[0019] According to the aspect of (5), when the lane change is automatically performed, since the display of the image of the object in the moving display in the direction where the second lane does not exist is suppressed, the lane change is automatically performed with the occupant feeling at ease.

[0020] According to the aspect of (6), when the vehicle changes lanes, appropriate information can be provided to the occupant according to the change in the position of the vehicle.

Brief Description of the Drawings

[0021] [Figure 1] It is a configuration diagram of a vehicle system 1 using a vehicle control system according to an embodiment. [Figure 2] It is a diagram (Part 1) for explaining a comparative example. [Figure 3] It is a diagram (Part 2) for explaining a comparative example. [Figure 4] It is a diagram for explaining the image displayed on the display unit of the embodiment. [Figure 5] It is a flowchart showing an example of the process flow executed by the driving support device 100.

Mode for Carrying Out the Invention

[0022] [Overall Configuration] FIG. 1 is a configuration diagram of a vehicle system 1 using a vehicle control system according to an embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled or other vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or discharge power of a secondary battery or a fuel cell. Although this embodiment will be described as being applied to a vehicle, it may be applied to other moving bodies instead of the vehicle.

[0023] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, a HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU 60, an operator 80, a direction indicator 90, a driving support device 100, a traveling driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. The driving support device 100 is an example of a “control device”.

[0024] Camera 10 is a digital camera that utilizes a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle (hereinafter referred to as vehicle M) on which the vehicle system 1 is installed. When imaging the area in front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, etc. Camera 10 periodically and repeatedly images the area around vehicle M. Camera 10 may also be a stereo camera.

[0025] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to determine at least the position (distance and bearing) of the object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of the object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0026] LIDAR14 irradiates light (or electromagnetic waves with a wavelength close to light) around vehicle M and measures the scattered light. Based on the time from emission to reception, LIDAR14 detects the distance to the target. The irradiated light is, for example, pulsed laser light. LIDAR14 can be attached to any location on vehicle M.

[0027] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 to recognize the position, type, speed, etc., of an object. The object recognition device 16 outputs the recognition results to the driver assistance device 100. The object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the driver assistance device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0028] The communication device 20 communicates with other vehicles in the vicinity of vehicle M, or with various server devices via a wireless base station, for example, by using a cellular network, Wi-Fi network, Bluetooth®, DSRC (Dedicated Short Range Communication), etc.

[0029] The HMI30 presents various information to the occupants of vehicle M and accepts input operations from the occupants. The HMI30 includes various display devices, speakers, buzzers, touch panels, switches, keys, etc. The HMI30 is equipped with a display device. The display device is a display device, also known as a multi-information display, that displays various information in vehicle M, such as a speedometer showing the vehicle's speed or a tachometer showing the rotational speed of the internal combustion engine in vehicle M, and is located in the center of the instrument panel of vehicle M.

[0030] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity around the vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0031] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, speakers, a touch panel, keys, etc. The navigation HMI 52 may be partially or completely shared with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter referred to as the route on the map) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52, by referring to the first map information 54. The first map information 54 is, for example, information in which the road shape is represented by links indicating roads and nodes connected by those links. The first map information 54 may also include information such as road curvature and POI (Point of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be implemented, for example, by the functions of a terminal device such as a smartphone or tablet held by an occupant. The navigation device 50 may transmit the current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0032] The MPU 60 includes, for example, a recommended lane determination unit 61 and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the map route provided by the navigation device 50 into multiple blocks (for example, every 100m with respect to the vehicle's direction of travel) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 makes decisions such as which lane from the left the vehicle should travel in. If there is a branching point on the map route, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branching point. For example, if the vehicle M reaches a predetermined distance before a branching road it is traveling on, the recommended lane determination unit 61 determines the lane connecting to the branching road as the recommended lane. The recommended lane determination unit 61 and the second map information 62 may be functional units or information included in other devices such as the driver assistance device 100.

[0033] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, etc. The second map information 62 may be updated as needed by the communication device 20 communicating with other devices.

[0034] The control elements 80 include, for example, the steering wheel 82, as well as the accelerator pedal, brake pedal, shift lever, and other control elements. The control elements 80 are equipped with sensors that detect the amount of operation or whether or not an operation is performed, and the detection results are output to the driver assistance device 100, or to some or all of the driving force output device 200, brake device 210, and steering device 220. The steering wheel 82 does not necessarily have to be annular, and may take the form of an irregularly shaped steering wheel, joystick, buttons, etc. The control elements 80 include a first control element 84. The turn signal 90 lights up or turns off in response to the operation of the first control element 84.

[0035] The first operator 84 is, for example, a turn signal lever switch. For example, when the driver operates the first operator 84, the turn signal 90 illuminates in response to the operation. When the driver performs a predetermined operation on the first operator 84, the control unit 150 activates the ALC (Auto Lane Change) function and causes the vehicle M to perform an automatic lane change. The predetermined operation is an operation that triggers the activation of the ALC function. The predetermined operation is, for example, operating the turn signal lever switch in the direction in which the lane change is desired for a predetermined time, or pushing the turn signal lever switch to a predetermined position. More specifically, the predetermined operation is to operate the turn signal lever switch in the direction in which the lane change is desired and maintain a predetermined position for a predetermined time. The first operator 84 may be in the form of a button or other means instead of a turn signal lever switch. The ALC function may also be activated by operating on a predetermined button or other operator.

[0036] The driver assistance device 100 includes, for example, a recognition unit 110 and a control unit 150. The recognition unit 110 and the control unit 150 are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), or SOC (System On Chip), or by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transient storage medium) such as the HDD or flash memory of the driver assistance device 100, or it may be stored in a removable storage medium such as a DVD or CD-ROM and installed in the HDD or flash memory of the driver assistance device 100 when the storage medium (non-transient storage medium) is mounted on a drive device.

[0037] The recognition unit 110 recognizes the position and state of objects around the vehicle M, such as their speed and acceleration, based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive axis) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or it may be represented by a region. The "state" of an object may include the object's acceleration, jerk, or "action state" (for example, whether or not it is changing lanes or attempting to change lanes).

[0038] The recognition unit 110 recognizes, for example, the lane in which the vehicle M is traveling. For example, the recognition unit 110 recognizes the driving lane by comparing the pattern of road markings (for example, an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings around the vehicle M recognized from the image captured by the camera 10. The recognition unit 110 may also recognize the driving lane by recognizing not only road markings, but also road boundaries (road boundaries) including road markings, shoulders, curbs, median strips, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results by INS may also be taken into consideration. The recognition unit 110 recognizes stop lines, obstacles, red lights, toll booths, and other road events.

[0039] When recognizing a driving lane, the recognition unit 110 recognizes the position and orientation of the vehicle M relative to the driving lane. For example, the recognition unit 110 may recognize the deviation of the vehicle M's reference point from the center of the lane, and the angle the vehicle M makes with a line connecting the centers of the lanes in the direction of travel, as the relative position and orientation of the vehicle M relative to the driving lane. Alternatively, the recognition unit 110 may recognize the position of the vehicle M's reference point relative to any side edge of the driving lane (road marking or road boundary), etc., as the relative position of the vehicle M relative to the driving lane.

[0040] The control unit 150 performs driver assistance control. For example, the control unit 150 automatically controls the driving force output device 200 and the brake device 210 without relying on the driver's operation to automatically control the speed of the vehicle M. The control unit 150 also performs so-called ACC (Adaptive Cruise Control). The control unit 150 controls the vehicle M to travel at a set speed, or to follow a preceding vehicle at a predetermined distance from the preceding vehicle.

[0041] The control unit 150 controls the steering device 220 to prevent the vehicle M from deviating from its lane. For example, the control unit 150 controls the steering device 220 so that the vehicle M travels in the center or near the center of the lane recognized by the recognition unit 110. Hereinafter, this control may be referred to as "lane keeping control". The control unit 150 performs both hands-on lane keeping control and hands-off lane keeping control.

[0042] Hands-on lane keeping control is a control system that is performed when the driver is holding the steering wheel (when the steering grip sensor, not shown, detects that the driver is gripping the steering wheel). The conditions under which hands-on lane keeping control can be performed are less stringent than the conditions under which hands-off lane keeping control can be performed.

[0043] Hands-off lane keeping control is a control system that is performed when the driver is not holding the steering wheel (when the steering grip sensor, not shown, does not detect that the driver is holding the steering wheel). Hands-off lane keeping control can be performed, for example, when the following conditions are met: the speed of vehicle M is above a predetermined speed, vehicle M is traveling on a predetermined road (for example, a road or type of road that has been set in advance as being capable of performing hands-off lane keeping control), and the driver is monitoring the road ahead. Hands-off lane keeping control is performed when the driver is monitoring the road ahead, and is not performed or is stopped when the driver is not monitoring the road ahead.

[0044] The conditions under which hands-on lane keeping control and hands-off lane keeping control can be performed, as described above, are examples, and other conditions (for example, that vehicle M is following the vehicle in front) may be included, or some conditions may be omitted. The conditions under which hands-on lane keeping control can be performed are less stringent than the conditions under which hands-off lane keeping control can be performed (and the conditions under which hands-off lane keeping control can be performed are stricter than the conditions under which hands-on lane keeping control can be performed). The driver assistance device 100 recognizes whether the driver is monitoring the road ahead based on images captured by a camera (not shown) that captures images of the driver.

[0045] The control unit 150 includes a processing unit 152 and a support control unit 154 (lane change control unit). The control unit 150 automatically changes the vehicle M's lane. Details of these will be described later. The control for automatic lane changes may be conditional on the execution of hands-off lane keeping control or hands-on lane keeping control.

[0046] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle M to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU that controls them. The ECU controls the above configuration according to information input from the driver assistance device 100 or from the operator 80.

[0047] The braking system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the driver assistance device 100 or from the control element 80, so that brake torque corresponding to the braking operation is output to each wheel.

[0048] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the driver assistance device 100 or from the control element 80.

[0049] [Comparative Example] Figure 2 is a diagram (part 1) illustrating a comparative example. For example, vehicle M is traveling in the first lane L1. Vehicle M is scheduled to change lanes to the second lane L2, which is adjacent to the first lane L1. The second lane L2 is a branching lane that branches off from the first lane L1. For example, the following control is executed when the target route for vehicle M is set to travel in the second lane L2. This control is also executed when the occupant (driver) of vehicle M performs an operation to instruct vehicle M to change lanes to the second lane L2 at a predetermined distance before the connection point where the first lane L1 and the second lane L2 connect. The operation to instruct is any operation, such as operating a predetermined lane change reservation button or operating the first operator 84. For example, by operating the reservation button, branching support control is executed, in which a lane change to the branching lane is performed automatically.

[0050] For example, when the support control unit 154 approaches a branching lane, it automatically decelerates, illuminates the turn signal 90 at the appropriate time, and automatically controls the steering to change the vehicle M into the branching lane without driver intervention. This support control may be performed, for example, when hands-off lane keeping control or hands-on lane keeping control is in operation.

[0051] The processing unit 152 displays the lane lines included in the recognition result of the recognition unit 110 (for example, the image captured by the camera 10) on the display unit of the HMI 30. The display unit is, for example, a display unit located in a position easily visible to the driver. The processing unit 152 converts the environment and objects around the vehicle M, such as the lane lines and the vehicle in front recognized by the recognition unit 110, into an image, and displays the converted image of the environment and objects on the display unit. The display unit is, for example, the meter display of the HMI 30, the multi-information display, or a display unit located in a position easily visible to the driver.

[0052] Time T is when vehicle M reaches the first predetermined distance before the connection point (the position at time T+3). Time T+1 is when vehicle M reaches the second predetermined distance before the connection point. The second predetermined distance is shorter than the first predetermined distance. Time T+2 is when vehicle M reaches the third predetermined distance before the connection point. The third predetermined distance is shorter than the second predetermined distance. At time T+2, turn signal 90 is illuminated. Time T+3 is when the lane change to the second lane L2 begins (or just before it begins).

[0053] From time T to time T+1, the driver assistance system 100 begins guiding vehicle M to change lanes to the destination lane. From time T+1 to time T+2, the driver assistance system 100 slows vehicle M down in advance to change lanes to the branching road.

[0054] Before time T, image IM1 is displayed on the display. Between time T and time T+1, image IM2 is displayed on the display, followed by image IM3. Between time T+1 and time T+2, images IM4 and IM5 are displayed on the display in that order. From time T+2 onward, images IM6, IM7, and IM8 are displayed on the display in that order.

[0055] The vehicle shown in the image is an example of an object image for vehicle M. The lane the vehicle is traveling in is an example of an object image for lane 1, L1. The lane adjacent to the lane the vehicle is traveling in (the lane adjacent to the right) is an example of an object image for lane 2, L2. The arrows extending from the vehicle in image IM3 (image IM3# described later) or image IM5 (image IM5# described later) are examples of objects for movement display.

[0056] Image IM1 includes information indicating the first lane L1, the third lane L3, vehicle M (the vehicle at the bottom of the image), and the future trajectory of vehicle M, as recognized by the recognition unit 110. The third lane L3 is the lane adjacent to the first lane L1. When image IM1 is displayed, navigation linkage is in place. Navigation linkage means that the processing unit 152 uses information from the navigation device 50 to display an image on the display unit, or that it is possible to display an image on the display unit.

[0057] Image IM2 is an image displayed using navigation integration. Images IM3-IM10, described later, are similar. When the navigation device 50 approaches the position where vehicle M will change lanes, the processing unit 152 displays information on the display unit indicating that vehicle M will change lanes. Image IM2 includes, for example, an arrow indicating the direction of the lane change. After image IM2 is displayed, when vehicle M approaches the connection point, the processing unit 152 displays image IM3 on the display unit. Image IM3 is an image that includes information (arrow) indicating the direction in which vehicle M will change lanes, associated with vehicle M in image IM1.

[0058] Image IM4 includes information indicating the direction in which vehicle M will change lanes and information indicating that vehicle M will decelerate in order to exit into a branching lane. This allows the driver to recognize that vehicle M is decelerating. Image IM5 is similar to image IM3.

[0059] Image IM6 includes the information from images IM3 and IM6, as well as an image of the second lane L2 recognized by the recognition unit 110. As vehicle M approaches the second lane L2, the recognition unit 110 becomes able to recognize the second lane L2, and therefore the image of the second lane L2 is displayed on the display unit.

[0060] Image IM7 shows the vehicle M's turn signal illuminated in Image IM6. Image IM8 shows the vehicle M's trajectory moving from the first lane L1 to the second lane L2 in Image IM7.

[0061] As described above, the section up to time T+2, and from time T+4 onwards (described later), is a period in which the crew can selectively change the image displayed on the display unit. From time T+2 to time T+4, the crew can select and display content (pre-set content) different from the content shown in Figure 2 or Figure 3 on the display unit.

[0062] Figure 3 is a diagram (part 2) illustrating a comparative example. Figure 3 shows the scene from time T+2 onwards, as shown in Figure 2. At time T+3, image IM8 is displayed. At this time, or thereafter, vehicle M begins to change lanes. At time T+4, when vehicle M changes lanes and moves to the second lane L2, image IM9 is displayed. Image IM9 is the same image as image IM1. At time T+5, an image similar to image IM9, or image IM10, is displayed. Image IM10 is an image that includes the lane in which vehicle M is traveling and the vehicle in front. After image IM10 is displayed, when vehicle M approaches a position where the curvature of the second lane L2 increases, image IM11 is displayed. Image IM11 contains information indicating that acceleration suppression of vehicle M is being performed according to the curvature, and information indicating that acceleration suppression can be released by access operation.

[0063] In the aforementioned images IM3 or IM5, the lateral movement display is shown in an area where no lane markings exist. This can cause discomfort to the occupants. In contrast, in this embodiment, the discomfort caused to occupants can be suppressed by superimposing the lateral movement display onto the lane markings as shown below. This improves user convenience.

[0064] [Display in this embodiment] The processing unit 152 displays the lanes included in the surrounding conditions of vehicle M recognized by the recognition unit 110 on the display unit. If the target path of vehicle M is on the side of a lane adjacent to the first lane in which vehicle M is traveling, the processing unit 152 displays on the display unit an image on which an image of an object representing movement indicating vehicle M's movement toward the adjacent lane is superimposed on an image of an object representing an adjacent lane.

[0065] When the recognition unit 110 does not recognize the second lane, the processing unit 152 displays the image of the object in the second lane on the display unit, even if the recognition unit 110 does not recognize the second lane when displaying the image of the moving display object on the display unit, if the adjacent lane is a second lane that connects to and increases from the first lane ahead of the direction of travel of the vehicle M.

[0066] For example, even if the recognition unit 110 cannot recognize the second lane because the vehicle M is located a predetermined distance before the starting point of the second lane, the processing unit 152 will display an image on the display unit that includes an image of the vehicle M object, an image of the second lane object, and an image of a moving display object associated with the image of the vehicle M object and the image of the second lane object.

[0067] The processing unit 152 identifies the presence of the second lane in the direction of travel based on road information and displays the second lane on the display unit. Road information includes, for example, information provided by the navigation device 50, and map information including road information (location information of branch roads) stored in the memory unit or provided by communication.

[0068] The processing unit 152 may display the image of the second lane object and the image of the moving display object on the display unit simultaneously, or it may display the image of the second lane object on the display unit first, and then display the image of the moving display object on the display unit.

[0069] When the occupant of vehicle M has instructed the support control unit 154 to automatically change lanes, even if the second lane is not included in the lanes recognized by the recognition unit 110 when the vehicle reaches a position of a first distance from the second lane, the processing unit 152 displays an image of the second lane object on the display unit, and then displays an image of a movement indicator object on the display unit, indicating that vehicle M is moving into the second lane, based on information obtained from the navigation device 50 indicating that the second lane exists, or displays an image of a movement indicator object on the display unit at the same time as the image of the second lane object is displayed. "When the vehicle reaches a position of a first distance from the second lane" refers to the timing when image IM3# or image IM5# in Figure 4, which will be described later, is displayed.

[0070] Figure 4 is a diagram illustrating the images displayed on the display unit of the embodiment. The same explanation as in Figure 2 or Figure 3 is omitted. In this embodiment, image IM3# is displayed instead of image IM3, and image IM5# is displayed instead of image IM5.

[0071] Image IM3# displays, for example, information from Image IM3, as well as images of objects in the second lane L2 and objects of road markings Ld in the second lane L2. The image of the second lane L2 is not the second lane L2 recognized by the recognition unit 110, but rather information obtained by the processing unit 152 in cooperation with the navigation device 50 from the first map information 54 (or information obtained from the map information in the storage unit). Even if the recognition unit 110 has not recognized the second lane L2, the processing unit 152 displays the image of the second lane L2 based on the timing for displaying the movement display.

[0072] As described above, the processing unit 152 displays the second lane L2, which the recognition unit 110 has not recognized, on the display unit at an appropriate time, thereby suppressing the display of moving indicators in areas of the image where no lanes exist. This reduces the feeling of discomfort for the occupants of vehicle M and improves the convenience for the occupants of vehicle M.

[0073] For example, the above process may be performed when the vehicle driver manually changes lanes, or it may be applied when the vehicle M automatically changes lanes. For example, the process of this embodiment may be applied when automatic lane change control is performed, in which the vehicle M automatically decelerates along the target path, the turn signal 90 is illuminated, and the support control unit 154 automatically changes lanes to control the vehicle M. In this case, when automatic lane change control is performed, the processing unit 152 superimposes the image of the second lane object and the image of the moving display object and displays them on the display unit. This prevents the image of the moving display object from being displayed in a direction where there is no second lane when the lane change is performed automatically, thereby improving the sense of security for the occupants.

[0074] [flowchart] Figure 5 is a flowchart showing an example of the processing flow performed by the driver assistance device 100. First, the processing unit 152 acquires the location information of the vehicle M (step S100). Next, the processing unit 152 acquires navigation information (step S102). The navigation information includes information about the destination set in the navigation device 50, information about the target route for the vehicle M guided by the navigation device 50, and the structure of the roads around the target route. The processing in steps S100 and S102 is repeatedly performed at predetermined intervals.

[0075] Next, the processing unit 152 determines whether the target route is set for the branch road based on the information obtained in steps S100 and S102 (step S104). If the target route is not set for the branch road, the process returns to step S100. If the target route is set for the branch road, the processing unit 152 determines whether the vehicle M has reached a predetermined distance before the branch road which is the target route (step S106). If the vehicle M has not reached a predetermined distance before the branch road which is the target route, the process returns to step S100.

[0076] When vehicle M reaches a predetermined distance before the target road (junction), the processing unit 152 displays an image on the display unit corresponding to the position of vehicle M relative to the position of the junction (step S108). For example, the processing unit 152 displays the image as explained in Figures 2-4. At the timing of displaying the moving display on the display unit, the processing unit 152 uses the junction (junction lane) information from the navigation information to display the moving display along with the image of the second lane L2.

[0077] Next, the processing unit 152 determines whether the recognition unit 110 has recognized the branch road (step S110). If the recognition unit 110 has not recognized the branch road, the process returns to step S108. If the recognition unit 110 has recognized the branch road, the processing unit 152 displays an image based on the recognition result of the recognition unit 110 (step S112). For example, the processing unit 152 displays a moving image and also displays the branch road information included in the recognition result on the display unit.

[0078] Next, the processing unit 152 determines whether predetermined conditions are met (step S114). These predetermined conditions include that vehicle M has moved to the branching road (moved to the vicinity of the center of the branching road), that the lane change to the branching road has been completed, or that vehicle M has traveled a predetermined distance after moving to the branching road or after changing lanes. If the predetermined conditions are not met, the process returns to step S112. If the predetermined conditions are met, the processing unit 152 clears the movement indicator (step S116). This completes the processing of one routine in this flowchart.

[0079] Instead of performing the process in step S110 described above, it is possible to determine whether or not a condition has been met. The condition is any condition, such as whether vehicle M has reached a predetermined position or whether the state of vehicle M has met a condition. The state of vehicle M having met a condition means, for example, whether or not the predetermined condition described above has been met or whether or not any other condition has been met, such as whether or not the turn signal has been turned off.

[0080] As described above, the driver assistance device 100 can improve user convenience by using navigation information to display branching roads in the image according to the movement display.

[0081] According to the embodiments described above, when the driving assistance device 100 displays an image of an object in the moving display on the display unit, even if the recognition unit 110 does not recognize the second lane when the lane adjacent to the first lane is a second lane that connects to the first lane and increases in the direction of travel of the vehicle M, the user convenience can be improved by displaying an image of an object in the second lane on the display unit.

[0082] The embodiments described above can be expressed as follows. A memory device that stores the program, Equipped with a hardware processor, The hardware processor executes the program stored in the memory device, Recognize the surrounding conditions of the moving object, The lanes included in the surrounding conditions are displayed on the display unit. If the target path of the moving object is on the side of the lane adjacent to the first lane on which the moving object is traveling, the display unit will display an image in which an image of an object representing movement of the moving object toward the lane is superimposed on an image of an object in the adjacent lane. If the adjacent lane is a second lane that connects to and increases in the direction of travel of the moving body, ahead of the position where the moving body is traveling, Even if the recognition unit does not recognize the second lane when displaying the image of the moving object on the display unit, the image of the object in the second lane will be displayed on the display unit. A control device configured in such a way.

[0083] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0084] 1. Vehicle System 10 Cameras 80 Operators 84 1st controller 90 Turn signal 100 Driving support devices 110 Recognition part 150 Control Unit 152 Processing Unit 154 Support Control Unit

Claims

1. A recognition unit that recognizes the surrounding conditions of a moving object, The recognition unit converts the lanes included in the surrounding conditions recognized by the recognition unit into an image, and displays the converted image on the display unit. The system includes a control unit that, when the target path of the moving body is on the side of a lane adjacent to the first lane on which the moving body is traveling, displays on the display unit an image in which an image of an object representing movement of the moving body toward the lane is superimposed on an image of an object in the adjacent lane. If the adjacent lane is a second lane that connects to and increases in the direction of travel of the moving body, ahead of the position where the moving body is traveling, The control unit displays an image of the object of the moving display on the display unit even if the recognition unit does not recognize the second lane. Control device.

2. The control unit, based on road information, identifies that the second lane is located in the direction of travel of the moving object and causes the display unit to display an image of the object in the second lane. The control device according to claim 1.

3. The control unit causes the display unit to simultaneously display the image of the object in the second lane and the image of the moving display object. The control device according to claim 1.

4. The control unit displays the image of the object in the second lane on the display unit, and then displays the image of the moving display object on the display unit. The control device according to claim 1.

5. The system further includes an automatic lane change control unit that performs automatic lane change control to automatically change the moving body into the second lane, The control unit, when the automatic lane change control is being performed, superimposes the image of the object in the second lane and the image of the moving display object and displays them on the display unit. The control device according to claim 1.

6. The control unit, Even if the recognition unit cannot recognize the second lane because the moving body is located a predetermined distance before the starting point of the second lane, if the moving body is planning to change lanes into the second lane and is notifying the occupants of the moving body of the lane change, the display unit will display an image including a first image of the moving body's object, a second image of the second lane's object, and a third image of the moving display object associated with the first and second images. The control device according to claim 1.

7. The control unit, The image of the lane object recognized by the recognition unit is displayed on the display unit. When the occupant of the moving vehicle gives an instruction to the automatic lane change control unit that automatically changes the lane of the moving vehicle to automatically change lanes, Even if the second lane is not included in the lanes recognized by the recognition unit when the vehicle reaches a position at a first distance from the second lane, Based on the information obtained from the navigation system indicating the existence of the second lane, After displaying an image of the object in the second lane on the display unit, display an image of a moving object indicating that the moving body is moving into the second lane on the display unit, or The image of the moving display object is displayed on the display unit at the timing when the image of the object in the second lane is displayed. The control device according to claim 6.

8. Computers Processing to recognize the surrounding environment of a moving object, The process involves converting the lanes included in the surrounding conditions into an image and displaying the converted image on the display unit. If the target path of the moving object is on the side of the lane adjacent to the first lane on which the moving object is traveling, the process of displaying an image on the display unit in which an image of an object representing movement of the moving object toward the lane is superimposed on an image of an object in the adjacent lane, If the adjacent lane is a second lane that connects to and increases in the direction of travel of the moving body, ahead of the position where the moving body is traveling, Even if the recognition process does not recognize the second lane when displaying the image of the moving display object on the display unit, the process of displaying the image of the second lane object on the display unit is performed. A control method for executing this.

9. On the computer, Processing to recognize the surrounding environment of a moving object, The process involves converting the lanes included in the surrounding conditions into an image and displaying the converted image on the display unit. If the target path of the moving object is on the side of the lane adjacent to the first lane on which the moving object is traveling, the process of displaying an image on the display unit in which an image of an object representing movement of the moving object toward the lane is superimposed on an image of an object in the adjacent lane, If the adjacent lane is a second lane that connects to and increases in the direction of travel of the moving body, ahead of the position where the moving body is traveling, Even if the recognition process does not recognize the second lane when displaying the image of the moving display object on the display unit, the process of displaying the image of the second lane object on the display unit is performed. A program to execute.

Images