Control device, control method, and storage medium

Abstract

The invention provides a control device, a control method, and a storage medium capable of improving convenience for an occupant of a moving body. A control device is provided with: a travel control unit that controls the travel of a moving body without depending on the operation of an occupant of the moving body; and a display control unit that displays, on a target display unit, a screen selected by the occupant, from among a first display screen that is a display relating to the control state of the travel control unit and a second display screen that is not a display relating to the control state of the travel control unit. When the display control unit selects that the passenger displays the second display screen on the display unit, the display control unit determines that the travel control unit actually controls the travel of the moving body, and displays the second display screen on the display unit. The first display screen is displayed on the display unit by switching the second display screen to the first display screen without depending on a selection operation by the occupant.

Description

Technical Field

[0001] The present invention relates to a control device, a control method and a storage medium. Background Art

[0002] In recent years, efforts to provide sustainable transportation systems that take into account various situations have intensified. To achieve this, research and development efforts are underway to further improve traffic safety and convenience through the development of driving support technologies. A vehicle guidance display device has been disclosed that displays an image of the road ahead of the vehicle on a display and creates guidance arrows at branch points, superimposing them on the road ahead (Japanese Patent Application Laid-Open No. 10-287177). Summary of the Invention

[0003] Conventional technologies do not disclose the display according to the travel of the moving object, and thus the convenience for the passengers of the moving object may be low.

[0004] The present invention was developed in consideration of this situation, and one of its objectives is to provide a control device, a control method, and a storage medium that can improve the convenience for occupants (e.g., drivers) of a mobile vehicle, thereby contributing to the development of a sustainable transportation system.

[0005] Solutions to Problems

[0006] The control device, control method, and storage medium according to the present invention employ the following structures.

[0007] (1): A control device according to one embodiment of the present invention comprises: a driving control unit for controlling the driving of a mobile body without depending on the operation of a passenger of the mobile body; and a display control unit for causing a screen selected by the passenger from among a first display screen and a second display screen to be displayed on a display unit of an object, wherein the first display screen is a display related to the control state of the driving control unit, and the second display screen is not a display related to the control state of the driving control unit. When the passenger selects to display the second display screen on the display unit, the display control unit switches the second display screen to the first display screen without depending on the selection operation of the passenger and causes the first display screen to be displayed on the display unit if it is determined that the driving control unit actually controls the driving of the mobile body.

[0008] (2): In the embodiment of (1) above, the display related to the control state is a display related to automatic lane change control in which the moving object automatically changes lanes.

[0009] (3): Based on the scheme of (2) above, the automatic lane change control is executed on a branch road connected to the first lane in which the mobile body is traveling, and the display control unit performs the following processing: when the mobile body is executing deceleration control for lane change based on the automatic lane change control in front of the branch road, the second display screen is displayed on the display unit; and after the deceleration control, the first display screen is displayed on the display unit.

[0010] (4): Based on the solution in (1) above, the second display screen includes a navigation screen indicating the direction of travel of the mobile body or the direction of the destination of the mobile body.

[0011] (5): Based on the above-mentioned solution (1), the second display screen includes a screen showing the fuel consumption or power consumption of the mobile object.

[0012] (6): Based on the solution of (5) above, when the display control unit determines that the driving control unit actually controls the driving of the mobile body, the display control unit switches the second display screen to the first display screen without relying on the passenger's selection operation and displays the first display screen on the display unit. When the driving control performed by the driving control unit is completed and the mobile body becomes a state of steady-state movement, the first display screen is switched back to the second display screen and displays the second display screen on the display unit.

[0013] (7): Based on the solution of (2) above, when the display control unit determines that the driving control unit executes the automatic lane change control, the display control unit switches the second display screen to the first display screen without relying on the occupant's selection operation and displays the first display screen on the display unit; when the automatic lane change control is completed, the display control unit switches the first display screen back to the second display screen and displays the second display screen on the display unit.

[0014] (8): The control method involved in other schemes of the present invention causes a computer to execute: a process of controlling the travel of the mobile body without relying on the operation of the passenger of the mobile body; a process of displaying the screen selected by the passenger from a first display screen and a second display screen on a display unit of an object, wherein the first display screen is a display related to the control state of the process of controlling the travel of the mobile body, and the second display screen is not a display related to the control state; and when the passenger selects to display the second display screen on the display unit, in the case where it is determined that the travel of the mobile body is controlled by actually controlling the process of the travel, the second display screen is switched to the first display screen without relying on the selection operation of the passenger and the first display screen is displayed on the display unit.

[0015] (9): Another embodiment of the present invention relates to a storage medium storing a program, wherein the program is used to cause a computer to execute: a process of controlling the travel of a mobile body without depending on the operation of a passenger of the mobile body; a process of displaying a screen selected by the passenger from a first display screen and a second display screen on a display unit of an object, wherein the first display screen is a display related to a control state of the process of controlling the travel of the mobile body, and the second display screen is not a display related to the control state; and a process of switching the second display screen to the first display screen without depending on the passenger's selection operation to display the first display screen on the display unit when the passenger selects to display the second display screen on the display unit and, in the case where it is determined that the travel of the mobile body is controlled by actually controlling the process of the travel, displaying the first display screen on the display unit.

[0016] Effects of the Invention

[0017] According to the schemes (1) to (9), user convenience can be improved. When the first screen display and the second screen display can be switched, the first display screen related to the control is displayed according to the timing of the control that actually affects the travel of the mobile body. Therefore, the passenger can understand what control is being performed and for what reason when the travel actually occurs or before the timing.

[0018] According to the aspect (2), when the automatic lane change control is performed at a branch road or the like, the occupant can grasp the timing of the lane change actually performed by the automatic lane change control by the driving control unit.

[0019] According to the solution (3), the first screen display is performed after deceleration before the branch. The timing of performing the first screen display is before the lateral movement after the braking control, so that the occupant can understand the lane change before the lateral movement.

[0020] According to the scheme (4), even if the navigation screen is selected, it is switched to the first screen display related to the driving control before the driving control that actually affects the driving of the mobile body occurs, so the occupant can understand what kind of driving control is being performed and for what reason before the driving of the mobile body actually occurs.

[0021] According to the scheme (5), even if a screen related to fuel consumption or power consumption is selected, it is switched to the first screen display related to driving control before the driving control that actually affects the driving of the mobile body occurs, so the occupants can understand what kind of driving control is being performed and for what reason before the driving of the mobile body actually occurs.

[0022] According to the scheme (6), when the lane change is completed, the display is quickly switched back to the second screen, thereby reducing the discomfort caused to the occupants by the display.

[0023] According to the aspect (7), when the lane change to the adjacent lane is completed, the display is switched back to the selected second screen, thereby preventing the occupants from feeling uncomfortable with the display. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a configuration diagram of a vehicle system using the vehicle control system according to the embodiment.

[0025] Figure 2 This is a diagram (part 1) for explaining the processing of this embodiment.

[0026] Figure 3 This is a diagram (part 2) for explaining the processing of this embodiment.

[0027] Figure 4 This is a flowchart showing an example of the flow of processing executed by the driving support device.

[0028] Figure 5 This is a diagram for explaining a scenario in which a vehicle changes lanes. DETAILED DESCRIPTION

[0029] [Overall structure]

[0030] Figure 1 This is a structural diagram of a vehicle system 1 utilizing a vehicle control system according to an embodiment. The vehicle equipped with vehicle system 1 is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its driving source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator coupled to the internal combustion engine, or power discharged from a secondary battery or fuel cell. While this embodiment is described as being applied to a vehicle, it can also be applied to other mobile objects instead of vehicles.

[0031] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) device 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, vehicle sensors 40, a navigation device 50, an MPU 60, an operating element 80, a direction indicator 90, a driving support device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected via multiplexed communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, and wireless communication networks. Figure 1 The illustrated configuration is merely an example, and a portion of the configuration may be omitted, or another configuration may be added. The driving support device 100 is an example of a "control device."

[0032] The camera 10 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is mounted anywhere on the vehicle (hereinafter referred to as the vehicle M) equipped with the vehicle system 1. To image the forward direction, the camera 10 is mounted on the upper portion of the windshield, behind the rearview mirror, or elsewhere. For example, the camera 10 periodically and repeatedly captures images of the surroundings of the vehicle M. The camera 10 may also be a stereo camera.

[0033] The radar device 12 radiates radio waves, such as millimeter waves, around the vehicle M and detects the radio waves (reflected waves) reflected by objects to detect at least the object's position (range and direction). The radar device 12 can be mounted anywhere on the vehicle M. The radar device 12 can also detect the object's position and velocity using the FM-CW (Frequency Modulated Continuous Wave) method.

[0034] LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to light) around the vehicle M and measures the scattered light. LIDAR 14 detects the distance to an object based on the time between light emission and light reception. The irradiated light is, for example, pulsed laser light. LIDAR 14 can be mounted anywhere on the vehicle M.

[0035] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, the radar device 12, and the LIDAR 14 to identify the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the driving support device 100. The object recognition device 16 may also output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the driving support device 100. The object recognition device 16 may also be omitted from the vehicle system 1.

[0036] The communication device 20 communicates with other vehicles around the vehicle M using, for example, a cellular network, Wi-Fi network, Bluetooth (registered trademark), or DSRC (Dedicated Short Range Communication), or communicates with various server devices via a wireless base station.

[0037] The HMI 30 presents various information to the occupants of the vehicle M and accepts input operations from the occupants. The HMI 30 includes various display devices, speakers, a buzzer, a touch panel, switches, buttons, and the like. The HMI 30 includes a display device. For example, the display device is located in the center of the instrument panel of the vehicle M and is a display device that displays various information about the vehicle M, such as a speedometer (speedometer) indicating the vehicle M's travel speed or a rotational speed meter (tachometer) indicating the rotational speed (rotational speed) of the internal combustion engine of the vehicle M. This is a so-called multi-information display.

[0038] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity about a vertical axis, an azimuth sensor that detects the orientation of the vehicle M, and the like.

[0039] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may also be determined or supplemented by an INS (Inertial Navigation System) utilizing the output of the vehicle sensors 40. The navigation HMI 52 includes a display, speakers, a touch panel, keys, etc. The navigation HMI 52 may partially or entirely be shared with the HMI 30 described above. The route determination unit 53, for example, refers to the first map information 54 to determine a route (hereinafter referred to as a "mapped route") from the position of the vehicle M determined by the GNSS receiver 51 (or an input arbitrary position) to a destination input by the occupant using the navigation HMI 52. The first map information 54 is information representing the shape of the road, for example, by representing road links and nodes connected by the links. The first map information 54 may also include road curvature, POI (Point of Interest) information, and the like. The route on the map is output to the MPU 60. The navigation device 50 may also provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may also be implemented as a function of a terminal device such as a smartphone or tablet computer held by the passenger. The navigation device 50 may also transmit the current location and destination to a navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0040] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as a HDD or flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into multiple blocks (for example, every 100 meters in the vehicle's travel direction) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines the lane number from the left to be traveled. If the route on the map branches, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to the branch destination. For example, if the vehicle M reaches a position a predetermined distance to the front of the branch road, the recommended lane determination unit 61 determines a lane connected to the branch road as the recommended lane. The recommended lane determination unit 61 and the second map information 62 may also be functional units or information included in other devices such as the driving support device 100.

[0041] The second map information 62 is higher-precision map information than the first map information 54. The second map information 62 includes, for example, information about lane centers or lane boundaries. The second map information 62 may include road information, traffic restriction information, address information (address, postal code), facility information, and telephone number information. The second map information 62 can be updated at any time by communicating with other devices via the communication device 20.

[0042] The operating elements 80 include, for example, an accelerator pedal, a brake pedal, a shift lever, and other operating elements in addition to the steering wheel 82. Sensors are mounted on the operating elements 80 to detect the amount of operation or the presence or absence of an operation. These sensors output the detection results to the driving support device 100, or to some or all of the driving force output device 200, the braking device 210, and the steering device 220. The steering wheel 82 need not necessarily be annular; it may also be a specially shaped steering wheel, a joystick, or a button. The operating elements 80 include a first operating element 84. The direction indicator 90 illuminates or extinguishes in response to the operation of the first operating element 84.

[0043] The first operating element 84 is, for example, a turn signal lever switch. For example, when the driver operates the first operating element 84, the direction indicator 90 illuminates in accordance with the operation. When the driver performs a predetermined operation on the first operating element 84, the control unit 150 activates the ALC (Auto Lane Change) function, causing the vehicle M to automatically change lanes. The predetermined operation is an operation that triggers the activation of the ALC function. Examples of predetermined operations include operating the turn signal lever switch in the desired lane change direction for a predetermined time or pressing the turn signal lever switch to a predetermined position. More specifically, the predetermined operation involves maintaining the turn signal lever switch in the desired lane change direction and a predetermined position for a predetermined time. The first operating element 84 may also be in another form, such as a button, instead of a turn signal lever switch. Furthermore, the ALC function may be activated by operating another operating element, such as a predetermined button.

[0044] The driving support device 100 includes, for example, a recognition unit 110 and a control unit 150. The recognition unit 110 and the control unit 150 are implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented using hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or a SoC (System on Chip), or through a combination of software and hardware. The program may be pre-stored in a storage device (including a non-transitory storage medium) such as an HDD or flash memory of the driving support device 100, or may be stored on a removable storage medium such as a DVD or CD-ROM and installed in the HDD or flash memory of the driving support device 100 by attaching the storage medium (non-transitory storage medium) to a drive device.

[0045] The recognition unit 110 identifies the position, velocity, acceleration, and other states of objects surrounding the vehicle M based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. The object's position is identified, for example, as an absolute coordinate position with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and used for control purposes. The object's position can be represented by a representative point such as the object's center of gravity or a corner, or by an area. The object's "state" can also include its acceleration, jerk, or "behavior" (e.g., whether it is currently changing lanes or intending to change lanes).

[0046] The recognition unit 110, for example, identifies the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 110 compares the road dividing line pattern (e.g., an arrangement of solid and dashed lines) obtained from the second map information 62 with the road dividing line pattern surrounding the vehicle M identified from the image captured by the camera 10 to identify the driving lane. The recognition unit 110 is not limited to identifying road dividing lines; it can also identify driving lanes by identifying road boundaries (road boundaries), including road dividing lines, shoulders, curbs, medians, guardrails, and the like. This recognition may also take into account the position of the vehicle M obtained from the navigation device 50 and the results of processing by the INS. The recognition unit 110 identifies stop signs, obstacles, red lights, toll booths, and other road features.

[0047] When identifying a driving lane, the recognition unit 110 identifies the position and posture of the vehicle M relative to the driving lane. For example, the recognition unit 110 may identify the deviation of the vehicle M's reference point from the lane center and the angle formed by the vehicle M's travel direction with respect to a line connecting the lane centers as the relative position and posture of the vehicle M relative to the driving lane. Alternatively, the recognition unit 110 may identify the position of the vehicle M's reference point relative to either side of the driving lane (a road dividing line or a road boundary) as the relative position of the vehicle M relative to the driving lane.

[0048] The control unit 150 performs driving support control. For example, the control unit 150 automatically controls the driving force output device 200 and the braking device 210 to automatically control the speed of the vehicle M, independent of the driver's operation. The control unit 150 performs so-called Adaptive Cruise Control (ACC). The control unit 150 controls the vehicle M so that it travels at a set speed and follows the preceding vehicle at a predetermined distance.

[0049] The control unit 150 controls the steering device 220 to prevent the vehicle M from leaving its lane. For example, the control unit 150 controls the steering device 220 to keep the vehicle M in the center or near the center of the lane identified by the recognition unit 110. This control is sometimes referred to as "lane keeping control" hereinafter. The control unit 150 performs both hands-on lane keeping control and hands-off lane keeping control.

[0050] Hands-on lane keeping control is executed when the driver is holding the steering wheel (a steering wheel grip sensor (not shown) detects the driver's grip). The conditions for executing hands-on lane keeping control are less stringent than those for executing hands-off lane keeping control.

[0051] Hands-off lane keeping control is executed when the driver is not gripping the steering wheel (when the steering wheel grip sensor (not shown) is not detecting the steering wheel grip). Hands-off lane keeping control can be executed, for example, when the following conditions are met: the vehicle M's speed is above a predetermined speed; the vehicle M is traveling on a predetermined road (e.g., a road or road type pre-defined as one capable of hands-off lane keeping control); and the driver is monitoring the road ahead. Hands-off lane keeping control is executed when the driver is monitoring the road ahead; otherwise, it is not executed or stopped.

[0052] The above-mentioned conditions for enabling hands-on lane keeping control and hands-off lane keeping control are merely examples and may include other conditions (e.g., vehicle M following a preceding vehicle) or omit some conditions. The conditions for enabling hands-on lane keeping control may be less stringent than those for enabling hands-off lane keeping control (or more stringent than those for enabling hands-off lane keeping control). The driving support device 100 identifies whether the driver is monitoring the road ahead based on images captured by a camera (not shown) that captures the driver.

[0053] The control unit 150 includes a processing unit 152 (display control unit) and an assistance control unit 154 (travel control unit). The processing unit 152 displays various images on the display unit. The assistance control unit 154 automatically causes the vehicle M to change lanes. Automatic lane change control can be conditional on the execution of hands-free lane keeping control or hands-on lane keeping control.

[0054] The driving force output device 200 outputs the driving force (torque) for driving the vehicle M to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU that controls these components. The ECU controls the aforementioned components based on information input from the driving support device 100 or information input from the operating element 80.

[0055] The brake system 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a brake ECU. The brake ECU controls the electric motor based on information input from the driving support device 100 or information input from the operating element 80, thereby outputting a braking torque corresponding to the braking operation to each wheel.

[0056] The steering system 220 includes, for example, a steering ECU and an electric motor. The electric motor applies force to, for example, a rack-and-pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor based on information input from the driving support device 100 or information input from the operating element 80 to change the direction of the steering wheel.

[0057] [Display Control]

[0058] Figure 2Figure 1 is used to illustrate the processing of this embodiment. For example, vehicle M is traveling in first lane L1. Vehicle M intends to change lanes to second lane L2, which is adjacent to first lane L1. Second lane L2 is a branch lane that branches off from first lane L1. For example, the following control is executed when the target path is set with vehicle M traveling in second lane L2. This control is executed when a passenger (driver) of vehicle M performs an operation instructing vehicle M to change lanes to second lane L2 from a position a predetermined distance ahead of the connection point between first lane L1 and second lane L2. The instructing operation refers to any operation such as operating a predetermined lane change reservation button or operating first operating element 84. For example, operation of the reservation button executes branch support control in which a lane change to the branch lane is automatically performed.

[0059] For example, when approaching a branch lane, the assistance control unit 154 automatically decelerates and illuminates the direction indicator 90 at an appropriate time, thereby automatically controlling the steering to change the lane of the vehicle M to the branch lane without relying on the driver's operation. This assistance control may also be performed when hands-free lane keeping control or hands-on lane keeping control is being executed.

[0060] Time T is when vehicle M reaches a position a first predetermined distance ahead of the connecting position. Time T+1 is when vehicle M reaches a second predetermined distance ahead of the connecting position. The second predetermined distance is shorter than the first predetermined distance. Time T+2 is when vehicle M reaches a third predetermined distance ahead of the connecting position. The third predetermined distance is shorter than the second predetermined distance. Time T+2 is when direction indicator 90 illuminates. Time T+3 is when a lane change to the second lane L2 is initiated (or immediately before it is initiated).

[0061] Before time T+2, the selected item can be displayed (or is displayed) on the display unit, and from time T+2 until time T+5 (described later), the dedicated item is displayed on the display unit. After time T+5, the selected item can be displayed (or is displayed) on the display unit.

[0062] An example of a selection item is "a second display screen that is not related to the control state of the driving control unit." An example of a dedicated item is "a first display screen that is related to the control state of the driving control unit." A dedicated item is a screen related to so-called ADAS (Advanced Driver-Assistance Systems). For example, a dedicated item is a display related to the control or function of ADAS. For example, before time T+2 or after time T+5, the processing unit 152 causes the selection item (one or more selection items) and the screen selected by the occupant in the dedicated item to be displayed on the target display unit.

[0063] Selectable items, such as images IM1 and IM2, are displayed separately from the support control unit 154's display of the vehicle M's control status (such as lane change, lane keeping control, and ACC). The occupant can display any of the multiple selectable items on the display unit by performing a predetermined operation. For example, the processing unit 152 displays image IM2 on the display unit instead of image IM1 in response to the occupant's operation. Image IM1 is a navigation screen showing the vehicle M's travel direction or the direction of the vehicle M's destination. Image IM2 is a screen showing the vehicle M's fuel consumption or electricity consumption.

[0064] When the occupant selects to display the selected item on the display unit, the processing unit 152 switches the selected item to a dedicated item and displays the dedicated item on the display unit, independent of the occupant's selection operation, if the processing unit 152 determines that the support control unit 154 is actually controlling the driving of the vehicle M. The timing for "determining that the support control unit 154 is actually controlling the driving of the vehicle M" includes, for example, the timing when the support control unit 154 initiates a lane change of the vehicle M, a predetermined time before the aforementioned initiation timing, a predetermined time after the aforementioned initiation timing, or the timing when the direction indicator 90 is illuminated in response to the initiation of a lane change.

[0065] At time T+2, when vehicle M begins a lane change, processing unit 152 displays a dedicated entry on the display unit instead of a selected entry. Dedicated entries, such as images IM3, IM4, and IM5, are displayed in connection with automatic lane change control, in which vehicle M automatically changes lanes. For example, in the section from time T+1 to time T+2 (the section immediately before the branch road), support control unit 154 decelerates vehicle M to perform a lane change. During this period, processing unit 152 displays the selected entry on the display unit. At time T+2, after deceleration control has been performed, processing unit 152 displays the dedicated entry on the display unit. While this example illustrates the display of the dedicated entry after deceleration control has been performed, the dedicated entry may also be displayed before deceleration control is initiated, or after the timing for a lane change has been determined.

[0066] Image IM3, image IM4, and image IM5 are sequentially displayed on the display unit. Image IM3 includes information representing the first lane L1, the third lane L3 adjacent to the first lane, the second lane L2, a branch lane adjacent to the first lane, the vehicle M (the vehicle at the bottom of the image), information about the future trajectory of the vehicle M, and directional information (arrow) indicating the direction in which the vehicle M will change lanes. The directional information is associated with the vehicle M in image IM3. The first lane L1, the second lane L2, and the third lane L3 are images generated based on the recognition results of the recognition unit 110 (the images captured by the camera 10).

[0067] Image IM4 is displayed with a different color from the color of the direction information in image IM3, indicating that the direction indicator 90 is illuminated in the image of vehicle M. The same applies to other aspects. Image IM5 is an image based on image IM4, showing the vehicle M moving from the first lane L1 to the second lane L2. Image IM5 is displayed on the display before the vehicle M enters the second lane L2 or before the lane change is completed. Completion of the lane change refers to, for example, the vehicle M moving to a branch road (towards the center of the branch road) or traveling a predetermined distance after entering the branch road.

[0068] As described above, the processing unit 152 displays a dedicated entry on the display unit instead of a selection entry when the vehicle M starts a lane change. This allows the occupant to understand what control is being performed and for what reason before or at the actual lane change. As a result, user convenience is improved.

[0069] Figure 3 This is a diagram (part 2) for explaining the processing of this embodiment. Figure 3 Shown Figure 2 The scene shown is from time T+2 onwards. Image IM5 is displayed at time T+3. At this time or thereafter, vehicle M begins a lane change. At time T+4, when vehicle M changes lanes and moves to the second lane L2, image IM6 is displayed. Image IM6 includes information indicating the first lane L1 recognized by recognition unit 110, the second lane L2 recognized by recognition unit 110, vehicle M (the vehicle on the lower side of the image), and the future trajectory of vehicle M.

[0070] While image IM6 is being displayed, navigation cooperation is being performed at time T+2 and before time T+2. Navigation cooperation means that the processing unit 152 uses information from the navigation device 50 (e.g., road information such as lanes) to display or enable the display of an image on the display. This allows the processing unit 152 to determine the timing for displaying a dedicated entry on the display.

[0071] When driving control by the assistance control unit 154 is completed and the vehicle M enters a steady-state state, the processing unit 152 switches the dedicated entry back to the selected entry and displays the selected entry on the display. For example, when automatic lane change control is completed, the processing unit 152 switches the dedicated entry back to the selected entry and displays the selected entry on the display. At time T+5, the lane change is completed. In this case, the processing unit 152 displays the selected entry displayed on the display before time T+2, instead of the dedicated entry.

[0072] As described above, the processing unit 152 quickly switches the display back to the selected item when the lane change is completed, thereby reducing the discomfort felt by the occupant and allowing the occupant to recognize the desired information in the selected item. As a result, user convenience is improved.

[0073] [flow chart]

[0074] Figure 4 This is a flowchart illustrating an example of the process flow executed by the driving support device 100. First, the processing unit 152 obtains the position information of the vehicle M (step S100). Next, the processing unit 152 obtains navigation information (step S102). The navigation information includes information such as the destination set in the navigation device 50, information on the target travel route of the vehicle M being guided by the navigation device 50, and the structure of the roads surrounding the target travel route. The processes of steps S100 and S102 are repeatedly executed at predetermined intervals.

[0075] Next, based on the information acquired in steps S100 and S102, the processing unit 152 determines whether a target travel route has been set for the branch road (step S104). If a target travel route has not been set for the branch road, the process returns to step S100. If a target travel route has been set for the branch road, the processing unit 152 determines whether the vehicle M has reached a position a predetermined distance in front of the branch road serving as the target travel route (step S106). If the vehicle M has not reached a position a predetermined distance in front of the branch road serving as the target travel route, the process returns to step S100. Prior to step S106, the selected item is displayed on the display unit.

[0076] When the vehicle M reaches a position a predetermined distance ahead of the branch road serving as the target travel path, the processing unit 152 switches the selected item to the dedicated item and displays the dedicated item on the display unit (step S108). Next, the processing unit 152 determines whether the lane change is complete (step S110). If the lane change is not complete, the process returns to step S108. If the lane change is complete, the processing unit 152 displays the selected item before switching to the dedicated item on the display unit (step S112). This concludes the processing of one routine in this flowchart.

[0077] Instead of the process of step S110 described above, it is also possible to determine whether a condition is satisfied. A condition may include, for example, the vehicle M arriving at a predetermined location or the vehicle M state satisfying an arbitrary condition. The vehicle M state satisfying an arbitrary condition may include, for example, the direction indicator being off or the predetermined time having passed since the direction indicator was off.

[0078] In the above example, the control in the scenario where the vehicle M travels toward a branch road is described. However, instead of (or in addition to) this, the same control as described above may be performed in the scenario where the vehicle M changes lanes.

[0079] Figure 5 is a diagram for explaining a scenario in which vehicle M changes lanes. For example, at time T+1, if a slow vehicle is present in front of vehicle M, it is determined that a lane change is to be performed. Figure 5 Surrounding vehicles are omitted in the figure. For example, the driver issues an automatic lane change instruction. At time T+2, vehicle M recognizes the conditions of the adjacent lane and determines that a lane change is possible. At time T+3, vehicle M initiates the lane change. At time T+4, the lane change is in progress. At time T+5, the lane change is complete.

[0080] Before time T+2, the selection item is displayed. From time T+2 to time T+5, the dedicated item is displayed. After time T+5, the selection item is displayed. For example, before time T+2 and after time T+5, the image IM2 is displayed as the selection item.

[0081] Furthermore, if the driver selects an image representing the surrounding conditions of vehicle M as a selection item, images IM11, IM12, and IM16 representing the surrounding conditions of vehicle M may be displayed before time T+2 and after time T+5. Image IM11 is an image displayed before time T+1. Image IM11 shows vehicles, road dividing lines, and other objects surrounding vehicle M (other vehicles are omitted from the road diagram shown in the upper portion of the image). Image IM12, in addition to the item in image IM11, also includes information representing the future trajectory of vehicle M and directional information (arrows) indicating the direction in which vehicle M will change lanes. Image IM11, IM12, or IM16 may also be another example of a selection item. For example, images IM11, IM12, IM16, and IM13, IM14, and IM15 are selection items and dedicated items. When images IM11, IM12, and IM16, which are images representing the surrounding conditions, are selected as selection items, images IM13, IM14, and IM15 representing the surrounding conditions corresponding to images IM11, IM12, and IM16 representing the surrounding conditions are directly displayed even during the period when dedicated items are displayed.

[0082] Image IM13 is displayed at time T+2, image IM14 is displayed at time T+3, and image IM15 is displayed at time T+4. Images IM13, IM14, and IM15 contain information about the surrounding area, such as the lane at which the driver is requested to confirm the lane change destination. Image IM13 contains information indicating the future trajectory of vehicle M and directional information (arrow) indicating the direction in which vehicle M will change lanes. The directional information in image IM14 may be displayed in a different color than the directional information in image IM13. The directional information in image IM15 may also be displayed in a different color than the directional information in image IM14. For example, the directional information may be displayed in a manner that further emphasizes the lane change of vehicle M.

[0083] When the lane change is completed at time T+5 and the vehicle M enters a steady-state state, the dedicated item is switched back to the selected item and displayed on the display unit. For example, image IM2 or image IM16 is displayed. Image IM16 is an item showing the surrounding conditions of the vehicle M.

[0084] When the general Figure 4 When the flowchart is applied to lane changes, the determinations in steps S100 and S102 are omitted. The process of step S104 can be read as "the processing unit 152 determines whether the vehicle M plans to change lanes," and the process of step S106 can be read as "the processing unit 152 determines whether the lane change is to be performed."

[0085] As described above, the processing unit 152 displays the dedicated item on the display unit instead of the selected item in accordance with the timing when the vehicle M starts changing lanes, thereby improving the convenience of the user.

[0086] According to the embodiment described above, when the passenger selects to display the selection item (second display screen) on the display unit, the driving support device 100 switches the selection item (second display screen) to the dedicated item (first display screen) without relying on the passenger's selection operation and displays the dedicated item on the display unit when it is determined that the support control unit 154 actually controls the driving of the vehicle M, thereby improving the convenience of the user.

[0087] The above-described embodiment can be expressed as follows.

[0088] The control device is configured to include:

[0089] a storage device storing a program; and

[0090] Hardware processor,

[0091] The control device executes the program stored in the storage device by the hardware processor to:

[0092] a process for controlling the travel of a mobile body independently of an operation by an occupant of the mobile body;

[0093] a process of causing the screen selected by the occupant, of a first display screen and a second display screen, to be displayed on a target display unit, wherein the first display screen is a display related to the control state and the second display screen is not a display related to the control state of the travel control unit; and

[0094] When the passenger chooses to display the second display screen on the display unit, and when it is determined that the travel of the mobile body is actually controlled by the process of controlling the travel, the second display screen is switched to the first display screen without relying on the passenger's selection operation, and the first display screen is displayed on the display unit.

[0095] While specific embodiments of the present invention have been described above, the present invention is not limited to these embodiments at all, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A control device, wherein: The control device comprises: a travel control unit that controls travel of the mobile body independently of an operation by a passenger of the mobile body; and a display control unit that causes the screen selected by the passenger, from among a first display screen and a second display screen, to be displayed on the target display unit, wherein the first display screen is a display related to the control state of the driving control unit, and the second display screen is a display not related to the control state of the driving control unit; When the passenger chooses to display the second display screen on the display unit, the display control unit switches the second display screen to the first display screen and displays the first display screen on the display unit without depending on the passenger's selection operation, if it is determined that the driving control unit actually controls the driving of the mobile body.

2. The control device according to claim 1, wherein: The display related to the control state is a display related to automatic lane change control in which the moving object automatically changes lanes.

3. The control device according to claim 2, wherein: The automatic lane change control is executed on a branch road connected to the first lane in which the vehicle is traveling. The display control unit performs the following processing: displaying the second display screen on the display unit when the mobile body is executing deceleration control for lane change based on the automatic lane change control in front of the branch road; as well as After the deceleration control, the first display screen is displayed on the display unit.

4. The control device according to claim 1, wherein: The second display screen includes a navigation screen indicating a traveling direction of the mobile body or a direction of a destination of the mobile body.

5. The control device according to claim 1, wherein: The second display screen includes a screen showing the fuel consumption or power consumption of the moving object.

6. The control device according to claim 1, wherein: When the display control unit determines that the driving control unit actually controls the driving of the mobile body, the display control unit switches the second display screen to the first display screen without relying on the passenger's selection operation and displays the first display screen on the display unit. Then, when the driving control performed by the driving control unit is completed and the mobile body becomes a state of steady-state movement, the display control unit switches the first display screen back to the second display screen and displays the second display screen on the display unit.

7. The control device according to claim 2, wherein: When the display control unit determines that the driving control unit executes the automatic lane change control, the display control unit switches the second display screen to the first display screen without relying on the occupant's selection operation and displays the first display screen on the display unit. When the automatic lane change control is completed, the display control unit switches the first display screen back to the second display screen and displays the second display screen on the display unit.

8. A control method, wherein: The control method enables the computer to execute: a process for controlling the travel of a mobile body independently of an operation by an occupant of the mobile body; Processing for causing the screen selected by the passenger, from among a first display screen and a second display screen, to be displayed on the target display unit, wherein the first display screen is a display related to a control state of a process for controlling travel of the mobile body, and the second display screen is a display not related to the control state; as well as When the passenger chooses to display the second display screen on the display unit, and when it is determined that the travel of the mobile body is controlled by actually controlling the travel, the second display screen is switched to the first display screen without relying on the passenger's selection operation, and the first display screen is displayed on the display unit.

9. A storage medium storing a program, wherein: The program is used to cause a computer to execute: a process for controlling the travel of a mobile body independently of an operation by an occupant of the mobile body; Processing for causing the screen selected by the passenger, from among a first display screen and a second display screen, to be displayed on the target display unit, wherein the first display screen is a display related to a control state of a process for controlling travel of the mobile body, and the second display screen is a display not related to the control state; as well as When the passenger chooses to display the second display screen on the display unit, and when it is determined that the travel of the mobile body is controlled by actually controlling the travel, the second display screen is switched to the first display screen without relying on the passenger's selection operation, and the first display screen is displayed on the display unit.

Images