Display control device, display control computer program, and display control method

Abstract

To provide a display control device that displays a target icon indicating a closing target according to a situation so as to prevent a driver from being given a wrong thinking for a closed traffic lane.SOLUTION: A display control device includes an acquisition unit that acquires detection information indicating a closing target indicating closing of a traffic lane detected around a vehicle, and operation information indicating an operation by a driver to an operation unit that operates the vehicle, and a display control unit that allows a display unit to display a target icon indicating the closing target. When the detection information is acquired via the acquisition unit and the operation information is not acquired via the acquisition unit, the display control unit allows the display unit to display a first target icon indicating the closing target, and when the detection information is acquired via the acquisition unit and the operation information is acquired via the acquisition unit, the display control unit allows the display unit to display a second target icon indicating the closing target, whose visibility to the driver is lower than that of the first target icon.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a display control device, a display control computer program, and a display control method. [Background technology]

[0002] It is known to display the positions of targets, such as other vehicles, relative to the vehicle detected using a sensor mounted on the vehicle on an in-vehicle display device, thereby notifying the driver of information about targets around the vehicle (see, for example, Patent Document 1).

[0003] Such a display device allows the driver to check the surrounding environment of the vehicle, and determines that the object displayed on the display device is recognized by the vehicle's automatic control system. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-142309 A Summary of the Invention [Problem to be solved by the invention]

[0005] For example, if a closure landmark such as a pylon is detected indicating that a lane is closed due to construction, a landmark icon indicating the closure landmark such as a pylon will be displayed on the display device within the closed lane.

[0006] When a target icon indicating a closed target is displayed on the display device, the driver determines that the closed target is recognized by the automatic control system of the vehicle. Furthermore, the driver may determine that it is not permitted to move the vehicle toward the closed lane because the closed target is recognized by the automatic control system of the vehicle.

[0007] However, in some situations it may be necessary to move the vehicle in the direction of the closed lane, and it would be unwise to give the driver the impression that it is not permitted to move the vehicle in the direction of the closed lane.

[0008] On the other hand, if the visibility of the target icon indicating the closed target displayed on the display device is reduced, the driver will be less likely to be given the impression that moving the vehicle in the direction of the closed lane is not permitted.

[0009] Therefore, an object of the present disclosure is to provide a display control device that displays a target icon representing a closed target depending on the situation so as not to give the driver the wrong idea about a closed lane. [Means for solving the problem]

[0010] According to one embodiment, a display control device is provided, which includes an acquisition unit that acquires detection information representing a closure target indicating that a lane is closed, detected around a vehicle, and operation information representing an operation by a driver on an operation unit that operates a vehicle, and a display control unit that displays a target icon representing the closure target on a display unit, and is characterized in that, when the detection information is acquired via the acquisition unit and the operation information is not acquired via the acquisition unit, the display control unit displays a first target icon representing the closure target on the display unit, and when the detection information is acquired via the acquisition unit and the operation information is acquired via the acquisition unit, the display control unit displays a second target icon representing the closure target, which has a lower visibility to the driver than the first target icon, on the display unit.

[0011] Furthermore, in this display control device, it is preferable that the second target icon being less visible to the driver than the first target icon includes either the second target icon being a color that is less visually prominent to the driver than the first target icon, or the second target icon being more transparent than the first target icon.

[0012] In addition, this display control device preferably has a position determination unit that determines a display position at which to display a landmark icon representing a closure landmark on the display unit based on the detection position of the landmark, and when a lane closure section in which multiple closure landmarks are placed in one lane begins, the position determination unit preferably determines the display positions of the multiple first landmark icons or second landmark icons representing the closure landmarks so that they are displayed at a predetermined interval along the lane dividing line representing the closed lane displayed on the display unit, without being based on the detection position of the closure landmark.

[0013] In addition, in this display control device, it is preferable that the operation information represents operations of an accelerator pedal, a brake pedal, and a steering wheel by the driver.

[0014] According to another embodiment, there is provided a display control computer program, which causes a processor to execute a process including acquiring detection information representing a closure target indicating that a lane is closed, which is detected around a vehicle, and operation information representing an operation by a driver on an operation unit for operating a vehicle, and displaying a target icon representing the closure target on a display unit, wherein, when the detection information is acquired but the operation information is not acquired, a first target icon representing the closure target is displayed on the display unit, and when the detection information is acquired and the operation information is acquired, a second target icon representing the closure target, which has a lower visibility to the driver than the first target icon, is displayed on the display unit.

[0015] According to still another embodiment, there is provided a display control method, which is executed by a display control device, and includes acquiring detection information representing a closure target indicating that a lane is closed, which is detected around a vehicle, and operation information representing an operation by a driver to an operation unit for operating a vehicle, and displaying a target icon representing the closure target on a display unit, wherein, when the detection information is acquired but the operation information is not acquired, a first target icon representing the closure target is displayed on the display unit, and when the detection information is acquired and the operation information is acquired, a second target icon representing the closure target, which has a lower visibility to the driver than the first target icon, is displayed on the display unit. Effect of the Invention

[0016] The display control device according to the present disclosure can display a target icon representing a closed target depending on the situation so as not to give the driver the wrong idea about a closed lane. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram (part 1) for explaining an outline of the operation of the display image generating device of the present embodiment. [Diagram 2] FIG. 2 is a diagram (part 2) for explaining an outline of the operation of the display image generating device according to the present embodiment. [Diagram 3] 1 is a schematic configuration diagram of a vehicle in which a display system according to an embodiment of the present invention is implemented. [Figure 4] 5 is an example of an operational flowchart relating to a display image generating process of the display image generating device of the present embodiment. [Diagram 5] 10 is an example of an operational flowchart relating to an icon selection process of the display image generating device of the present embodiment. [Figure 6] 13 is an example of an operational flowchart relating to a display position determination process in a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] 1 and 2 are diagrams for explaining an outline of the operation of the display image generating device 13 of this embodiment. The display image generating device 13 is an example of a display control device. Hereinafter, an outline of the operation of the display image generating device 13 disclosed in this specification regarding vehicle control processing will be described with reference to FIG.

[0019] 1 and 2, a vehicle 10 is traveling on lane 52 of a road 50 having lanes 51, 52, and 53. Lane 51 is divided by lane dividing line 54 and lane dividing line (lane boundary line) 55, and lane 52 is divided by lane dividing line 55 and lane dividing line 56 (lane boundary line). Furthermore, lane 53 is divided by lane dividing line 56 and lane dividing line 57.

[0020] The vehicle 10 has an automatic control system (not shown) that drives the vehicle 10. The vehicle 10 can be driven by the vehicle 10 itself. The driver can also drive the vehicle 10 by operating operation units (not shown) such as a steering wheel, a brake pedal, or an accelerator pedal. The vehicle 10 can also assist the driver in driving the vehicle 10 when the driver is driving the vehicle 10.

[0021] Even when the vehicle 10 is being driven by the driver, the driver can intervene in the driving of the vehicle 10 by operating an operating part such as the steering wheel, the brake pedal, or the accelerator pedal.

[0022] The vehicle 10 has a display device 7a and a display image generating device 13. The display image generating device 13 displays targets around the vehicle 10 detected by a sensor (not shown) mounted on the vehicle 10 on the display device 7a so that the driver can confirm that information used to control the vehicle 10 is correct.

[0023] 1 and 2, a sensor (not shown) mounted on the vehicle 10 detects another vehicle 30 and a closure object 60 indicating that the lane 53 is closed. The vehicle 30 is traveling in the lane 51. The closure objects 60 are disposed on the right side of the lane dividing line 56, indicating that the lane 53 is closed. Examples of the closure objects 60 include pylons, cones, etc.

[0024] Since the closure target 60 indicating that the lane 53 is closed has been detected and the driver has not operated the steering wheel, the brake pedal, the accelerator pedal, or other operating parts (not shown), the display image generating device 13 selects the first closure target icon representing the closure target as the icon to be displayed on the display device 7a. When the closure target 60 is a pylon, an amber or red icon can be used as the first closure target icon.

[0025] As shown in FIG. 1, the display image generating device 13 generates a display image 40 showing lane markings 54 to 57, a vehicle 10, a vehicle 30, and a closure target 60, and displays the display image 40 on a display device 7a.

[0026] Displayed in the display image 40 are lane markings 54a, 55a, 56a, and 57a representing lane markings 54, 55, 56, and 57, a vehicle icon 10a representing a vehicle 10, a vehicle icon 30a representing a vehicle 30, and a first closure target icon 60a.

[0027] The vehicle icon 10a is displayed between the demarcation line 55a and the demarcation line 56a at the bottom of the display image 40. The vehicle icon 30a is displayed between the demarcation line 54a and the demarcation line 55a at the top of the display image 40.

[0028] The multiple first closure target icons 60a indicate that the lane 52 in which the vehicle 10 is traveling and the adjacent lane 53 are closed. When the first closure target icon 60a is displayed on the display device 7a, the driver determines that the closure target 60 is recognized by the vehicle 10. Furthermore, the driver may determine that since the closure target 60 is recognized by the vehicle 10, it is not permitted to move the vehicle 10 toward the closed lane. The vehicle 10 does not drive in such a way that the vehicle 10 moves into the closed lane 53, but does not perform any special control with respect to the closed lane 53.

[0029] On the other hand, when a closure target 60 indicating that the lane 53 is closed is detected and the driver operates an operating unit (not shown) such as the steering wheel, the brake pedal, or the accelerator pedal, the display image generating device 13 selects a second closure target icon representing a closure target having a lower visibility to the driver than the first closure target icon as an icon to be displayed on the display device 7a. When the closure target 60 is a pylon, a gray icon can be used as the second closure target icon.

[0030] As shown in FIG. 2, the display image generating device 13 generates a display image 40 representing lane markings 54 to 57, a vehicle 10, a vehicle 30, and a closure target 60, and displays the display image 40 on the display device 7a.

[0031] The display image 40 of the display device 7a displays lane markings 54a, 55a, 56a, and 57a representing lane markings 54, 55, 56, and 57, a vehicle icon 10a representing a vehicle 10, a vehicle icon 30a representing a vehicle 30, and a second closure target icon 60b.

[0032] By reducing the visibility of the closure target icon indicating the closure target displayed on the display device 7a, it should be possible to lessen the driver's impression that moving the vehicle 10 in the direction of the closed lane is not permitted.

[0033] As described above, when the closure target 60 indicating that the lane 53 is closed is detected and an operation of the operation unit by the driver is detected, the display image generating device 13 displays the second target icon 60b representing the closure target 60 having low visibility to the driver. When the driver operates the vehicle 10, the display image generating device 13 can inform the driver that the closure target has a low weighting as information used for driving the vehicle 10. This makes it possible for the display image generating device 13 to prevent the driver from getting the wrong idea that the lane 53 is closed.

[0034] 2 is a schematic configuration diagram of a vehicle 10 in which the display system 1 of the present embodiment is implemented. The vehicle 10 includes a camera 2, a LiDAR sensor 3, a torque sensor 4, a pedal sensor 5, a positioning information receiver 6, a user interface (UI) 7, a map information storage device 11, an object detection device 12, a display image generation device 13, and the like. Furthermore, the vehicle 10 may include a distance measurement sensor (not shown) such as a millimeter wave radar for measuring distances to objects around the vehicle 10. The display system 1 includes at least the UI 7 and the display image generation device 13.

[0035] The camera 2, the LiDAR sensor 3, the torque sensor 4, the pedal sensor 5, the positioning information receiver 6, the UI 7, the map information storage device 11, the object detection device 12, and the display image generating device 13 are communicatively connected via an in-vehicle network 14 that complies with a standard such as a controller area network.

[0036] The camera 2 is an example of an imaging unit provided on the vehicle 10. The camera 2 is attached to the vehicle 10 so as to face the front of the vehicle 10. The camera 2 captures a camera image showing the environment of a predetermined area in front of the vehicle 10 at a camera image capture time that is set, for example, at a predetermined cycle. The camera image may show a road included in the predetermined area in front of the vehicle 10 and road features such as lane markings on the road surface. The camera image also shows other vehicles and pylons indicating lane closures. The camera 2 has a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that forms an image of the area to be captured on the two-dimensional detector. The camera image is an example of surrounding environment information showing the environment around the vehicle.

[0037] Every time the camera 2 captures an image, it outputs the image and the time the image was captured to the object detection device 12 or the like via the in-vehicle network 14. The camera image is used by the object detection device 12 in a process of detecting other objects around the vehicle 10.

[0038] The LiDAR sensor 3 is attached to, for example, the outer surface of the vehicle 10 so as to face the front of the vehicle 10. The LiDAR sensor 3 emits a laser to scan the front of the vehicle 10 at a reflected wave information acquisition time set at a predetermined cycle, and receives a reflected wave reflected by a reflecting object. The time required for the reflected wave to return contains distance information between the vehicle 10 and another object located in the direction of the radar irradiation. The LiDAR sensor 3 outputs reflected wave information including the radar irradiation direction and the time required for the reflected wave to return, together with the reflected wave information acquisition time when the radar was emitted, to the object detection device 12 via the in-vehicle network 14. The reflected wave information is used in the object detection device 12 to detect other objects around the vehicle 10. In this embodiment, the reflected wave information acquisition time is synchronized with the camera image capture time. The reflected wave information is an example of surrounding environment information that represents the environment around the vehicle.

[0039] The torque sensor 4 detects the force that rotates the rotation shaft of the steering wheel (not shown) generated by the driver's operation of the steering wheel, and outputs a torque detection signal representing the rotating force to the display image generating device 13, etc. via the in-vehicle network 14. The torque sensor 4 does not generate a torque detection signal when the driver is not operating the steering wheel. The torque sensor 4 is an example of a detection unit that detects the driver's operation of an operating unit that operates the vehicle 10. The torque detection signal is an example of operation information that represents the driver's operation of the operating unit.

[0040] The pedal sensor 5 detects the operation of the brake pedal or accelerator pedal (not shown) by the driver, and outputs a brake pedal detection signal or an accelerator pedal detection signal indicating the amount of operation to the display image generating device 13, etc. via the in-vehicle network 14. The pedal sensor 5 does not generate a detection signal when the driver is not operating the brake pedal or accelerator pedal. The pedal sensor 5 is an example of a detection unit that detects the operation by the driver of an operating unit that operates the vehicle 10. The brake pedal detection signal and the accelerator pedal detection signal are examples of operation information indicating the operation of the operating unit by the driver.

[0041] The positioning information receiver 6 outputs positioning information indicating the current position of the vehicle 10. For example, the positioning information receiver 6 may be a GNSS receiver. Every time the positioning information receiver 6 acquires positioning information at a predetermined reception cycle, the positioning information receiver 6 outputs the positioning information and the positioning information acquisition time when the positioning information was acquired to the map information storage device 11 or the like.

[0042] The UI 7 is controlled by the display image generating device 13 and the like to notify the driver of information used to control the vehicle 10. Examples of information used to control the vehicle 10 include information on other vehicles around the vehicle 10 and target information such as pylons indicating lane closure. The UI 7 has a display device 7a such as a liquid crystal display or a touch panel to display information. The UI 7 may also have an audio output device (not shown) to notify the driver of information. The UI 7 may also notify the driver of information other than information used to control the vehicle 10 (for example, information related to the travel of the vehicle 10, such as a travel route).

[0043] The map information storage device 11 stores map information of a relatively wide area (for example, a range of 10 to 30 km square) including the current position of the vehicle 10. This map information has high-precision map information including three-dimensional information of the road surface, road speed limits, road curvature, road features such as lane markings on the road, and information indicating the types and positions of structures.

[0044] The map information storage device 11 receives wide-area map information from an external server via a base station by wireless communication via a wireless communication device (not shown) mounted on the vehicle 10 according to the current position of the vehicle 10, and stores the information in the storage device. Every time positioning information is input from the positioning information receiver 6, the map information storage device 11 refers to the stored wide-area map information, and outputs map information of a relatively small area (for example, a range of 100 m square to 10 km square) including the current position represented by the positioning information to the object detection device 12, the display image generation device 13, etc. via the in-vehicle network 14.

[0045] The object detection device 12 detects targets and their types around the vehicle 10 based on a camera image. The targets include road features, moving objects such as pedestrians and other vehicles moving around the vehicle 10, and pylons indicating lane closure. The object detection device 12 has, for example, a classifier that identifies targets shown in an image by inputting a camera image. For example, a deep neural network (DNN) that has been trained in advance to detect targets shown in an input image from the image can be used as the classifier. The object detection device 12 may use a classifier other than a DNN. For example, the object detection device 12 may use, as the classifier, a support vector machine (SVM) that has been trained in advance to input a feature amount (e.g., Histograms of Oriented Gradients, HOG) calculated from a window set on an image and output a confidence level that the target to be detected is shown in the window. Alternatively, the object detection device 12 may detect a target area by performing template matching between a template in which the target to be detected is shown and the image.

[0046] The object detection device 12 may also detect targets around the vehicle 10 based on the reflected wave information. The object detection device 12 may also determine the direction of the target relative to the vehicle 10 based on the position of the target in the camera image, and determine the distance between the target and the vehicle 10 based on the direction and the reflected wave information output by the LiDAR sensor 3. The object detection device 12 estimates the position of the target, for example, expressed in a vehicle coordinate system having an origin on the vehicle 10, based on the current position of the vehicle 10 and the distance and direction to the target relative to the vehicle 10. For example, the center of gravity of an area representing the target can be used as the position of the target. Alternatively, the position of the target may be determined by offsetting the position of the center of gravity of the area representing the target in the traveling direction of the vehicle 10 or in a direction perpendicular to the traveling direction. The object detection device 12 outputs target detection information including information indicating the type of the detected target (for example, a pedestrian, another vehicle, or a pylon) and information indicating its position to the display image generation device 13, etc. The target detection information is an example of detection information that indicates a closing target detected around the vehicle 10, which indicates that a lane is to be closed.

[0047] The object detection device 12 may detect targets based on reflected wave information without using camera images. In this case, the object detection device 12 has, for example, a classifier that inputs the reflected wave information to identify targets represented in the reflected wave information. In this way, the object detection device 12 estimates the type of target and the positions of other objects based on the reflected wave information.

[0048] The display image generating device 13 executes a display control process, an icon selection process, and a position determination process. To this end, the display image generating device 13 has a communication interface (IF) 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 are connected via a signal line 24. The communication IF 21 has an interface circuit for connecting the display image generating device 13 to the in-vehicle network 14. The communication IF 21 is an example of an acquisition unit.

[0049] The memory 22 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 stores computer programs of applications used in the information processing executed by the processor 23 and various data.

[0050] All or part of the functions of the display image generating device 13 are functional modules realized by, for example, a computer program running on the processor 23. The processor 23 has a display control unit 231, an icon selection unit 232, and a position determination unit 233. Alternatively, the functional modules of the processor 23 may be dedicated electronic circuits provided in the processor 23. The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may further have other arithmetic circuits such as a logic arithmetic unit, a numerical arithmetic unit, or a graphic processing unit.

[0051] The map information storage device 11, the object detection device 12, and the display image generation device 13 are, for example, an electronic control unit (ECU). In Fig. 3, the map information storage device 11, the object detection device 12, and the display image generation device 13 are described as separate devices, but all or some of these devices may be configured as a single device.

[0052] Fig. 4 is an example of an operation flowchart relating to the display image generation process of the display image generation device 13 of this embodiment. The display image generation process of the display image generation device 13 will be described below with reference to Fig. 4. The display image generation device 13 executes the display image generation process in accordance with the operation flowchart shown in Fig. 3 at a display image generation time having a predetermined cycle. The cycle in which the display image generation process is executed can be determined by the relationship between the cycle of the camera image capture time and the reflected wave information acquisition time.

[0053] First, the display control unit 231 generates a display image including lane markings included in a camera image captured by the camera 2 (step S101). The display control unit 231 acquires the positions of the lane markings in the area represented in the camera image at the position of the vehicle 10 at the time the camera image was captured. In the map information, the area represented in the camera image can be calculated based on internal parameters such as installation position information such as the optical axis direction and mounting position of the camera 2, and the focal length and angle of view of the imaging optical system.

[0054] The display control unit 231 expresses the coordinates of the lane markings of the area shown in the camera image expressed in the world coordinate system in a vehicle coordinate system having an origin on the vehicle 10. For example, the vehicle coordinate system has an origin at the midpoint between the left and right rear wheels of the vehicle 10 and a point on the ground, a zv axis representing the traveling direction of the vehicle 10, an xv axis representing a direction perpendicular to the zv axis and parallel to the ground, and a yv axis representing the vertical direction. Such coordinate transformation can be expressed by rotation and translation of coordinates.

[0055] The display control unit 231 generates a projection image by projecting the lane markings expressed in the vehicle coordinate system onto a predetermined projection plane perpendicular to the zv axis, and generates a display image including the lane markings. This predetermined projection plane may be disposed near the driver's seat of the vehicle 10.

[0056] The display control unit 231 may generate a display image including lane markings based on lane markings identified from the camera image.

[0057] Then, the display control unit 231 places a vehicle icon representing the vehicle 10 in the lane in which the vehicle 10 is located in the display image. The display control unit 231 selects the lane in which the vehicle 10 is located based on the current position of the vehicle 10 and the map information. The display control unit 231 places the vehicle icon representing the vehicle 10 in the lane in which the vehicle 10 is located, below the display image.

[0058] Next, the position determination unit 233 determines whether or not a target other than a lane marking has been detected based on the target detection information (step S102). If target detection information representing a target other than a lane marking has been input, the position determination unit 233 determines that a target has been detected. On the other hand, if target detection information representing a target other than a lane marking has not been input, the position determination unit 233 determines that a target has not been detected.

[0059] If a target has been detected (step S102-Yes), the icon selection unit 232 selects a target icon representing the target (step S103). The icon selection process for selecting the target icon will be described later.

[0060] Next, the position determination unit 233 determines a display position at which the target icon is displayed on the display device 7a (step S104). The position determination unit 233 projects the detection position of the target represented in the vehicle coordinate system onto the above-mentioned projection plane, and determines the display position of the target icon within the display image.

[0061] Next, the display control unit 231 arranges the target icon at a display position within the display image (step S105). The size of the target icon is preferably changed according to the distance between the current position of the vehicle 10 and the target. In this case, the size of the target icon is determined so that the closer the distance between the current position of the vehicle 10 and the target, the larger the target icon.

[0062] When target detection information representing a plurality of targets is input to the display image generating device 13, the above-mentioned processes in steps S103 to S105 are performed for each target.

[0063] Next, the display control unit 231 displays the display image on the display device 7a (step S106), and ends the series of processes.

[0064] On the other hand, if no object other than the lane markings is detected (step S102-No), the display control unit 231 displays a display image including the lane markings on the display device 7a (step S106), and ends the series of processes.

[0065] 5 is an example of an operation flowchart relating to the icon selection process of the display image generating device of this embodiment. Next, the icon selection process of the display image generating device 13 will be described below with reference to FIG.

[0066] First, the icon selection unit 232 determines whether or not the target is a closure target indicating that a lane is closed based on the type of the target included in the target detection information (step S201). Examples of the closure target include a pylon, a cone bar, and the like.

[0067] If the target object is a lane closure target indicating that a lane is closed (step S201-Yes), the icon selection unit 232 determines whether or not operation information indicating an operation by the driver to an operation unit that operates the vehicle 10 is acquired via the communication IF 21 (step S202). Examples of the operation unit include a steering wheel, a brake pedal, and an accelerator pedal, which are not shown. The driver's operation includes operations to the steering wheel, the accelerator pedal, and the brake pedal.

[0068] When any one of the torque detection signal, the brake pedal detection signal, and the accelerator pedal detection signal has been acquired, the icon selection unit 232 determines that operation information has been acquired.

[0069] On the other hand, when none of the torque detection signal, the brake pedal detection signal, and the accelerator pedal detection signal has been acquired, the icon selection unit 232 determines that no operation information has been acquired.

[0070] If the operation information has not been acquired (step S202-No), the icon selection unit 232 selects a first closure target icon representing a closure target as an icon to be displayed on the display device 7a (step S203), and ends the series of processes. If the closure target is a pylon, an amber or red icon can be used as the first closure target icon. Target icons representing various targets including closure targets are stored in advance in the memory 22. The target icon is selected according to the type of target. For example, a normal car and a large car such as a truck are generated as target icons representing different vehicles.

[0071] On the other hand, if operation information has been input (step S202-Yes), the icon selection unit 232 selects a second closure target icon representing a closure target having a lower visibility to the driver than the first closure target icon as an icon to be displayed on the display device 7a (step S204), and ends the series of processes.

[0072] The second occlusion target icon being less visible to the driver than the first occlusion target icon may include the second occlusion target icon being a color that is less visually prominent to the driver than the first occlusion target icon, or the second occlusion target icon being more transparent than the first occlusion target icon. Also, the second occlusion target icon being less visible to the driver than the first occlusion target icon may include the second occlusion target icon being smaller than the first occlusion target icon.

[0073] The icon selection unit 232 may select a second occlusion target icon having the same shape as the first occlusion target icon and a color that is less visually emphasized to the driver than the first occlusion target icon. For example, when the first occlusion target icon is amber or red, a gray icon can be used as the second occlusion target icon.

[0074] The icon selection unit 232 may also select a second obstruction target icon that has the same shape as the first obstruction target icon and is more transparent than the first obstruction target icon. For example, the icon selection unit 232 may select a second obstruction target icon that is more than 50% transparent, preferably more than 70% transparent, relative to the first obstruction target icon. The icon selection unit 232 may select a transparent obstruction target icon so that the second obstruction target icon is not substantially displayed on the display device 7a.

[0075] Also, if the target object is not a closure target object indicating that a lane is closed (step S201-No), the icon selection unit 232 selects a target object icon corresponding to the type of target object from among the target object icons stored in the memory 22 (step S205), and ends the series of processes.

[0076] Next, a process in which the display image generating device 13 generates a display image including a closed target will be described below with reference to FIGS.

[0077] In the example shown in Figures 1 and 2, another vehicle 30 and a closure object 60 indicating that lane 53 is closed are detected. Vehicle 30 is traveling in lane 54. A plurality of closure objects 60 are disposed to the right of lane dividing line 56, indicating that lane 53 is closed. Examples of closure objects 60 include pylons, cones, etc.

[0078] A closure target 60 indicating that the lane 53 is closed is detected, but operation information indicating an operation of an operating unit such as a steering wheel, a brake pedal, or an accelerator pedal by a driver is not acquired. Therefore, the display image generating device 13 selects a first closure target icon indicating the closure target as an icon to be displayed on the display device 7a. When the closure target 60 is a pylon, an amber or red icon can be used as the first closure target icon.

[0079] It is preferable that the color of the first obstruction target icon be close to the color of the obstruction target 60 as seen by the driver, in order to ensure that the information displayed on the display device 7a does not differ from the surrounding environment of the vehicle as seen by the driver.

[0080] As shown in FIG. 1, the display image generating device 13 generates a display image 40 showing lane markings 54 to 57, a vehicle 10, a vehicle 30, and a closure target 60, and displays the display image 40 on a display device 7a.

[0081] The display image 40 of the display device 7a displays lane markings 54a, 55a, 56a, and 57a representing lane markings 54, 55, 56, and 57, a vehicle icon 10a representing a vehicle 10, a vehicle icon 30a representing a vehicle 30, and a first closure target icon 60a.

[0082] The multiple first closure target icons 60a indicate that the lane 52 in which the vehicle 10 is traveling and the adjacent lane 53 are closed. The driver can confirm that the display of the first closure target icon 60a indicating that the lane 53 is closed on the display device 7a does not differ from the actual field of vision.

[0083] When the first closure target icon 60a is displayed on the display device 7a, the driver determines that the closure target 60 is recognized by the vehicle 10. Furthermore, the driver may determine that since the closure target 60 is recognized by the vehicle 10, it is not permitted to move the vehicle 10 toward the closed lane. The vehicle 10 does not drive in such a way that the vehicle 10 moves into the closed lane 53, but does not perform any special control with respect to the closed lane 53.

[0084] On the other hand, in the example shown in FIG. 2, a closure target 60 indicating that the lane 53 is closed is detected, and operation information indicating an operation of an operating unit such as a steering wheel, a brake pedal, or an accelerator pedal by a driver is acquired. Therefore, the display image generating device 13 selects a second closure target icon, which has the same shape as the first closure target icon and indicates a closure target having a lower visibility to the driver than the first closure target icon, as an icon to be displayed on the display device 7a. When the closure target 60 is a pylon, a gray icon can be used as the second closure target icon. Also, the second closure target icon may be one with high transparency.

[0085] In this way, when the driver is operating the operating unit, the second obstruction target icon representing the obstruction target 60 is made less visible to the driver, thereby preventing the driver from recognizing that the obstruction target 60 is being used to control the vehicle 10.

[0086] In the judgment of step S202 described above, even if the steering wheel operation steers the vehicle 10 toward the opposite side of the closed lane, the steering direction is not important because it is not known whether the driver actually intends to move the vehicle 10 toward the opposite side of the closed lane.

[0087] In addition, the purpose of displaying the second obstruction target icon on the display device 7a is to inform the driver that obstruction targets are given low weight as information used for driving the vehicle 10, so the operation information also includes the driver's operation of the brake pedal.

[0088] On the other hand, when the driver is operating the operating unit and the first closure target icon 60a is displayed on the display device 7a, the driver may think that the closure target is not low-weighted as information used for driving the vehicle 10.

[0089] For example, even when the driver is primarily or partially driving the vehicle 10, it is possible for the driver to move the vehicle 10 into a closed lane 53 by operating the steering wheel.

[0090] However, if the first obstruction target icon 60a is displayed on the display device 7a when the driver is operating the operating unit, the driver may think that such an operation is not permitted by the vehicle 10.

[0091] Therefore, as described above, when a closure target indicating that a lane is closed is detected and an operation of the operation unit by the driver is detected, the display image generating device of this embodiment displays a second target icon representing a closure target with low visibility to the driver. When the driver operates the vehicle, the display image generating device informs the driver that the closure target has a low weight as information used for driving the vehicle. This makes it possible for the display image generating device to prevent the driver from getting the wrong idea that a lane is closed.

[0092] Next, a modified example of the display image generating device of the above-mentioned embodiment will be described below with reference to Fig. 6. Fig. 6 is an example of an operational flowchart relating to a display position determination process in the modified example of the display image generating device.

[0093] The detection position of a closure target by the camera 2 or the LiDAR sensor 3 mounted on the vehicle 10 may not be stable. The reason for this is that the detection accuracy of closure targets such as pylons and cone bars is low.

[0094] Therefore, when the display position of the closure target icon representing the closure target is determined on the display device 7a based on the detection position of the closure target, the display position of the closure target icon representing the closure target may fluctuate. If the display position of the closure target icon on the display device 7a fluctuates, there is a risk that the driver's visibility of the surroundings of the vehicle 10 may decrease.

[0095] In this modified example, the display position determination process for determining the display position of the target icon in step S104 in FIG. 4 is executed as shown in FIG. 6, thereby stabilizing the display position of the obstruction target icon representing the obstruction target on the display device 7a, thereby improving the driver's visibility of the surroundings of the vehicle 10.

[0096] First, the position determination unit 233 determines whether or not the target is a closing target indicating that a lane is closed, based on the type of the target included in the target detection information (step S301).

[0097] If the object is a closure object indicating that a lane is closed (step S301-Yes), the position determination unit 233 selects the lane in which the closure object is located (step S302). The position determination unit 233 refers to map information and selects a lane including a position where the detection position of the closure object is projected onto a road plane. Then, the position determination unit 233 selects, from a pair of lane dividing lines that divide the selected lane, a lane dividing line that is adjacent to the position where the detection position of the closure object is projected onto a road plane, as an adjacent lane dividing line. The closure object is arranged along the adjacent lane dividing line to indicate that the selected lane is closed.

[0098] Next, the position determination unit 233 determines whether a closed lane section in which a lane is closed by a closure target has started (step S303). When the position determination unit 233 detects a number of closure targets equal to or greater than a predetermined reference number for a lane, it determines that a closed lane section has started for this lane. On the other hand, when the position determination unit 233 does not detect a number of closure targets equal to or greater than the predetermined reference number for a lane, it determines that a closed lane section has not started for this lane.

[0099] As described above, when target detection information representing a plurality of targets is input to the display image generating device 13, the process of step S103 described above is performed for each target. This allows the position determining unit 233 to count the number of closed targets detected for one lane.

[0100] When a closed lane section starts (step S303-Yes), the position determination unit 233 determines a predetermined interval at which a closure target icon representing a closure target is displayed in a display image along a lane dividing line representing a closed lane (step S304). The closure target icon is a first closure target icon or a second closure target icon. A fixed value (for example, 10 m to 20 m) may be used as the predetermined interval.

[0101] The position determination unit 233 may determine the predetermined interval based on the speed of the vehicle 10. Also, the position determination unit 233 may determine the predetermined interval based on the speed limit of the road on which the vehicle 10 travels. Also, the position determination unit 233 may determine the predetermined interval based on the curvature of the road on which the vehicle 10 travels. In this case, it is preferable to shorten the interval as the curvature of the road increases. Furthermore, the position determination unit 233 may determine the predetermined interval based on the projection angle at which the placement position of the closed target expressed in the vehicle coordinate system is projected onto the above-mentioned projection plane.

[0102] Next, the position determination unit 233 determines the display position of the closure target icon in the display image (step S305), and ends the series of processes. The position determination unit 233 arranges the closure target icon at a predetermined interval along the adjacent lane dividing line of the closed lane. Then, the position determination unit 233 projects the arrangement position of the closure target icon expressed in the vehicle coordinate system onto the above-mentioned projection surface, and determines the display position of the target icon in the display image. Here, the icon selected in step S103 is used as the closure target icon.

[0103] The multiple closure target icons, the display positions of which have been determined in this manner, are displayed at a predetermined interval along the lane dividing lines that represent the lanes to be closed in the display image. Since the display positions of the closure target icons are not related to the detection positions of the actual closure targets, the display positions of the closure target icons displayed on the display device 7a can be stabilized, thereby improving the driver's visibility of the surroundings of the vehicle 10.

[0104] On the other hand, if the closed lane section has not started (step S303-No), the position determination unit 233 determines the display position of the closure target icon in the display image (step S305), and ends the series of processes. The position determination unit 233 projects the detection position of the closure target expressed in the vehicle coordinate system onto the above-mentioned projection plane, and determines the display position of the closure target icon in the display image.

[0105] If the target object is not a closed target object indicating that a lane is closed (step S301-No), the position determination unit 233 determines whether or not a closed lane section has started (step S306). The start of a closed lane section means a state in which it has been determined in step S303 that the closed lane section has started, and has not been determined in step S307 that the closed lane section has ended.

[0106] If the closed lane section has started (step S306-Yes), the position determination unit 233 determines whether the closed lane section has ended (step S307). If a closed object is not detected in the closed lane for a predetermined time or a predetermined travel distance, the position determination unit 233 determines that the closed lane section has ended. On the other hand, if the time or travel distance during which a closed object is not detected in the closed lane has not reached the predetermined time or the predetermined travel distance, the position determination unit 233 determines that the closed lane section has not ended.

[0107] When the closed lane section ends (step S307-Yes), the position determination unit 233 determines the display position of the target icon in the display image (step S308), and ends the series of processes. The position determination unit 233 projects the detection position of the target represented in the vehicle coordinate system onto the above-mentioned projection plane, and determines the display position of the target icon in the display image.

[0108] On the other hand, if the closed lane section has not ended (step S307-No), the process proceeds to step S304. In this case, multiple closure target icons are displayed at predetermined intervals along the lane dividing line representing the closed lane.

[0109] If the closed lane section has not started (step S306-No), the process proceeds to step S308. The fact that the closed lane section has not started means that it has not been determined in step S303 that the closed lane section has started.

[0110] In the present disclosure, the display control device, the computer program for display control, and the display control method of the above-mentioned embodiments can be modified as appropriate without departing from the spirit of the present disclosure. Furthermore, the technical scope of the present disclosure is not limited to those embodiments, but extends to the inventions described in the claims and their equivalents.

[0111] For example, the method of generating the display image to be displayed on the display unit is not limited to the above-mentioned method. For example, a display image including lane markings and vehicle icons may be generated using a known method.

[0112] In addition, in the above-described embodiment, the obstruction target icon was selected from among a plurality of target icons stored in the memory by the icon selection unit, but the obstruction target icon may also be generated by a display image generation device.

[0113] In the above-described modified example, in a closed lane section, the lane dividing line indicating the closed lane and other vehicles on the closed lane do not have to be displayed in the displayed image. In this case, if another vehicle is detected moving from the closed lane section to an adjacent travel lane, the lane dividing line indicating the closed lane and this other vehicle may be displayed. [Explanation of symbols]

[0114] 1 Display System 2 Camera 3 LiDAR Sensor 4 Torque Sensor 5 Pedal Sensor 6 Positioning information receiver 7 User Interface 7a Display device 10 Vehicles 11 Map information storage device 12 Object detection device 13 Display image generation device 21 Communication Interface 22 Memory 23 Processors 231 Display control unit 232 Icon Selection Section 233 Position determination section 14 In-vehicle network

Claims

1. an acquisition unit that acquires detection information representing a closing target indicating that a lane is to be closed, which is detected around the vehicle, and operation information representing an operation by a driver on an operation unit that operates the vehicle; a display control unit that displays a target icon representing the occlusion target on a display unit; having When the detection information is acquired via the acquisition unit and the operation information is not acquired via the acquisition unit, the display control unit displays a first target icon representing the occluded target on the display unit, and when the detection information is acquired via the acquisition unit and the operation information is acquired via the acquisition unit, the display control unit displays a second target icon representing the occluded target and having a lower visibility to the driver than the first target icon on the display unit. The display control device, wherein the operation information includes a driver operating a steering wheel to move the vehicle to the opposite side of a closed lane.

2. 2. The display control device according to claim 1, wherein the second target icon being less visible to the driver than the first target icon includes: the second target icon being in a color that is less visually prominent to the driver than the first target icon; or the second target icon being more transparent than the first target icon.

3. A display control device as described in claim 1 or 2, wherein the operation information represents operations by a driver on an accelerator pedal, a brake pedal, and a steering wheel.

4. An acquisition unit that acquires detection information representing a closure target indicating that a lane is to be closed, which is detected around a vehicle, and operation information representing an operation by a driver to an operation unit that operates the vehicle; a display control unit that displays a target icon representing the occlusion target on a display unit; a position determination unit that determines a display position at which a target icon representing the occlusion target is to be displayed on a display unit based on the detected position of the occlusion target; having When the detection information is acquired via the acquisition unit and the operation information is not acquired via the acquisition unit, the display control unit displays a first target icon representing the occluded target on the display unit, and when the detection information is acquired via the acquisition unit and the operation information is acquired via the acquisition unit, the display control unit displays a second target icon representing the occluded target and having a lower visibility to the driver than the first target icon on the display unit. when a lane closure section in which a plurality of the closure targets are arranged in one lane begins, the position determination unit determines the display positions of the plurality of first target icons or the second target icons representing the closure targets so that they are displayed at a predetermined interval along the lane dividing line representing the closed lane displayed on the display unit, without being based on the detected positions of the closure targets.

5. Acquire detection information representing a closing target indicating that a lane is to be closed, which is detected around the vehicle, and operation information representing an operation by a driver on an operation unit for operating the vehicle; A target icon representing the closure target is displayed on a display unit. causing a processor to execute a process including: When the detection information has been acquired and the operation information has not been acquired, a first target icon representing the occlusion target is displayed on the display unit, and when the detection information has been acquired and the operation information has been acquired, a second target icon representing the occlusion target and having a lower visibility to the driver than the first target icon is displayed on the display unit. The display control computer program, wherein the operation information includes a driver operating a steering wheel to move the vehicle to the opposite side of a closed lane.

6. A display control method executed by a display control device, comprising: Acquire detection information representing a closing target indicating that a lane is to be closed, which is detected around the vehicle, and operation information representing an operation by a driver on an operation unit for operating the vehicle; A target icon representing the closure target is displayed on a display unit. Including When the detection information has been acquired and the operation information has not been acquired, a first target icon representing the occlusion target is displayed on the display unit, and when the detection information has been acquired and the operation information has been acquired, a second target icon representing the occlusion target and having a lower visibility to the driver than the first target icon is displayed on the display unit. The display control method, wherein the operation information includes a driver operating a steering wheel to move the vehicle to the opposite side of a closed lane.

Images