Display control device, display control method, and display control program

Abstract

Aspects of non-limiting embodiments of the present disclosure relate to causing an occupant of a vehicle to easily ascertain a specific target object corresponding to a specific target object image, as compared with a case where characteristics derived from an appearance of the specific target object are not reflected in the specific target object image indicating the specific target object.SOLUTION: A display control device includes a control unit configured to cause a display unit to display a vehicle image indicating a vehicle when viewed from a virtual viewpoint based on detection information that can be detected by the vehicle, and cause the display unit to display a specific target image indicating a specific target present around the vehicle in a predetermined color simulating a characteristic color derived from an appearance of the specific target.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a display control device, a display control method, and a display control program. [Background technology]

[0002] Patent Document 1 discloses a vehicle control device that can perform automatic driving that gives a sense of security to the occupants. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7048398 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, an image showing another vehicle present in the vicinity of the vehicle is displayed on a display unit (see FIG. 8 of Patent Document 1). When an image showing a target present in the vicinity of the vehicle, such as the image showing the other vehicle, is displayed on a display unit, it has conventionally been difficult for an occupant of the vehicle to know which target the image showing the target corresponds to.

[0005] Therefore, the present disclosure aims to provide a display control device, a display control method, and a display control program that enable vehicle occupants to easily grasp a specific target corresponding to a specific target image, compared to a case where features derived from the appearance of the specific target are not reflected in the specific target image showing the specific target. [Means for solving the problem]

[0006] The display control device of claim 1 includes a control unit that causes a display unit to display a vehicle image showing the vehicle as seen from a virtual viewpoint based on detection information that the vehicle can detect, and also causes the display unit to display a specific target image showing a specific target that exists around the vehicle in a predetermined color that imitates a characteristic color derived from the appearance of the specific target.

[0007] In the display control device according to claim 1, the control unit causes the display unit to display a vehicle image showing the vehicle as seen from a virtual viewpoint based on the detection information. The control unit then causes the display unit to display a specific target image showing a specific target present around the vehicle in a predetermined color that simulates a characteristic color derived from the appearance of the specific target. As a result, the display control device reflects the characteristic derived from the appearance of the specific target in the color of the specific target image. Therefore, the display control device allows the vehicle occupant to more easily recognize the specific target corresponding to the specific target image compared to a case where the characteristic derived from the appearance of the specific target is not reflected in the specific target image.

[0008] A display control device according to claim 2 is the display control device according to claim 1, wherein the control unit displays a portion of the specific target image in the predetermined color, and displays the remaining portion in a predetermined manner.

[0009] In the display control device according to claim 2, the control unit displays a part of the specific target image in a predetermined color and displays the remaining part in a predetermined manner. This reduces the processing load on the control unit compared to when the entire specific target image is displayed in a predetermined color.

[0010] A display control device according to claim 3 is the display control device according to claim 1 or 2, wherein the control unit selects, as the predetermined color, a color having the smallest color difference from the characteristic color from among a plurality of colors prepared in advance.

[0011] In the display control device according to claim 3, the control unit selects the color having the smallest color difference from the characteristic color as the predetermined color from among a plurality of colors prepared in advance. This reduces the processing load on the control unit compared to when the display control device generates a predetermined color that imitates the characteristic color each time a specific target image is displayed.

[0012] A display control device according to claim 4 is the display control device according to any one of claims 1 to 3, wherein the specific target is a target having a predetermined attribute.

[0013] In the display control device according to claim 4, the specific target is a target having a predetermined attribute, thereby making it possible for the display control device to prompt the vehicle occupants to notice targets having the predetermined attribute that are present around the vehicle.

[0014] A display control device according to claim 5 is the device of claim 4, wherein the control unit causes the display unit to display the specific target image in a mode according to the attribute of the specific target.

[0015] In the display control device according to claim 5, the control unit causes the display unit to display the specific target image in a manner according to the attributes of the specific target, thereby increasing the degree to which the vehicle occupant can recognize the specific target.

[0016] The display control device of claim 6 is any one of claims 1 to 5, wherein when the control unit causes the display unit to display, together with the specific target image, an other target image indicating a target other than the specific target, based on the detection information, the other target image is displayed in a predetermined manner.

[0017] In the display control device according to claim 6, when the control unit causes the display unit to display the other target image together with the specific target image based on the detection information, the control unit displays the other target image in a predetermined manner. As a result, the display control device can increase the attention of vehicle occupants to the specific target while reducing the processing load on the control unit compared to when features derived from the appearances of the other targets are reflected in the other target image.

[0018] A display control device according to a seventh aspect of the present invention is the display control device according to any one of the first to sixth aspects, wherein the display unit is a meter display provided in front of a driver's seat of the vehicle.

[0019] In the display control device according to claim 7, the display unit is a meter display provided in front of the driver's seat of the vehicle. This allows the display control device to enable an occupant who cannot turn his or her eyes to multiple places while driving the vehicle to efficiently grasp information indicating the meters of the vehicle's measuring instruments, the vehicle image, and the specific target image.

[0020] The display control device of claim 8 is any one of claims 1 to 7, wherein the control unit causes the display unit to display a specific target image indicating the target requiring the highest level of attention among the specific targets in the predetermined color.

[0021] In the display control device according to claim 8, the control unit causes the display unit to display a specific target image indicating the target with the highest level of caution among the specific targets in a predetermined color. This allows the display control device to prompt the vehicle occupants to notice the target with the highest level of caution among the specific targets present around the vehicle.

[0022] The display control method of claim 9 is a method in which a computer executes a process to display on a display unit a vehicle image showing the vehicle as seen from a virtual viewpoint based on detection information detectable by the vehicle, and to display on the display unit a specific target image showing a specific target present around the vehicle in a predetermined color that imitates a characteristic color derived from the appearance of the specific target.

[0023] In a display control method according to claim 9, a vehicle image showing the vehicle as seen from a virtual viewpoint is displayed on a display unit based on detection information. In the display control method, a specific target image showing a specific target present around the vehicle is displayed on the display unit in a predetermined color that imitates a characteristic color derived from the appearance of the specific target. In this way, the display control method reflects the characteristic derived from the appearance of the specific target in the color of the specific target image. Therefore, the display control method allows a vehicle occupant to easily recognize the specific target corresponding to the specific target image compared to a case in which the characteristic derived from the appearance of the specific target is not reflected in the specific target image.

[0024] The display control program of claim 10 causes a computer to execute processing to display on a display unit a vehicle image showing the vehicle as seen from a virtual viewpoint based on detection information detectable by the vehicle, and to display on the display unit a specific target image showing a specific target present around the vehicle in a predetermined color that imitates a characteristic color derived from the appearance of the specific target.

[0025] In a display control program according to claim 10, a vehicle image showing the vehicle as seen from a virtual viewpoint is displayed on a display unit based on detection information. The display control program then displays a specific target image showing a specific target present around the vehicle in a predetermined color that simulates a characteristic color derived from the appearance of the specific target. As a result, the display control program reflects the characteristic derived from the appearance of the specific target in the color of the specific target image. Therefore, the display control program allows a vehicle occupant to more easily identify the specific target corresponding to the specific target image than when the characteristic derived from the appearance of the specific target is not reflected in the specific target image. [Effects of the Invention]

[0026] As described above, the display control device, display control method, and display control program disclosed herein allow vehicle occupants to easily grasp the specific target corresponding to the specific target image, compared to when features derived from the appearance of the specific target are not reflected in the specific target image showing the specific target. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a block diagram showing a schematic configuration of a vehicle. [Figure 2] 1 is a schematic view of a front portion of a vehicle interior as viewed from the rear side of the vehicle. [Figure 3] FIG. 2 is an explanatory diagram illustrating a display area of ​​a first display unit. [Figure 4] FIG. 1 is a block diagram showing a schematic configuration of a display control device. [Figure 5] FIG. 2 is a block diagram illustrating an example of a functional configuration of a display control device. [Figure 6] 10 is a flowchart showing the flow of a specification process. [Figure 7] FIG. 10 is a first explanatory diagram showing an example of a display displayed in the display area of ​​the first display unit. [Figure 8] FIG. 2 is an explanatory diagram showing the appearance of a specific target; [Figure 9] FIG. 10 is an explanatory diagram showing the appearance of another target. [Figure 10] FIG. 2 is a second explanatory diagram showing an example of a display displayed in the display area of ​​the first display unit. [Figure 11] FIG. 10 is a third explanatory diagram showing an example of a display displayed in the display area of ​​the first display unit. [Figure 12] FIG. 4 is a fourth explanatory diagram showing an example of a display displayed in the display area of ​​the first display unit. [Figure 13] FIG. 5 is a fifth explanatory diagram showing an example of a display displayed in the display area of ​​the first display unit. DETAILED DESCRIPTION OF THE INVENTION

[0028] The vehicle 10 according to this embodiment will be described below. (First embodiment) First, a first embodiment of a vehicle 10 according to the present embodiment will be described.

[0029] FIG. 1 is a block diagram showing a schematic configuration of a vehicle 10. As shown in FIG. 1, the vehicle 10 includes a display device 21, a sensor group 30, a camera 40, and a display control device 50. The display device 21, the sensor group 30, the camera 40, and the display control device 50 are connected to each other via an external bus (communication bus) 15.

[0030] The display device 21 displays, for example, suggestions for operation related to functions of the vehicle 10 and images explaining the functions. In this embodiment, a first display unit 24 and a second display unit 25 are provided as the display device 21. The first display unit 24 is an example of a "display unit" in the present disclosure. Details of the first display unit 24 and the second display unit 25 will be described later.

[0031] The sensor group 30 includes sensors for detecting the state and surrounding conditions of the vehicle 10, such as a 3D-LiDAR, a millimeter wave sensor, an infrared sensor, a blinker sensor, an accelerator position sensor, a shift position sensor, an illuminance sensor, a vehicle speed sensor, a steering angle sensor, an angular velocity sensor, a GPS (Global Positioning System) sensor, a gyro sensor, and an acceleration sensor. The sensor group 30 outputs the detection results of each sensor to the display control device 50.

[0032] The camera 40 is an imaging device that captures images using an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The cameras 40 are provided on the front, rear, left side, and right side of the vehicle 10, and capture images of the surroundings of the vehicle 10. For example, images captured by the camera 40 provided on the front of the vehicle 10 are used to recognize other vehicles traveling in front of the vehicle, the distance between the other vehicles, lanes, traffic lights, etc. The images captured by the camera 40 are output to the display control device 50. The camera 40 may also be configured as an imaging device for other purposes, such as a drive recorder and an ADAS (Advanced Driver Assistance System).

[0033] The display control device 50 is, for example, an ECU (Electrical Control Unit) that performs various controls.

[0034] 2 is a schematic diagram of the front of the passenger compartment 12 of the vehicle 10 as seen from the rear of the vehicle. The arrow UP in FIG. 2 indicates the upper side in the vertical direction of the vehicle, and the arrow RH indicates the right side in the width direction of the vehicle. In the following description, the vertical direction and the left-right direction refer to the up and down in the vertical direction of the vehicle and the left and right in the width direction of the vehicle, respectively.

[0035] As shown in Fig. 2, an instrument panel 14 is provided in the front of a passenger compartment 12 of a vehicle 10. The instrument panel 14 extends in the vehicle width direction, and a steering wheel 16 is provided on the right side of the instrument panel 14. That is, as an example, the vehicle 10 is a right-hand drive vehicle in which the steering wheel 16 is provided on the right side, and the driver's seat is provided on the right side of the vehicle. However, the present invention is not limited to this, and the vehicle 10 may also be a left-hand drive vehicle in which the driver's seat is provided on the left side of the vehicle.

[0036] A windshield glass 18 is provided at the front end of the instrument panel 14. The windshield glass 18 extends in the vertical direction and the width direction of the vehicle, and separates the interior of the vehicle compartment 12 from the exterior of the vehicle compartment 12.

[0037] The right end of the windshield glass 18 is fixed to a front pillar 20 on the right side of the vehicle. The front pillar 20 extends in the vertical direction of the vehicle, and the windshield glass 18 is fixed to the inner end of the front pillar 20 in the vehicle width direction. In addition, the front ends of front side windows 22 are fixed to the outer end of the front pillar 20 in the vehicle width direction. The left end of the windshield glass 18 is fixed to a front pillar on the left side of the vehicle (not shown).

[0038] Here, the instrument panel 14 is provided with a first display unit 24 having an image display area V. The first display unit 24 is configured as a meter display provided in front of the driver's seat on the right side of the instrument panel 14. The first display unit 24 is connected to various meter devices mounted on the vehicle 10, and is provided in a position that is within the field of view of the driver when the driver is looking ahead of the vehicle.

[0039] The instrument panel 14 is also provided with a second display unit 25. The second display unit 25 is configured by a center display disposed in the center of the instrument panel 14 in the vehicle width direction.

[0040] FIG. 3 is an explanatory diagram illustrating the display area of ​​the meter display as the display area V of the first display unit 24.

[0041] 3 shows a display area V of the first display unit 24. In this embodiment, a vehicle image 10A (see FIG. 7, etc.) that is an image showing the vehicle 10 and a surrounding situation image that is an image showing the surrounding situation of the vehicle 10 are displayed in a display area X that is a part of the display area V. Specific examples of these images will be described later.

[0042] The display area X is an area that can be seen by a driver seated in the driver's seat through an opening 17 (see FIG. 2) of the steering wheel 16. Specifically, as shown in FIG. 3, the center of the area that can be seen between an upper ridge line L1 and a lower ridge line L2 of the opening 17 is the display area X. On the left and right sides of the display area X in the display area V, meter displays M1 and M2 that indicate meters of measuring instruments of the vehicle 10 are displayed.

[0043] 4 is a block diagram showing a schematic configuration of the display control device 50. The display control device 50 is an example of the "computer" of the present disclosure.

[0044] 4, the display control device 50 includes a CPU (Central Processing Unit) 51, a ROM (Read Only Memory) 52, a RAM (Random Access Memory) 53, a storage 54, a communication I / F (Interface) 55, and an input / output I / F 56. The CPU 51, the ROM 52, the RAM 53, the storage 54, the communication I / F 55, and the input / output I / F 56 are connected via an internal bus 57 so as to be able to communicate with each other.

[0045] The CPU 51 is a central processing unit that executes various programs and controls various parts. That is, the CPU 51 reads out programs from the ROM 52 and executes the programs using the RAM 53 as a work area.

[0046] The ROM 52 stores various programs and various data. The RAM 53 serves as a working area for temporarily storing programs or data.

[0047] The storage 54 is a NAND memory configured by an eMMC (embedded multi media card) or a UFS (universal flash storage), etc. The storage 54 and the CPU 51 are connected via an eMMC or UFS interface.

[0048] Furthermore, each component included in the display control device 50, such as the ROM 52 and the storage 54, is preferably a tamper-resistant medium such as an HSM (Hardware Security Module).

[0049] The communication I / F 55 is an interface for connecting with other ECUs, and uses the Ethernet (registered trademark) communication standard.

[0050] The input / output I / F 56 is an interface for communicating with on-board devices such as the display device 21, the sensor group 30, and the camera 40 mounted on the vehicle 10.

[0051] Here, the ROM 52 stores a display control program 52A, map data 52B, an object detection model 52C, illustration data 52D, and color data 52E.

[0052] The display control program 52A is a program for causing the CPU 51 to execute various processes. When executing the display control program 52A, the display control device 50 executes processes based on the display control program 52A using the configuration shown in Fig. 4. The display control program 52A is an example of a "display control program" of the present disclosure.

[0053] The map data 52B is high-precision map data that shows a map of the entire country in which the vehicle 10 is located.

[0054] The object detection model 52C is a model that recognizes the area of ​​a target object shown in the captured image and determines the class (type) of the target object when an image captured by the camera 40 is input. The object detection model 52C is configured by appropriately using known models such as YOLO (YOU Only Look Once) and SSD (Single Shot Multibox Detector).

[0055] The targets include, for example, humans, animals, vehicles, etc. The classes are set up as follows: children, adults, and elderly people for humans, dogs, cats, and horses for animals, and two-wheeled vehicles, three-wheeled vehicles, and four-wheeled vehicles for vehicles.

[0056] The illustration data 52D is data showing the illustration design of each target to be displayed on the first display unit 24.

[0057] The color data 52E is data indicating colors with which an image showing a specific target (described later) can be colored. For example, the color data 52E is RGB values ​​corresponding to a plurality of colors prepared in advance. As a result, the color data 52E stores RGB values ​​corresponding to red, blue, white, black, and the like.

[0058] Next, the functional configuration of the display control device 50 will be described. FIG. 5 is a block diagram showing an example of the functional configuration of the display control device 50. As shown in FIG.

[0059] 5, the CPU 51 of the display control device 50 has, as functional components, an acquisition unit 51A and a control unit 51B. Each functional component is realized by the CPU 51 reading and executing a display control program 52A stored in the ROM 52. The control unit 51B is an example of the "control unit" of the present disclosure.

[0060] The acquisition unit 51A acquires various types of information. For example, the acquisition unit 51A acquires, as the various types of information, detection information that can be detected by the vehicle 10. The detection information includes detection results by each sensor constituting the sensor group 30, captured images by the camera 40, etc.

[0061] The control unit 51B performs display control related to the display of the display device 21. For example, as the display control, the control unit 51B causes the first display unit 24 to display the vehicle image 10A and the surrounding situation image as viewed from a virtual viewpoint based on the detection information acquired by the acquisition unit 51A. The virtual viewpoint is set in a three-dimensional virtual space with the position of the vehicle image 10A as the origin, and is defined by viewpoint coordinates and a viewpoint angle (orientation) in the virtual space. As an example, the virtual viewpoint is a viewpoint viewed at a specific viewpoint angle from specific viewpoint coordinates above the vehicle image 10A in the virtual space. The surrounding situation image also includes a road image showing a road including a sidewalk and a roadway on which the vehicle 10 travels, and target images showing targets other than the vehicle 10, such as other vehicles and pedestrians.

[0062] 6 is a flowchart showing the flow of the specific processing executed by the display control device 50. The specific processing is performed by the CPU 51 reading the display control program 52A from the ROM 52, expanding it in the RAM 53, and executing it. As an example, the specific processing is automatically and repeatedly performed at regular intervals.

[0063] 6, the CPU 51 acquires detection information that can be detected by the vehicle 10. Then, the CPU 51 proceeds to step S11.

[0064] In step S11, the CPU 51 displays a road image on the first display unit 24 based on the detection information acquired in step S10. For example, the CPU 51 identifies the current location of the vehicle 10 from the detection result of a GPS sensor included in the detection information. Thereafter, the CPU 51 creates a 3D model of the roads around the vehicle 10 corresponding to the current location shown in the map data 52B and draws the 3D model on the first display unit 24. The CPU 51 then proceeds to step S12.

[0065] In step S12, the CPU 51 displays the vehicle image 10A on the first display unit 24 based on the detection information acquired in step S10. The CPU 51 acquires an illustration design of the vehicle 10 from the illustration data 52D, and draws the vehicle image 10A generated based on the illustration design on the first display unit 24. At this time, the CPU 51 displays the vehicle image 10A on the road image in an arrangement according to the current location of the vehicle 10 identified from the detection information and from a viewpoint seen from the virtual viewpoint. The CPU 51 then proceeds to step S13.

[0066] In step S13, the CPU 51 determines whether or not a specific target exists around the vehicle 10 based on the detection information acquired in step S10. If the CPU 51 determines that a specific target exists (step S13: YES), the process proceeds to step S14. On the other hand, if the CPU 51 determines that a specific target does not exist (step S13: NO), the process proceeds to step S17.

[0067] Here, the specific target is a target having a predetermined attribute. It is predetermined which of various targets existing in the real space corresponds to the specific target. In this embodiment, the attribute is a human being classified into a child or elderly person class as the specific target. Note that the attributes that can correspond to the specific target are not limited to those described above, and may be, for example, a human being classified into a pedestrian class (i.e., pedestrians in general), or a vehicle classified into a class such as a motorcycle or small vehicle. The attribute indicates that caution is required, i.e., a high level of caution is required. "A high level of caution" means that there is a high possibility that the driver of the vehicle 10 will experience an incident such as a near miss, or that an accident will occur, such as a collision between the vehicle 10 and a target.

[0068] The CPU 51 inputs the image captured by the camera 40 included in the detection information acquired in step S10 into the object detection model 52C, and determines whether or not a specific target exists based on the output result of the object detection model 52C. Specifically, the CPU 51 determines that a specific target exists when the output result of the object detection model 52C includes a human classified as a child or an elderly person. On the other hand, the CPU 51 determines that a specific target does not exist when the output result of the object detection model 52C does not include a human classified as a child or an elderly person.

[0069] In step S14, the CPU 51 derives a characteristic color from the appearance of the specific target. For example, the characteristic color is a representative color that indicates a representative color of the image. The CPU 51 derives the representative color by averaging, finding the mode, or finding the median of the RGB values ​​of each pixel in the area of ​​the specific target in the image captured by the camera 40, which is included in the detection information acquired in step S10. Then, the CPU 51 proceeds to step S15.

[0070] In step S15, the CPU 51 selects a predetermined color that resembles the characteristic color derived in step S14. For example, the CPU 51 selects the color that has the smallest color difference from the characteristic color from among the multiple colors represented in the color data 52E as the predetermined color. The CPU 51 then proceeds to step S16.

[0071] In step S16, the CPU 51 displays a specific target image, which is an image showing a specific target, on the first display unit 24 after coloring it with the predetermined color selected in step S15. The CPU 51 acquires an illustration design of the specific target from the illustration data 52D, and draws the specific target image generated by coloring the illustration design with the predetermined color on the first display unit 24. At this time, the CPU 51 displays the specific target image on the road image in an arrangement according to the current position of the specific target identified from the detection information and from a viewpoint seen from the virtual viewpoint. Then, the CPU 51 proceeds to step S17.

[0072] In step S17, the CPU 51 determines, based on the detection information acquired in step S10, whether or not there is another target other than the specific target around the vehicle 10. If the CPU 51 determines that there is another target (step S17: YES), the process proceeds to step S18. On the other hand, if the CPU 51 determines that there is no other target (step S17: NO), the process proceeds to step S19.

[0073] The CPU 51 inputs the image captured by the camera 40 included in the detection information acquired in step S10 into the object detection model 52C, and determines whether or not another target exists based on the output result of the object detection model 52C. Specifically, the CPU 51 determines that another target exists when the output result of the object detection model 52C includes a human classified into a class other than children and elderly people, or a target classified into a class other than the human. On the other hand, the CPU 51 determines that another target does not exist when the output result of the object detection model 52C includes only humans classified into the child or elderly class, or does not include any targets classified into any class.

[0074] In step S18, the CPU 51 displays an other target image, which is an image showing the other target, on the first display unit 24 in a predetermined manner. The predetermined manner is, for example, a default color scheme. The CPU 51 acquires an illustration design of the other target from the illustration data 52D and draws an other target image generated based on the illustration design on the first display unit 24. At this time, the CPU 51 displays the other target image on the road image in an arrangement according to the current location of the other target identified from the detection information and from a viewpoint seen from the virtual viewpoint. Then, the CPU 51 proceeds to step S19.

[0075] In step S19, the CPU 51 updates the display content of the first display unit 24 based on the detection information acquired in step S10. For example, the CPU 51 changes the numerical value indicated in vehicle speed information 72 (see FIG. 7, etc.) described later that is displayed on the first display unit 24 when the speed of the vehicle 10 increases or decreases, or when the vehicle 10 approaches a pedestrian, reduces the distance between the vehicle image 10A on the first display unit 24 and the image showing the pedestrian. Then, the CPU 51 proceeds to step S20.

[0076] In step S20, the CPU 51 determines whether a predetermined termination condition is met. If the CPU 51 determines that the termination condition is met (step S20: YES), the CPU 51 terminates the specific processing. On the other hand, if the CPU 51 determines that the termination condition is not met (step S20: NO), the CPU 51 returns to step S10. As an example, the CPU 51 determines that the termination condition is met when the ignition switch of the vehicle 10 is turned off.

[0077] Next, examples of display on the first display unit 24 based on the above-described specification process will be described with reference to FIGS.

[0078] Fig. 7 is a first explanatory diagram showing an example of a display displayed in the display area X of the first display unit 24. As an example, Fig. 7 shows the display content of the display area X when the vehicle 10 is traveling on an open road.

[0079] Shift information 70 and vehicle speed information 72 are displayed in the upper part of the display area X shown in FIG.

[0080] The shift information 70 indicates the shift position of the vehicle 10. In Fig. 7, "D" is displayed as the shift information 70, indicating that the shift position is in the D range.

[0081] The vehicle speed information 72 indicates the speed of the vehicle 10. In Fig. 7, "20km / h" is displayed as the vehicle speed information 72, indicating that the vehicle speed is 20km / h.

[0082] 7, a general road image 60 is displayed below the shift information 70 and vehicle speed information 72. The general road image 60 is an illustrated image that is a 3D model of the general road around the vehicle 10 shown in the map data 52B. The general road image 60 is divided into a roadway Y that is an area showing the roadway, and sidewalks Z1 and Z2 that are areas showing the sidewalks. A stop line image 74 showing a stop line is displayed on the roadway Y in a portion that overlaps with the vehicle image 10A, and a crosswalk image 76 showing the crosswalk for crossing between sidewalks Z1 and Z2 is displayed.

[0083] 7, a vehicle image 10A, a pedestrian image 78 as a specific target image, and a pedestrian image 80 as a different target image are displayed on an ordinary road image 60. In FIG.

[0084] Vehicle image 10A is an illustration image generated based on the illustration design of vehicle 10 shown in illustration data 52D. As an example, vehicle 10 is passing the stop line shown in stop line image 74 and turning right at an intersection. Therefore, in FIG. 7, vehicle image 10A is displayed in the portion of roadway Y that overlaps with stop line image 74 from the viewpoint seen from the virtual viewpoint.

[0085] The pedestrian image 78 is a specific target image showing the user U1 (see FIG. 8) crossing the crosswalk shown in the crosswalk image 76. As such, since the user U1 is crossing the crosswalk, in FIG. 7 the pedestrian image 78 is displayed in the part of the roadway Y that overlaps with the crosswalk image 76 from a viewpoint seen from the virtual viewpoint.

[0086] Furthermore, pedestrian image 78 is an illustration image in which a portion of the illustration design of a person shown in illustration data 52D is colored in a predetermined color (e.g., black). Here, pedestrian image 78 can be divided into head 78A, upper body 78B, and lower body 78C. Head 78A is the part corresponding to the head of the person shown in pedestrian image 78. Upper body 78B is the part of the person from the neck to the hip joints. Lower body 78C is the part of the person below the hip joints. In FIG. 7, pedestrian image 78 has upper body 78B colored in a predetermined color (e.g., black), and head 78A and lower body 78C are displayed in a predetermined manner (e.g., white).

[0087] Pedestrian image 80 is a target image showing user U2 (see FIG. 9) who is present on sidewalk Z2. In this way, since user U2 is present on the sidewalk indicated by sidewalk Z2, in FIG. 7, pedestrian image 80 is displayed in the part of sidewalk Z2 from a viewpoint seen from the virtual viewpoint.

[0088] Furthermore, pedestrian image 80 is an illustration image generated by using a predetermined aspect of the illustration design of a person shown in illustration data 52D, specifically, a default color scheme. Here, pedestrian image 80 can be divided into head 80A, upper body 80B, and lower body 80C. Head 80A is the part of the person shown in pedestrian image 80 that corresponds to the head. Upper body 80B is the part of the person from the neck to the hip joints. Lower body 80C is the part of the person below the hip joints. In FIG. 7, pedestrian image 80 displays head 80A, upper body 80B, and lower body 80C in a predetermined aspect (e.g., white).

[0089] FIG. 8 is an explanatory diagram showing the appearance of a user U1 as a specific target. As an example, the user U1 is an 8-year-old child. In the above-described identification process, the user U1 is classified into the child class and is identified as a specific target present in the vicinity of the vehicle 10.

[0090] Here, user U1 has a head 90A, an upper body 90B, and a lower body 90C. The head 90A is the part of user U1 that corresponds to his / her head. The upper body 90B is the part of user U1 from the neck to the hip joints. The lower body 90C is the part of user U1 that corresponds to below the hip joints. User U1 is wearing a black long-sleeved turtleneck 91 on the upper body 90B and black long pants 92 on the lower body 90C.

[0091] When user U1 has the appearance shown in FIG. 8, the characteristic color derived from the appearance in the above-described identification process is, for example, black with RGB values ​​of (0:0:0). Assume also that the RGB value corresponding to black stored in color data 52E is (0:0:0). In this case, CPU 51 selects, as the predetermined color, the black with RGB values ​​of (0:0:0) that has the smallest color difference from the characteristic color black with RGB values ​​of (0:0:0) from among the multiple colors represented in color data 52E. Therefore, in pedestrian image 78, which is an image of user U1 shown in FIG. 7, the upper body 78B is colored black with RGB values ​​of (0:0:0), which is the predetermined color. Note that in this embodiment, when the specific target image is an image of a person, the portion to be colored with the predetermined color is preset to the upper body.

[0092] FIG. 9 is an explanatory diagram showing the appearance of a user U2 as another target. As an example, the user U2 is an adult aged 30. In addition, the user U2 is classified into the adult class in the above-described identification process and is identified as another target present in the vicinity of the vehicle 10.

[0093] Here, user U2 has a head 95A, an upper body 95B, and a lower body 95C. The head 95A is the part of user U2 that corresponds to his / her head. The upper body 95B is the part of user U2 from the neck to the hip joints. The lower body 95C is the part of user U2 that corresponds to below the hip joints. User U2 is wearing a white sleeveless top 96 with a moon mark 96A design representing the moon on the upper body 95B, and white shorts 97 with a vertical stripe design on the lower body 95C.

[0094] User U2 has the appearance shown in Figure 9, but because he has been identified as another target, the pedestrian image 80 showing user U2 shown in Figure 7 has the head 80A, upper body 80B, and lower body 80C represented in a predetermined white color scheme.

[0095] As described above, in the display control device 50, the CPU 51 displays the vehicle image 10A, which shows the vehicle 10 as seen from a virtual viewpoint, on the first display unit 24 based on the detection information. Then, the CPU 51 displays the pedestrian image 78, which shows the user U1 as a specific target present around the vehicle 10, on the first display unit 24 in a predetermined color that imitates a characteristic color derived from the appearance of the user U1. As a result, in the display control device 50, the characteristics derived from the appearance of the user U1 are reflected in the color of the pedestrian image 78. Therefore, in the display control device 50, it is possible for the driver, who is an occupant of the vehicle 10, to more easily recognize the user U1 corresponding to the pedestrian image 78 than in a case where the characteristics derived from the appearance of the user U1 are not reflected in the pedestrian image 78.

[0096] Furthermore, in the display control device 50, the CPU 51 displays a portion of the pedestrian image 78 in a predetermined color and displays the remaining portion in a predetermined manner. This reduces the processing load on the CPU 51 in the display control device 50 compared to when the entire pedestrian image 78 is displayed in a predetermined color.

[0097] Furthermore, in the display control device 50, the CPU 51 selects a color that has the smallest color difference from the characteristic color as the predetermined color from among a plurality of colors prepared in advance. This reduces the processing load on the CPU 51 compared to when the display control device 50 generates a predetermined color that imitates the characteristic color each time a specific target image is displayed.

[0098] Furthermore, in the display control device 50, the user U1 as a specific target is a target having a predetermined attribute. As a result, the display control device 50 can prompt the driver of the vehicle 10 to notice targets having the predetermined attribute that are present around the vehicle 10.

[0099] Furthermore, in the display control device 50, when the CPU 51 causes the first display unit 24 to display a pedestrian image 80 showing the user U2 as another target together with the pedestrian image 78 based on the detection information, the CPU 51 displays the pedestrian image 80 in a predetermined manner. As a result, the display control device 50 can increase the attention of the driver of the vehicle 10 to the user U1 as a specific target while reducing the processing load on the CPU 51 compared to when features derived from the appearance of the user U2 are reflected in the pedestrian image 80.

[0100] In the display control device 50, the first display unit 24 is a meter display provided in front of the driver's seat of the vehicle 10. This allows the display control device 50 to enable the driver, who is unable to look at multiple places while driving the vehicle 10, to efficiently grasp information indicating the meters of the measuring instruments of the vehicle 10, the vehicle image 10A, and the pedestrian image 78.

[0101] (Second embodiment) Next, a second embodiment of the vehicle 10 according to the present invention will be described while omitting or simplifying parts that overlap with the above embodiment.

[0102] The second embodiment differs from the first embodiment in that the CPU 51 displays a specific target image on the first display unit 24 in a manner according to the attributes of the specific target. As an example, the manner according to the attributes of the specific target is a manner that makes it possible to recall the class of the specific target. Specifically, if the specific target is a human classified as a child, the display size of the specific target image on the first display unit 24 is smaller than images showing humans classified as other classes, as a manner according to the attributes of the specific target.

[0103] Fig. 10 is a second explanatory diagram showing an example of a display displayed in the display area X of the first display unit 24. As an example, Fig. 10 shows the display content of the display area X when the vehicle 10 is traveling on an open road, similar to Fig. 7.

[0104] 7, the display area X shown in Fig. 10 displays a vehicle image 10A, a general road image 60, shift information 70, vehicle speed information 72, a stop line image 74, a crosswalk image 76, a pedestrian image 78, and a pedestrian image 80. Note that the display manner of each of the above information other than the pedestrian image 78 as a specific target image is the same as in Fig. 7, and therefore description thereof will be omitted.

[0105] The pedestrian image 78 is a specific target image showing the user U1, an 8-year-old child, crossing the crosswalk shown in the crosswalk image 76. The user U1 is classified into the child class in the above-described identification process and is recognized as a specific target present in the vicinity of the vehicle 10.

[0106] In the above case, the CPU 51 displays the pedestrian image 78 on the first display unit 24 in a mode according to the attributes of the user U1 as a specific target. In FIG. 10, the pedestrian image 78 has a smaller display size in the display area X than the pedestrian image 80 representing the user U2 who is classified as an adult. This allows the display control device 50 according to the second embodiment to increase the degree to which the driver of the vehicle 10 recognizes the user U1.

[0107] (Third embodiment) Next, a third embodiment of the vehicle 10 according to the present invention will be described while omitting or simplifying parts that overlap with the above-described embodiment.

[0108] The third embodiment differs from the above embodiments in that the CPU 51 displays a specific target image, which indicates the target with the highest level of caution among specific targets present around the vehicle 10, in a predetermined color on the first display unit 24. In this embodiment, a person classified into the pedestrian class is set as the specific target as the attribute. In addition, as an example, the level of caution is set to be the highest for a person classified into the child class among pedestrians in general.

[0109] 7 to 9, in this embodiment, users U1 and U2 are humans classified as pedestrians and correspond to specific targets. In this case, the CPU 51 displays the upper body 78B of a pedestrian image 78 representing user U1, who is classified as a child, a target with the highest level of caution among the specific targets, in a predetermined color (e.g., black). On the other hand, the CPU 51 displays the head 80A, upper body 80B, and lower body 80C of a pedestrian image 80 representing user U2, who is not a target with the highest level of caution among the specific targets, in a predetermined manner (e.g., white). With the above configuration, the display control device 50 according to the third embodiment can prompt the driver of the vehicle 10 to notice a target with the highest level of caution among specific targets present around the vehicle 10.

[0110] Note that examples of targets requiring the highest level of caution are not limited to those listed above. For example, if a person classified as a pedestrian is the specific target, a person classified as a child or elderly person among pedestrians in general may be the target with the highest level of caution, with a tie for first place. Furthermore, if a vehicle classified as a preceding vehicle of vehicle 10 is the specific target, a vehicle classified as a cutting-in vehicle may be the target with the highest level of caution.

[0111] Here, a display example of a specific target image in the case where a vehicle classified into the class of a cutting-in vehicle is set as a target requiring the highest level of caution will be described.

[0112] 11 is a third explanatory diagram showing an example of a display displayed in the display area X of the first display unit 24. As an example, FIG. 11 shows the display content of the display area X when the vehicle 10 is traveling on a public road different from that shown in FIG. 7.

[0113] Shift information 70 and vehicle speed information 72 are displayed in the upper part of the display area X shown in Fig. 11. In Fig. 11, "D" is displayed as the shift information 70, indicating that the shift position is in the D range. Also in Fig. 11, "54 km / h" is displayed as the vehicle speed information 72, indicating that the vehicle speed is 54 km / h.

[0114] 11, a general road image 65 as a road image is displayed below the shift information 70 and the vehicle speed information 72. The general road image 65 is an illustrated image showing the general roads around the vehicle 10 shown in the map data 52B.

[0115] 11, a vehicle image 10A, another vehicle image 10B, and another vehicle image 10C are displayed on an ordinary road image 65. In FIG.

[0116] As an example, the vehicle 10 is traveling in the center lane of a road with three lanes in each direction, and therefore, in Fig. 11, the vehicle image 10A is displayed in the center lane of the road shown in the general road image 65.

[0117] The other vehicle image 10B is an illustration image generated based on the illustration design of the other vehicle shown in the illustration data 52D. As an example, the other vehicle shown in the other vehicle image 10B is a preceding vehicle traveling in the same lane as the vehicle 10. Therefore, in FIG. 11, the other vehicle image 10B is displayed above the vehicle image 10A in the center lane of the road shown in the general road image 65.

[0118] The other vehicle image 10C is an illustration image generated based on the illustration design of the other vehicle shown in the illustration data 52D. As an example, the other vehicle shown in the other vehicle image 10C is a preceding vehicle traveling in an adjacent lane adjacent to the right side of the vehicle 10. Therefore, in FIG. 11, the other vehicle image 10C is displayed above the vehicle image 10A in the rightmost lane of the road shown in the general road image 65.

[0119] 11, it is assumed that there are no preceding vehicles classified as cutting-in vehicles, i.e., no specific target requiring the highest level of caution, among the preceding vehicles. Therefore, the CPU 51 displays the other vehicle image 10B and the other vehicle image 10C in a predetermined manner (e.g., white).

[0120] Fig. 12 is a fourth explanatory diagram showing an example of a display displayed in the display area X of the first display unit 24. As an example, Fig. 12 shows the display content of the display area X when the CPU 51 detects a cutting-in vehicle.

[0121] 12 displays a vehicle image 10A, another vehicle image 10B, another vehicle image 10C, an ordinary road image 65, shift information 70, and vehicle speed information 72. Note that the positions and display modes of the above information other than the other vehicle image 10C are the same as those in FIG. 11, and therefore will not be described.

[0122] 12, it is assumed that the CPU 51 has detected, based on the detection information, that another vehicle traveling in an adjacent lane adjacent to the right of the vehicle 10 is rapidly cutting in front of the vehicle 10. In other words, it is assumed that the CPU 51 has detected a preceding vehicle that is classified into the class of a cutting-in vehicle.

[0123] In the above case, the CPU 51 reflects the real-space position of the other vehicle that has become the specific target with the highest level of caution on the display area X. As an example, the other vehicle is straddling the white line on the right side of a three-lane road. Therefore, in FIG. 12, the other vehicle image 10C showing the other vehicle is displayed on the line 65A that divides the center lane of the road shown in the general road image 65 and the adjacent lane adjacent to the right of the center lane.

[0124] In the above case, the CPU 51 displays the other vehicle image 10C, which has become the specific target image showing the specific target with the highest level of caution, in a predetermined color (e.g., amber) that imitates the characteristic color derived from the appearance of the other vehicle. As a result, the CPU 51 highlights the other vehicle image 10C showing the other vehicle that is the specific target with the highest level of caution compared to the other vehicle image 10B showing the other vehicle present in the own lane. Note that in FIG. 12, for convenience of explanation, dots are added to the other vehicle image 10C instead of adding a predetermined color.

[0125] (others) Although the embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims, and it is understood that these modifications or alterations also naturally fall within the technical scope of the present disclosure.

[0126] Furthermore, the effects described in the above embodiments are explanatory or exemplary and are not limited to those described in the above embodiments. In other words, the technology according to the present disclosure may achieve other effects that are obvious to a person skilled in the art of the present disclosure from the description in the above embodiments, in addition to or instead of the effects described in the above embodiments.

[0127] The processes described in the above embodiments can also be realized by dedicated hardware circuits, in which case they may be executed by a single piece of hardware or by multiple pieces of hardware.

[0128] In the above embodiment, the display control program 52A, the map data 52B, the object detection model 52C, the illustration data 52D, and the color data 52E are stored in the ROM 52. However, this is not limiting, and these data may be stored in the storage 54, or each data may be stored in the ROM 52 and the storage 54 in a distributed manner.

[0129] In the above embodiment, the specific target is a human being classified as a child or elderly person, but the specific target is not limited to this. For example, instead of or in addition to the human being, an animal classified as a small animal such as a dog or a cat, or a vehicle classified as a motorcycle, may be the specific target. In other words, the specific target, which is a target having a predetermined attribute, is not limited to those shown in the above embodiment. The attribute may include the child or elderly person class among humans, the small animal class among animals such as a dog or a cat, and the motorcycle class among vehicles.

[0130] In the above embodiment, the predetermined color is black with the same RGB value (0:0:0) as the characteristic color, but the predetermined color and the characteristic color do not have to be the same color. For example, the three attributes of the predetermined color and the characteristic color may be different as long as the predetermined color is the color with the smallest color difference from the characteristic color among the multiple colors indicated in color data 52E.

[0131] In the above embodiment, when the specific target image is an image of a human, the part to be colored in the predetermined color is the part corresponding to the upper body, but this is not limiting. For example, when the specific target image is an image of a human, the part to be colored in the predetermined color may be the part corresponding to the head or the part corresponding to the lower body.

[0132] In the above embodiment, as an example of a mode according to the attribute of the specific target, the display size of the specific target image on the first display unit 24 is made smaller than that of images showing people classified into other classes, but this is not limiting. For example, if the specific target is a person classified into the elderly person class, as an example of a mode according to the attribute of the specific target, the image showing the elderly person displayed as the specific target image may be holding a predetermined item (for example, a cane).

[0133] In the above embodiment, there are no particular limitations on the viewpoint coordinates and viewpoint angle that constitute the virtual viewpoint when displaying the vehicle image 10A or the like in the display area X. Hereinafter, with reference to Fig. 13, an example will be described in which a person classified into a child or elderly person class as a predetermined attribute is used as a specific target.

[0134] 13 is a fifth explanatory diagram showing an example of a display displayed in the display area X of the first display unit 24. As an example, FIG. 13 shows the display content of the display area X when the vehicle 10 is traveling on an ordinary road similar to that shown in FIG. 11.

[0135] 13 displays a vehicle image 10A, another vehicle image 10B, an ordinary road image 65, shift information 70, vehicle speed information 72, and a pedestrian image 82. Note that, of the above information, the information other than the pedestrian image 82 is the same as in FIG. 11, and therefore description thereof will be omitted.

[0136] Here, the virtual viewpoint looking down on the vehicle image 10A, etc. in Fig. 13 is different from the virtual viewpoint looking down on the vehicle image 10A, etc. in Fig. 7. As an example, the virtual viewpoint in Fig. 13 is a viewpoint looking down diagonally on the vehicle image 10A from a position behind the vehicle in the traveling direction, and is a viewpoint looking at the vehicle image 10A from a lower position than the virtual viewpoint in Fig. 7.

[0137] Furthermore, pedestrian image 82 is a specific target image showing an 8-year-old child approaching the general road shown in general road image 65. As the child is approaching the general road in this way, in Fig. 13 , pedestrian image 82 is displayed outside general road image 65 from a viewpoint seen from the virtual viewpoint.

[0138] Pedestrian image 82 is an illustration image in which a portion of the illustration design of a person shown in illustration data 52D is colored in a predetermined color (e.g., black). Here, pedestrian image 82 can be divided into head 82A, upper body 82B, and lower body 82C. Head 82A is the part corresponding to the head of the person shown in pedestrian image 82. Upper body 82B is the part of the person from the neck to the hip joints. Lower body 82C is the part of the person below the hip joints. In FIG. 13, pedestrian image 82 has head 82A and upper body 82B colored in a predetermined color (e.g., black), and lower body 82C is displayed in a predetermined manner (e.g., white).

[0139] In the above embodiment, the first display unit 24, which is a meter display, is an example of a "display unit" in the present disclosure, but the "display unit" is not limited to a meter display. For example, the "display unit" may be another display such as a center display or a head-up display (HUD). Furthermore, the "display unit" may be a combination of multiple displays such as a meter display and a center display.

[0140] In the above embodiment, the display control device 50 executes the specific processing shown in FIG. 6 . However, the present invention is not limited to this, and the specific processing may be executed by the display control device 50 in cooperation with another ECU. For example, the results of the specific processing are displayed on the first display unit 24 and the second display unit 25. In this case, the display control device 50 may control the first display unit 24 of the display device 21, and the other ECU may control the second display unit 25. In this case, the specific processing may be executed by a plurality of ECUs, including the display control device 50 and the other ECUs, in cooperation with each other, or may be executed by either the display control device 50 or one of the other ECUs as a representative.

[0141] In the above embodiment, various processes executed by the CPU 21 after reading software (programs) may be executed by various processors other than a CPU. Examples of such processors include programmable logic devices (PLDs) such as field-programmable gate arrays (FPGAs), whose circuit configuration can be changed after fabrication, and application-specific integrated circuits (ASICs), which are dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processes. Furthermore, various processes may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0142] In the above embodiment, the display control program 52A is pre-stored (installed) in the ROM 52, but this is not limiting. The display control program 52A may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The display control program 52A may also be downloaded from an external device via a network. The technology disclosed herein may also be applied to programs and program products. [Explanation of symbols]

[0143] 10 vehicles 10A vehicle image 24 1st display section (display section) 50 Display control device 51B Control section 52A Display control program

Claims

1. a control unit that displays, on a display unit, a vehicle image showing the vehicle as seen from a virtual viewpoint based on detection information detectable by the vehicle, and displays, on the display unit, a specific target image showing a specific target present around the vehicle in a predetermined color that imitates a characteristic color derived from the appearance of the specific target; Display control device.

2. the control unit displays a portion of the specific target image in the predetermined color, and displays the remaining portion in a predetermined manner. The display control device according to claim 1 .

3. the control unit selects, as the predetermined color, a color having the smallest color difference from the characteristic color from among a plurality of colors prepared in advance; The display control device according to claim 1 .

4. The specific target is a target having a predetermined attribute. The display control device according to claim 1 .

5. the control unit causes the display unit to display the specific target image in a manner according to the attribute of the specific target. The display control device according to claim 4 .

6. When the control unit causes the display unit to display, together with the specific target image, another target image indicating another target different from the specific target based on the detection information, the control unit causes the other target image to be displayed in a predetermined manner. The display control device according to claim 1 .

7. The display unit is a meter display provided in front of the driver's seat of the vehicle. The display control device according to claim 1 .

8. the control unit causes the display unit to display, in the predetermined color, a specific target image indicating a target having the highest level of caution among the specific targets. The display control device according to claim 1 .

9. a display unit that displays a vehicle image showing the vehicle as seen from a virtual viewpoint based on detection information that can be detected by the vehicle, and a specific target image that shows a specific target that exists around the vehicle, in a predetermined color that imitates a characteristic color derived from the appearance of the specific target, on the display unit; A display control method in which processing is performed by a computer.

10. a display unit that displays a vehicle image showing the vehicle as seen from a virtual viewpoint based on detection information that can be detected by the vehicle, and a specific target image that shows a specific target that exists around the vehicle, in a predetermined color that imitates a characteristic color derived from the appearance of the specific target, on the display unit; A display control program that causes a computer to execute processing.

Images