Display control device, display device, display system, vehicle, display control method, and non-transitory storage medium

Abstract

The present application provides a display control device, a display device, a display system, a vehicle, a display control method, and a non-transitory storage medium that appropriately deliver information of a detected object when the object is detected at an outer side of an image displayed in a display area. The display control device includes a position information acquisition unit that acquires position information of an object based on detection information of an object detection unit that detects an object in a periphery of a vehicle; a virtual origin setting unit that sets a virtual origin within an output range of an output unit that outputs an image to a display area provided in a passenger compartment; a virtual line setting unit that sets a virtual line that links the virtual origin and a position acquired by the position information acquisition unit; and a mark generation unit that generates a mark based on the virtual line, the mark being superimposed and displayed in the display area and alerting attention toward the object.

Description

Technical Field

[0001] This disclosure relates to a display control device, a display device, a display system, a vehicle, a display control method, and a non-transitory storage medium. Background Technology

[0002] There has long been a display device, such as a tilting display, which projects an image onto a vehicle's windshield or similar surface to overlay a virtual image of an AR (Augmented Reality) image onto the foreground of the vehicle.

[0003] For example, Japanese Patent Application Publication No. 2021-037916 discloses a technique that uses relative position information of the object to be indicated, obtained by a detection information acquisition unit, to depict a shape extending in a thin, straight strip from the main image portion toward the outside of the image. This provides the driver with an easily understandable indication of the relative position of the object to be indicated on the outside of the image.

[0004] In display devices such as upward-viewing displays, when an object is detected at the outer edge of the image, it is necessary to appropriately transmit information about the detected object. In contrast, the technology disclosed in Japanese Patent Application Publication No. 2021-037916 only depicts a shape extending in a straight, thin strip towards the outer edge of the image, and there is room for improvement in how to appropriately transmit information about the detected object when it is detected at the outer edge of the image displayed in the display area. Summary of the Invention

[0005] When an object is detected outside an image displayed in the display area, this disclosure will appropriately transmit information about the detected object.

[0006] The display control device according to the first method includes: a position information acquisition unit that acquires position information of an object based on detection information from an object detection unit that detects objects around the vehicle; a virtual origin setting unit that sets a virtual origin within the output range of an output unit that outputs an image to a display area set inside the vehicle compartment; an imaginary line setting unit that sets an imaginary line connecting the virtual origin and the position of the position information acquired by the position information acquisition unit; and a marker generation unit that generates a marker based on the imaginary line, the marker being overlaid in the display area and facing the object to draw attention.

[0007] According to the display control device of the first method, by using an imaginary line connecting a virtual origin set within the output range of the output unit to the position of the object, a marker pointing towards the object is overlaid and displayed. This ensures that even when the object detection unit detects an object outside the output range of the output unit, the marker pointing towards the object is still overlaid and displayed in the display area. Therefore, when an object is detected outside the output range of the output unit, information about the direction in which the detected object exists can be overlaid and displayed in the display area.

[0008] As a result, it is possible to appropriately transmit targets located outside the output range of the output unit to the occupant. Here, the output range of the output unit refers to the range of the image that can be displayed in the display area.

[0009] The display control device involved in the second method is, in the display control device of the first method, the mark generation unit generates a plurality of marks on the imaginary line.

[0010] According to the display control device involved in the second method, multiple markers are generated on an imaginary line, causing the multiple markers to be displayed overlappingly towards the object. Therefore, the markers can be set as indicators of the direction in which the object exists. As a result, information about the direction in which the object exists can be appropriately transmitted.

[0011] The third method involves a display control device in which, in the first or second method, the identifier generation unit generates an identifier that is closer to the virtual origin as larger than an identifier that is farther from the virtual origin.

[0012] According to the display control device involved in the third method, since the markers located closer to the virtual origin are generated to be larger compared to the markers located farther away, they are set to represent a sense of depth (proximity). Therefore, information about the orientation of the object can be conveyed more appropriately.

[0013] The fourth method involves a display control device in any of the first to third methods, where the virtual origin setting unit sets the virtual origin to a position offset from the viewpoint of the vehicle occupant toward the front of the vehicle.

[0014] According to the display control device involved in the fourth method, since the virtual origin is set to a position offset from the occupant's viewpoint towards the front of the vehicle, the virtual origin is set based on the occupant's viewpoint. Therefore, the imaginary line will be set according to the occupant's viewpoint. As a result, the sign can be overlaid and displayed at an appropriate position in the display area according to the occupant's viewpoint.

[0015] The display control device involved in the fifth method is, in any one of the first to fourth methods, the identifier generation unit generates at least one identifier within the output range at a position offset from the virtual origin toward the position of the position information obtained by the position information acquisition unit.

[0016] According to the display control device of the fifth method, since at least one identifier is generated at a position offset from the virtual origin within the output range of the output unit, when the object exists outside the output range of the output unit, at least one identifier will be overlaid and displayed in the display area. Therefore, when the object exists outside the output range of the output unit, information about the direction in which the object exists can be transmitted.

[0017] The sixth method involves a display control device in any of the first to fifth methods, wherein the identifier generation unit generates the identifier in either the left or right side relative to the virtual origin.

[0018] According to the display control device involved in the sixth method, when the object is located outside the output range of the output unit and on the right side of the vehicle, a mark is displayed on the right side relative to the virtual origin. When the object is located outside the output range of the output unit and on the left side of the vehicle, a mark is displayed on the left side relative to the virtual origin. Therefore, information on whether the object is on the right or left side of the vehicle will be readily apparent. As a result, the speed at which occupants can identify the direction of an object located outside the output range of the output unit can be improved.

[0019] The seventh method involves a display control device in any one of the first to sixth methods, wherein the mark generation unit generates the mark as a three-dimensional arrow mark indicating the direction of the object.

[0020] According to the display control device involved in the seventh method, by generating the mark as a three-dimensional arrow mark, the three-dimensional arrow mark is superimposed and displayed towards the object. Therefore, information about the direction in which the object exists can be transmitted more appropriately.

[0021] The display control device involved in the eighth method is a display control device in any one of the first to seventh methods, which includes a size detection unit, the size detection unit detects the size of the object, and the mark generation unit generates the mark based on the size of the object detected by the size detection unit.

[0022] According to the display control device of the eighth method, the identifier is generated based on the size of the object. For example, if the object is a vehicle with a large lateral width, it will be set as an identifier with a large lateral width. On the other hand, if the object is a person with a small lateral width, it will be set as an identifier with a small lateral width. Therefore, it is possible to set the size of the identifier to correspond to the size of the object. As a result, information about the size of the object located outside the output range of the output unit can be transmitted.

[0023] The display device involved in the ninth method includes: any one of the first to eighth methods of display control device; an output unit that outputs an image; and a display area that is disposed in the carriage and overlays the image output by the output unit.

[0024] According to the display device involved in the ninth method, it is possible to appropriately transmit the target located outside the image of the output section to the occupant.

[0025] The display system according to the tenth method includes: the display device of the ninth method; and an object detection unit having a detection range that is larger than the output range of the image output by the output unit, and detecting the objects around the vehicle.

[0026] According to the display system involved in the tenth method, it is possible to appropriately transmit objects located outside the output range of the output unit to the occupants.

[0027] The vehicle involved in the eleventh method includes: the display system of the tenth method; and a windshield that constitutes the display area.

[0028] According to the vehicle involved in the eleventh method, since the markings are displayed in the display area of ​​the windshield, occupants can recognize the markings while keeping their eyes forward. Therefore, occupants can recognize the markings without taking their eyes off the road while driving.

[0029] The display control method involved in the twelfth method performs the following processing by a processor: obtaining the position information of the object based on the detection information of the object detection unit that detects the objects around the vehicle; setting a virtual origin within the output range of the output unit that outputs an image to the display area set in the passenger compartment; setting an imaginary line connecting the virtual origin and the position information; generating an indicator based on the imaginary line, which is overlaid in the display area and faces the object to draw attention.

[0030] According to the display control method involved in the twelfth method, it is possible to appropriately transmit objects located outside the output range of the output unit to the occupants.

[0031] The thirteenth method involves a non-temporary storage medium storing a program that causes a processor to perform the following processing: obtaining the position information of an object based on detection information from an object detection unit that detects objects around the vehicle; setting a virtual origin within the output range of an output unit that outputs an image to a display area set inside the vehicle compartment; setting an imaginary line connecting the virtual origin and the position information; and generating an identifier based on the imaginary line, which is overlaid on the display area set inside the vehicle compartment and directed toward the object to draw attention.

[0032] According to the non-temporary storage medium involved in the thirteenth method, it is possible to appropriately transmit objects located outside the output range of the output unit to the crew.

[0033] According to this disclosure, when an object is detected outside an image displayed in a display area, information about the detected object can be appropriately transmitted. Attached Figure Description

[0034] Embodiments of the present invention will be described in detail with reference to the following figures, wherein:

[0035] Figure 1 This is a top-view view of a vehicle equipped with the display system according to the first embodiment, and it shows the case where the object is outside the projection range of the projection unit.

[0036] Figure 2 This is a schematic diagram showing the view of the front side of a vehicle from inside the passenger compartment of a vehicle equipped with the display system according to the first embodiment.

[0037] Figure 3 This is a block diagram illustrating the hardware structure of the display system according to the first embodiment.

[0038] Figure 4 This is a block diagram illustrating the functional structure of the display control device according to the first embodiment.

[0039] Figure 5 This is a top-view view of a vehicle equipped with the display system according to the first embodiment, and a view showing the object within the projection range of the projection unit.

[0040] Figure 6 This diagram shows a display example of the display area of ​​the display device according to the first embodiment, and also shows the case where the object is located outside the projection range of the projection section.

[0041] Figure 7 This diagram shows a display example of the display area of ​​the display device according to the first embodiment, and also shows the case where the object is within the projection range of the projection section.

[0042] Figure 8 This is a top-view view of a vehicle equipped with the display system according to the first embodiment, and it shows the case where the object is outside the projection range of the projection unit.

[0043] Figure 9 This diagram shows a display example of the display area of ​​the display device according to the first embodiment, and also shows the case where the object is located outside the projection range of the projection section.

[0044] Figure 10 The diagram shows a display example of the display area of ​​the display device according to the first embodiment, and also shows the case where the bus, as an object, is located outside the projection range of the projection unit.

[0045] Figure 11 This is a flowchart illustrating the display processing flow of the display control device according to the first embodiment.

[0046] Figure 12 This diagram illustrates a display example of the display device according to the second embodiment. Detailed Implementation

[0047] [First Implementation]

[0048] Hereinafter, the display system according to the first embodiment will be described with reference to the accompanying drawings. Furthermore, Figure 1 The arrow marker FR indicates the front side of the vehicle, and the arrow marker RH indicates the right side of the vehicle. Furthermore, in the first embodiment, an example of the vehicle 10 traveling forward at speed V will be described.

[0049] [Structure of a vehicle equipped with a display system]

[0050] like Figure 1 As shown, a camera 12 serving as a target detection unit is provided in the vehicle 10. The camera 12 is mounted on the front bumper of the vehicle 10 and is capable of capturing images of the area in front of the vehicle. The camera 12 is configured to have a shooting range 12A that serves as the detection range.

[0051] As object 90, it can be set as pedestrian 92 (refer to...) Figure 1 ) or bus 94 (see reference) Figure 10 In addition, the object 90 can also be a vehicle traveling in front of the vehicle, a vehicle traveling in the opposite lane, a parked vehicle, or other obstacles.

[0052] like Figure 2 As shown, an instrument panel 18 and a windshield 22 are provided on the front side of the passenger compartment in vehicle 10.

[0053] (Dashboard 18)

[0054] The dashboard 18 is arranged to extend in the width direction of the vehicle. A steering wheel 20 is provided on the right side of the dashboard 18. That is, in the first embodiment, as an example, the vehicle 10 is configured as a right-hand drive vehicle with the steering wheel 20 provided on the right side of the vehicle.

[0055] The dashboard 18 is provided with a first display unit 28 as an output unit and a second display unit 30 as an output unit, wherein the first display unit 28 has a first display area G1 as a display area and the second display unit 30 has a second display area G2 as a display area.

[0056] The first display unit 28 is located on the right side of the instrument panel 18 and in front of the steering wheel 20. The first display unit 28 is configured, for example, as a speedometer that displays the vehicle speed, or an instrument display that displays directional indicators and warnings.

[0057] The second display unit 30 is positioned at the center of the instrument panel 18 in the vehicle width direction. The second display unit 30 is configured, for example, as a central display showing images output by the navigation system.

[0058] (Windshield 22)

[0059] The windshield 22 is supported by a front pillar 24. The front pillar 24 is located at the front, right, and left sides of the vehicle body and extends in a generally vertical direction. The upper part of the windshield 22 is covered by a headliner 25 mounted on the roof panel.

[0060] A third display unit 32 is provided on the windshield 22, and the third display unit 32 has a third display area G3 as a display area.

[0061] Furthermore, based on information output from the display control device 35 located inside the dashboard 18, the image projected from the projection unit 51 (which serves as the output unit) is projected onto the third display unit 32 on the windshield 22. Thereby, the image is superimposed as a virtual image onto the virtual image plane T (see reference ) in front of the occupant (driver) H. Figure 1 )superior.

[0062] The display device 34, which includes a display control device 35, a projection unit 51, and a third display unit 32, constitutes a bottom-view display device. The third display unit 32, on which images projected from the projection unit 51 are projected, constitutes the projection surface of the bottom-view display device.

[0063] like Figure 1As shown, the projection unit 51 has a projection range 51A that is narrower than the shooting range 12A of the camera 12. The projection range 51A of the projection unit 51, which is the output unit, is configured with the viewpoint E of the occupant H as the origin. The distance D from the viewpoint E of the occupant H to the virtual image plane T can be set to, for example, 20m. Furthermore, the distance D from the viewpoint E of the occupant H to the virtual image plane T can be appropriately changed depending on the vehicle equipped with the display device 34.

[0064] like Figure 3 As shown, the display device 34 and the camera 12 constitute the display system 16.

[0065] [Hardware Structure of the Display System]

[0066] like Figure 3 As shown, the display system 16 inputs the camera image captured by the camera 12 into the display control device 35, and the processed information processed by the display control device 35 is output to the first display unit 28, the second display unit 30, and the projection unit 51.

[0067] Camera 12 captures images of the area in front of the vehicle. The images captured by camera 12 of the area in front of the vehicle are input into the display control device 35.

[0068] The display control device 35 is configured as an ECU (Electronic Control Unit) that implements various controls. The display control device 35 is configured to include a CPU (Central Processing Unit) 36, a ROM (Read Only Memory) 38, a RAM (Random Access Memory) 40, a memory 42, a communication interface (communication I / F) 44, and an input / output interface (input / output I / F) 46. All components are connected together via a bus 48 in a manner enabling communication between them.

[0069] CPU 36 is a central processing unit that executes various programs or controls various components. Specifically, as a processor, CPU 36 reads programs from ROM 38 (memory) or memory 42 (memory), and executes the programs using RAM 40 as its working area. Furthermore, CPU 36 performs control of the aforementioned structures and various arithmetic operations based on the programs recorded in ROM 38 or memory 42.

[0070] ROM 38 stores various programs and data. RAM 40 serves as a working area for temporary storage of programs or data. Storage 42 is a non-temporary recording medium, constructed of HDD (Hard Disk Drive) or SSD (Solid State Drive), that stores various programs, including the operating system, and various data. In the first embodiment, ROM 38 or storage 42 stores programs, etc., for implementing the display processing described later.

[0071] The input / output interface 46 is connected to a first display unit 28, a second display unit 30, a projection unit 51, and a camera 12.

[0072] [Functional Structure of the Display Control Device]

[0073] The function of the display control device 35 in the first embodiment will be explained.

[0074] like Figure 4 As shown, the display control device 35 functionally includes a position information acquisition unit 54, a virtual origin setting unit 56, an imaginary line setting unit 58, a size detection unit 60, and a mark generation unit 62. Additionally, Figure 4 The functional structures shown are implemented, for example, by having the CPU 36 execute programs stored in the ROM 38 or the memory 42.

[0075] (Location Information Acquisition Department)

[0076] The location information acquisition unit 54 acquires the location information of the object 90 based on the camera image of the camera 12 that has captured the object 90 in front of the vehicle 10.

[0077] Specifically, such as Figure 1 As shown, the location information acquisition unit 54 acquires the position P2 of the pedestrian 92 based on a camera image obtained by the camera 12 capturing a pedestrian 92 located outside the projection range 51A of the projection unit 51. Furthermore, as... Figure 5 As shown, the location information acquisition unit 54 acquires the position P2 of the pedestrian 92 based on the camera image obtained by the camera 12 capturing the pedestrian 92 inside the projection range 51A of the projection unit 51.

[0078] (Virtual Origin Setting Department)

[0079] The virtual origin setting unit 56 sets a virtual origin P1 inside the projection range 51A of the projection unit 51, which is narrower than the shooting range 12A of the camera 12.

[0080] Specifically, such as Figure 1 as well as Figure 5 As shown, the virtual origin setting unit 56 sets the position P1 as a distance D offset from the viewpoint E of the occupant H of the vehicle 10 toward the front of the vehicle.

[0081] In addition, the virtual origin P1 can be set as the intersection of the line K extending from the viewpoint E of the occupant H of vehicle 10 to the front of the vehicle and the virtual image plane T.

[0082] (Imaginary Line Setting Department)

[0083] The imaginary line setting unit 58 sets an imaginary line connecting the virtual origin P1 and the position P2 obtained by the position information acquisition unit 54. Specifically, as follows: Figure 1 as well as Figure 5 As shown, the imaginary line setting unit 58 sets an imaginary line 66 that connects the virtual origin P1 and the position P2 of the pedestrian 92.

[0084] Furthermore, the position P2 of pedestrian 92 can be set to a point on the horizontal plane, including the virtual origin P1. In other words, the virtual origin P1 and the position P2 of pedestrian 92 can be set to the same height.

[0085] (Dimensioning Inspection Department)

[0086] The size detection unit 60 detects the size of the object 90. Specifically, the size detection unit 60 detects the size of the object 90 reflected in the camera image captured by the camera 12. The size detection unit 60 can detect at least one of the width, height, and length of the object 90 reflected in the camera image.

[0087] (Identification Generation Department)

[0088] The label generation unit 62 generates a label 68 (see reference) that points toward the object 90 based on the imaginary line 66. Figure 1 The marker 68, which points to the object 90 to draw attention, is defined as a marker indicating the position of the object 90.

[0089] <The situation where the pedestrian is outside the projection range of the projection unit>

[0090] Specifically, such as Figure 1 As shown, when pedestrian 92 is outside the projection range 51A of projection unit 51, the mark generation unit 62 generates a first mark 70 as mark 68 at a position offset by a distance L1 from the virtual origin P1 toward the position P2 obtained by position information acquisition unit 54, inside the projection range 51A of projection unit 51.

[0091] The identifier generation unit 62 generates a third identifier 74 as identifier 68 at a position offset by a distance L2 from the virtual origin P1 from the position P2 obtained from the position information acquisition unit 54.

[0092] The identifier generation unit 62 generates a second identifier 72 as an identifier 68 at a position offset by a distance L3 from the position P2 obtained by the position information acquisition unit 54 starting from the first identifier 70 and offset by a distance L3 from the virtual origin P1 starting from the third identifier 74.

[0093] That is, the identifier generation unit 62 generates multiple identifiers 68 at equal intervals on the imaginary line 66.

[0094] like Figure 3 As shown, the image information of the identifier 68 generated by the identifier generation unit 62 is output to the projection unit 51, and the image projected from the projection unit 51 is projected onto the third display unit 32 of the windshield 22.

[0095] Specifically, such as Figure 6 As shown, when pedestrian 92 is outside the projection range 51A of projection unit 51, the first sign 70 and the second sign 72 are superimposed as virtual images on the third display area G3 of third display unit 32. That is, the first sign 70 and the second sign 72 are superimposed as virtual images on the foreground of windshield 22. Furthermore, although in Figure 6 For reference, the virtual origin P1, imaginary line 66, and third identifier 74 are marked with dashed lines, but in reality, these information can be omitted.

[0096] <Situation where pedestrians are within the projection range of the projection unit>

[0097] On the other hand, such as Figure 5 As shown, when pedestrian 92 is inside the projection range 51A of projection unit 51, the mark generation unit 62 generates a first mark 70 as mark 68 at a position offset by a distance L1 from the virtual origin P1 toward the position P2 obtained by position information acquisition unit 54, inside the projection range 51A of projection unit 51.

[0098] The identifier generation unit 62 generates a third identifier 74 as identifier 68 at a position offset by a distance L2 from the virtual origin P1 from the position P2 obtained from the position information acquisition unit 54.

[0099] The identifier generation unit 62 generates a second identifier 72 as an identifier 68 at a position offset by a distance L4 from the position P2 obtained by the position information acquisition unit 54 starting from the first identifier 70 and offset by a distance L4 from the virtual origin P1 starting from the third identifier 74.

[0100] That is, the mark generation unit 62 generates a plurality of marks 68 at equal intervals on the imaginary line 66. Furthermore, the mark generation unit 62 adjusts the distance between the plurality of marks 68 according to the distance between them and the object 90.

[0101] Moreover, such as Figure 7 As shown, when pedestrian 92 is inside the projection range 51A of projection unit 51, the first sign 70, the second sign 72, and the third sign 74 are superimposed as virtual images on the third display area G3 of the third display unit 32. That is, the first sign 70, the second sign 72, and the third sign 74 are superimposed as virtual images on the foreground of the windshield 22. Furthermore, although in Figure 7 For reference, the virtual origin P1, the imaginary line 66, and the position P2 obtained by the position information acquisition unit 54 are recorded using dashed lines, but in reality, these information may not be displayed.

[0102] (Further characteristic structures)

[0103] The identifier generation unit 62 can generate identifier 68 on either the left or right side relative to the virtual origin P1. Specifically, as follows... Figure 1 As shown, when pedestrian 92 is located outside the left side of the vehicle within the projection range 51A of the projection unit 51, the sign generation unit 62 generates a sign 68 on the left side relative to the virtual origin P1. In this case, as... Figure 6 As shown, the first identifier 70 and the second identifier 72 are displayed overlapping on the left side relative to the virtual origin P1.

[0104] like Figure 8 As shown, when pedestrian 92 is located outside the right side of the vehicle within the projection range 51A of the projection unit 51, the sign generation unit 62 generates a representation 68 on the right side relative to the virtual origin P1. In this case, as... Figure 9 As shown, a portion of the first identifier 70, the second identifier 72, and the third identifier 74 are displayed overlapping on the right side relative to the virtual origin P1.

[0105] The identifier generation unit 62 generates an identifier 68 located closer to the virtual origin P1 that is larger than an identifier 68 located farther from the virtual origin P1. Specifically, as... Figure 1 as well as Figure 5 As shown, the identifier generation unit 62 generates a first identifier 70 located closer to the virtual origin P1, which is larger than the second identifier 72 located farther from the virtual origin P1. The identifier generation unit 62 generates a second identifier 72 located farther from the virtual origin P1, which is larger than the third identifier 74 located farther from the virtual origin P1.

[0106] In this case, such as Figure 6 as well as Figure 7 As shown, the first identifier 70 is overlapped with the second identifier 72 in a manner that is larger than the second identifier 72, and the second identifier 72 is overlapped with the third identifier 74 in a manner that is larger than the third identifier 74.

[0107] The marking generation unit 62 can generate the marking 68 as a three-dimensional arrow mark indicating the direction of the marking 90.

[0108] The label generation unit 62 can generate a label 68 based on the size of the object 90 detected by the size detection unit 60. Specifically, as follows: Figure 1 as well as Figure 5 As shown, when the object 90 is a pedestrian 92, the sign generation unit 62 can generate a sign 68 with a width corresponding to the horizontal width of the pedestrian 92. In this case, as... Figure 6 as well as Figure 7 As shown, the label 68, which corresponds to the width of the pedestrian 92, is displayed overlapping.

[0109] On the other hand, when the object 90 is a bus 94, the sign generation unit 62 can generate a sign 68 with a width corresponding to the lateral width of the bus 94. In this case, as... Figure 10 As shown, the label 68, which corresponds to the width of the bus 94, is displayed in an overlapping manner.

[0110] The label generation unit 62 can also generate a label indicating the speed V1 of the vehicle 10, and as shown in the image. Figure 6 As shown, the indicator 76 showing the speed V1 of the vehicle 10 is overlaid on the third display area G3 of the third display unit 32.

[0111] [The process of display processing performed by the display device]

[0112] based on Figure 11 The flowchart shown illustrates the display processing flow performed by the display device 34.

[0113] like Figure 11 As shown, when the display processing begins, the position information acquisition unit 54 acquires the position P2 of the object 90 based on the image of the camera 12 (step S101).

[0114] Next, the size detection unit 60 detects the size of the object 90 reflected in the camera image captured by the camera 12 (step S102).

[0115] Next, the virtual origin setting unit 56 sets a virtual origin P1 inside the projection range 51A of the projection unit 51 (step S103).

[0116] Next, the imaginary line setting unit 58 sets an imaginary line that connects the virtual origin P1 and the position P2 obtained by the position information acquisition unit 54 (step S104).

[0117] Next, the identifier generation unit 62 generates identifier 68 based on imaginary line 66 (step S105).

[0118] Next, the information of the identifier 68 generated by the identifier generation unit 62 is output to the projection unit 51, and the identifier 68 is superimposed as a virtual image in the third display area G3 of the third display unit 32 (step S106), and the display process ends.

[0119] [Effect of the first implementation method]

[0120] Next, the function and effects of the first embodiment will be explained.

[0121] The display control device 35 of the first embodiment includes: a position information acquisition unit 54, which acquires the position P2 of the object 90 based on a camera image of a camera 12 that captures an object 90 in front of the vehicle 10; a virtual origin setting unit 56, which sets a virtual origin P1 inside the projection range 51A of a projection unit 51, which has a detection range narrower than the shooting range 12A of the camera 12; an imaginary line setting unit 58, which sets an imaginary line 66 connecting the virtual origin P1 and the position P2 acquired by the position information acquisition unit 54; and a mark generation unit 62, which generates a mark 68 based on the imaginary line 66, the mark 68 being overlaid and displayed in a display area G3 provided in the passenger compartment and facing the object 90 to draw attention. (See reference) Figure 4 ).

[0122] By using an imaginary line 66 connecting the virtual origin P1, set inside the projection range 51A of the projection unit 51, and the position P2 of the target 90, the marker 68 pointing towards the target 90 is overlaid and displayed. This ensures that even if the camera 12 captures the target 90 outside the projection range 51A of the projection unit 51, the marker 68 pointing towards the target 90 can still be overlaid and displayed in the third display area G3. Therefore, when the camera 12 captures the target 90 outside the projection range 51A of the projection unit 51, i.e., outside the image displayed in the third display area G3, information about the direction in which the captured target 90 is located can be overlaid and displayed in the third display area G3. As a result, the occupant H can recognize the target 90 located outside the projection range 51A of the projection unit 51.

[0123] In the display control device 35 of the first embodiment, the mark generation unit 62 generates a plurality of marks 68 at equal intervals on the imaginary line 66 (see reference). Figure 1 as well as Figure 5 ).

[0124] By generating multiple markers 68 at equal intervals along an imaginary line, the markers 68 can be overlapped and displayed at equal intervals, facing the object 90. Therefore, the markers 68 can be set as indicators of the direction in which the object 90 exists. As a result, information about the direction in which the object 90 exists can be appropriately conveyed.

[0125] In the display control device 35 of the first embodiment, the identifier generation unit 62 generates a first identifier 70 located closer to the virtual origin P1 that is larger than a second identifier 72 located farther from the virtual origin P1 (see reference). Figure 1 as well as Figure 5 ).

[0126] Because the first identifier 70, which is closer to the virtual origin P1, is generated to be larger compared to the second identifier 72, which is farther away, therefore, Figure 6 as well as Figure 7 As shown, the first marker 70, located closer to the virtual origin P1, is displayed as larger compared to the second marker 72, located further away. Therefore, the marker 68 can be configured to convey a sense of depth (proximity). As a result, information about the orientation of the object 90 can be conveyed more appropriately.

[0127] In the display control device 35 of the first embodiment, the virtual origin setting unit 56 sets the position offset from the viewpoint E of the occupant H of the vehicle 10 toward the front of the vehicle 10 as the virtual origin P1 (see reference). Figure 1 as well as Figure 5 ).

[0128] Since the virtual origin P1 is set to a position offset from the occupant H's viewpoint E towards the front of the vehicle, the virtual origin P1 will be set based on the occupant H's viewpoint E. Therefore, the imaginary line 66 will be set according to the occupant H's viewpoint E. As a result, the sign 68 can be overlaid and displayed at an appropriate position in the third display area G3 according to the occupant H's viewpoint E.

[0129] In the display control device 35 of the first embodiment, the mark generation unit 62 generates at least one mark 68 (see reference) inside the projection range 51A of the projection unit 51 at a position offset from the virtual origin P1 toward the position P2 of the target 90 obtained by the position information acquisition unit 54. Figure 1 and Figure 5 ).

[0130] Since at least one identifier 68 is generated inside the projection range 51A of the projection unit 51 at a position offset from the virtual origin P1, at least one identifier 68 will be overlaid and displayed in the third display area G3 when the object 90 is outside the projection range 51A of the projection unit 51. Therefore, when the object 90 is outside the projection range 51A of the projection unit 51, information about the direction in which the object 90 exists can be transmitted.

[0131] In the display control device 35 of the first embodiment, the identifier generation unit 62 generates an identifier 68 (see reference) on either the left or right side relative to the virtual origin P1. Figure 6 as well as Figure 9 ).

[0132] like Figure 1 As shown, when the target 90 is located outside the projection range 51A of the projection unit 51 and on the left side of the vehicle, as Figure 6 As shown, the identifier 68 will be displayed on the left relative to the virtual origin P1. Figure 8 As shown, when the target 90 is located outside the projection range 51A of the projection unit 51 and on the right side of the vehicle, as Figure 9 As shown, the marker 68 will be displayed on the right side relative to the virtual origin P1. Therefore, it will be immediately clear whether the object 90 is located on the right or left side of the vehicle. As a result, the occupant H can identify the direction of the object 90, which is located outside the projection range 51A of the projection unit 51, at a faster pace.

[0133] In the display control device 35 of the first embodiment, the mark generation unit 62 generates the mark 68 as a three-dimensional arrow mark indicating the direction of the mark 90 (see reference). Figures 6 to 10 ).

[0134] By generating the identifier 68 as a three-dimensional arrow mark, the three-dimensional arrow mark is displayed overlapping towards the object 90. Therefore, information about the direction in which the object 90 exists can be conveyed more appropriately.

[0135] The display control device 35 in the first embodiment includes a size detection unit 60, which detects the size of an object 90, and a label generation unit 62 generates a label 68 (see reference) based on the size of the object 90 detected by the size detection unit 60. Figure 6 as well as Figure 10 ).

[0136] The identifier 68 is generated based on the size of the object 90, thus, for example, like Figure 10As shown, if the object 90 is an object with a large lateral width, such as a bus 94, it will be designated as a sign 68 with a large lateral width. On the other hand, as... Figure 6 As shown, when the object 90 is an object with a small lateral width, such as pedestrian 92, it will be set as a label 68 with a small lateral width. Therefore, the size of the label 68 can be set to correspond to the size of the object 90. As a result, information about the size of the object 90 located outside the projection range 51A of the projection unit 51 can be appropriately transmitted.

[0137] In the display system 16 of the first embodiment, a display device 34 and a camera 12 are provided. The camera 12 has a shooting range 12A that is larger than the projection range 51A of the projection unit 51, and it captures images of an object 90 in front of the vehicle 10 (see reference). Figure 1 ).

[0138] Since the display system 16 includes a display device 34 and a camera 12 with a shooting range 12A that is larger than the projection range 51A of the projection unit 51, even if the camera 12 captures an object 90 outside the projection range 51A of the projection unit 51, i.e., outside the image displayed in the third display area G3, the information of the direction in which the captured object 90 exists can be overlaid and displayed in the third display area G3. As a result, the object 90 located outside the projection range 51A of the projection unit 51 can be appropriately transmitted to the occupant H.

[0139] In the vehicle 10 of the first embodiment, a third display area G3 (see reference) is provided on the windshield 22. Figure 2 ).

[0140] Since the symbol 68 is displayed in the third display area G3 of the windshield 22, the occupant H can identify the symbol 68 while keeping their eyes forward. Therefore, the occupant H can identify the symbol 68 while driving without taking their eyes off the road.

[0141] [Second Implementation]

[0142] The display system of the second embodiment differs from the display system of the first embodiment in that the display area in which the markers reminding people to pay attention to the object are displayed overlapped is different.

[0143] The structure of the display system according to the second embodiment will be described below. Furthermore, the same terms or symbols will be used to describe parts that are the same as or equivalent to those described in the first embodiment.

[0144] In the second embodiment, such as Figure 12As shown, in the second display area G2 of the second display unit 30, the mark 68 that reminds attention to the object 90 is displayed in an overlapping manner.

[0145] The camera image F captured by camera 12 is displayed on the upper side of the second display area G2, and the current location 10A of vehicle 10 and the path (guide path) R to the destination are displayed on the map via the navigation system on the lower side of the second display area G2.

[0146] The image of the identifier 68 generated by the identifier generation unit 62 is output to the second display unit 30, which serves as the output unit, and the identifier 68 is overlaid on the second display area G2.

[0147] Even with this structure, the same effect as the display system in the first embodiment can be achieved.

[0148] The display system of this disclosure has been described above based on the embodiments described above. However, the specific structure is not limited to these embodiments, and design changes are permitted as long as they do not depart from the spirit of this disclosure.

[0149] In the above embodiment, an example is shown where the position information acquisition unit 54 acquires the position information of the object 90 based on a camera image from a camera 12 that captures an image of the object 90 in front of the vehicle 10. However, the position information acquisition unit may also acquire the position information of the object based on a camera image from a camera that detects objects around the vehicle.

[0150] In the above embodiment, an example is shown where the object detection unit is a camera 12. Furthermore, in the above embodiment, an example is shown where the projection range 51A of the projection unit 51, which is the output unit, is narrower than the shooting range 12A of the camera 12, which is the detection range, and the object detection unit is narrower. However, the object detection unit is not limited to this configuration; for example, it can be a radar, LiDAR (Laser Radar), or sonar. Furthermore, the output range of the output unit is not necessarily narrower than the detection range of the object detection unit. Depending on the configuration of the radar, LiDAR, or sonar, the projection range 51A of the projection unit 51, which is the output unit, may also be larger than the detection range of the radar, LiDAR, or sonar, which is the object detection unit.

[0151] In the above embodiment, an example is shown where the imaginary line setting unit 58 sets an imaginary line 66 connecting the virtual origin P1 and the position P2 of the pedestrian 92 on a horizontal plane including the virtual origin P1. However, the imaginary line setting unit may also set an imaginary line connecting the virtual origin P1 and the position P2 of the pedestrian 92 in three-dimensional space.

[0152] In the above embodiment, an example is shown where the size detection unit 60 detects the size of the object 90 projected onto the camera image based on the camera image captured by the camera 12. However, the size detection unit is not limited to this method; for example, the size of the object can also be detected based on detection information from radar, LiDAR, or sonar.

[0153] In the above embodiment, an example is shown where the marker generation unit 62 changes the distance between multiple markers 68 based on the distance to the object 90. However, the marker generation unit may also set the distance between markers to a fixed value without considering the distance to the object. Furthermore, the marker generation unit may change the color or size of the generated markers based on the distance to the object. This allows information about the distance to the object to be transmitted.

[0154] In the above embodiment, an example is shown in which the markings that draw attention to the object are overlapped when the object 90 is located outside or inside the projection range 51A of the projection unit 51. However, it is also possible to overlap the markings that draw attention to the object when the object is outside the projection range of the projection unit, and overlap the markings that draw attention to the object (marking the position of the object) when the object is inside the projection range of the projection unit.

[0155] In the above embodiment, an example is shown where the mark generation unit 62 generates a mark 68 as a three-dimensional arrow mark indicating the direction of the mark 90. ​​However, the mark generation unit can generate either a planar arrow mark or a shape other than an arrow mark (e.g., a sphere or a triangular prism).

[0156] In the above embodiment, an example is shown where the virtual origin P1 is set as the intersection of a straight line K extending from the viewpoint E of the occupant H of the vehicle 10 towards the front of the vehicle and the virtual image plane T. However, the virtual origin is not limited to this method and can be set within the output range of the output unit.

[0157] In the above embodiment, an example is shown where the label generation unit 62 generates a label 68 based on the lateral width of the object 90 detected by the size detection unit 60. However, the label generation unit can also generate a label based on the height or length of the object detected by the size detection unit.

[0158] In the first embodiment, an example is shown where the image of the logo 68 generated by the logo generation unit 62 is projected onto the third display unit 32 of the windshield 22. In the second embodiment, an example is shown where the image of the logo 68 generated by the logo generation unit 62 is displayed on the second display unit 30. However, the image of the logo generated by the logo generation unit can be displayed on the first display unit 28 or projected onto a synthesizer provided on the upper surface of the dashboard.

[0159] In the above embodiments, an example of applying the display system of this disclosure to a vehicle 10 that is moving forward is shown. However, the display system of this disclosure can also be applied to a vehicle that is driving on a curve, a vehicle that is turning a corner, or a vehicle that is stopped.

[0160] Although the processing performed by the display control device 35 in the above embodiment has been described as software processing implemented by executing a program, it is not limited to this. For example, it may also be a process implemented by hardware. Alternatively, it may be a process that combines both software and hardware. Furthermore, in the case of software processing, the program may be stored in various non-transitory storage media such as CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), USB (Universal Serial Bus) memory, etc., and circulated therein, so that a processor such as CPU 36 can execute it. The program may also be provided by being downloaded from an external device via a network.

Claims

1. A display control device, comprising: The location information acquisition unit acquires the location information of the object based on the detection information of the object detection unit that detects objects around the vehicle; The virtual origin setting unit sets a virtual origin within the output range of the output unit that outputs images to the display area set inside the carriage; The imaginary line setting unit sets an imaginary line that connects the virtual origin to the position of the position information obtained by the position information acquisition unit. The label generation unit generates multiple labels based on the imaginary lines. These labels are overlaid in the display area and oriented towards the object to draw attention. The identifier generation unit changes the distance between the plurality of identifiers based on the distance between itself and the object.

2. The display control device as described in claim 1, wherein, The identifier generation unit generates identifiers that are closer to the virtual origin as larger than identifiers that are farther from the virtual origin.

3. The display control device as described in claim 1, wherein, The virtual origin setting unit sets the position offset from the viewpoint of the vehicle occupants towards the front of the vehicle as the virtual origin.

4. The display control device as described in claim 1, wherein, The identifier generation unit generates at least one identifier within the output range at a position offset from the virtual origin toward the position of the position information obtained by the position information acquisition unit.

5. The display control device as claimed in claim 1, wherein, The identifier generation unit generates the identifier on either the left or right side relative to the virtual origin.

6. The display control device as claimed in claim 1, wherein, The identifier generation unit generates the identifier as a three-dimensional arrow mark indicating the direction of the object.

7. The display control device as claimed in claim 1, wherein, It includes a size detection unit that detects the size of the object. The label generation unit generates the label based on the size of the object detected by the size detection unit.

8. A display device comprising: The display control device according to any one of claims 1 to 7; The output section outputs the image. The display area is located inside the carriage and overlays the images output by the output unit.

9. A display system comprising: The display device according to claim 8; The object detection unit has a detection range that is larger than the output range of the image output by the output unit, and detects the objects around the vehicle.

10. A vehicle, comprising: The display system according to claim 9; The windshield forms the display area.

11. A display control method, wherein, The following processing is performed by the processor: The location information of the object is obtained based on the detection information of the object detection unit that detects objects around the vehicle; A virtual origin is set within the output range of the output unit that outputs images to the display area set inside the carriage; An imaginary line is set to connect the virtual origin and the position information. Multiple markers are generated based on the imaginary line, and the markers are overlaid in the display area and oriented toward the object to draw attention; The distance between the plurality of identifiers is changed based on the distance to the object.

12. A non-transitory storage medium storing a program, wherein, The program causes the processor to perform the following processing: The location information of the object is obtained based on the detection information of the object detection unit that detects objects around the vehicle; A virtual origin is set within the output range of the output unit that outputs images to the display area set inside the carriage. An imaginary line is set to connect the virtual origin and the position information. Multiple markers are generated based on the imaginary line. These markers are overlaid and displayed in a display area set inside the carriage, pointing towards the object to draw attention. The distance between the plurality of identifiers is changed based on the distance to the object.

Images