Display control device, display system, display method, and non-transitory storage medium

Abstract

The present disclosure relates to a display control device, a display system, a display method, and a non-transitory storage medium. The display control device includes a processor configured to: in a first display portion fixed to a mobile body or a second display portion worn on a human body, decide content to be superimposed on a real image visually recognized through the first display portion or the second display portion as an object, select which of the first display portion and the second display portion to display the content based on a prescribed condition, and superimpose the content on the object in the selected first display portion or second display portion.

Description

Technical Field

[0001] This disclosure relates to display control devices, display systems, display methods, and non-transitory storage media. Background Technology

[0002] Japanese Patent Application Publication No. 2019-138773 discloses an AR-HUD (Augmented Reality Head-Up Display) that overlays an image onto a windshield, while U.S. Patent Application Publication No. 2016 / 0097940 discloses a smart contact lens that incorporates a display within the contact lens.

[0003] When using HUDs and contact lenses as AR displays, the field of view of HUDs is limited, and contact lenses cannot achieve the required display accuracy. Summary of the Invention

[0004] This disclosure provides a display control device, display system, display method, and non-transitory storage medium that compensates for the individual characteristics of different display units to provide a display that is easily visually recognizable to the user.

[0005] The first embodiment of the present disclosure provides a display control device comprising a processor configured to: in a first display unit fixed to a movable body or a second display unit worn on a human body, take a real image visually recognized through the first display unit or the second display unit as an object, determine the content to be overlaid on the object, select which of the first display unit and the second display unit to display the content based on predetermined conditions, and make the content overlay the object in the selected first display unit or the second display unit.

[0006] In the display control device of the first embodiment of this disclosure, a processor is configured to display content to a first display unit or a second display unit. Alternatively, the first display unit may be fixed to a moving body such as a vehicle, and the second display unit may be worn on the human body, such as the head or eyes. In this display control device, the processor selects the display unit to display content based on predetermined conditions. Among the selected display units, a real image that can be visually recognized through the display unit is taken as the object, and the content is overlaid on the object. According to this display control device, it is possible to compensate for the characteristics of different display units to provide a display that is easily visually recognizable to the user.

[0007] In the display control device of the first embodiment of this disclosure, the processor may be configured to select which of the first display unit and the second display unit to display the content based on at least one of the following conditions: the object to which the content is to be displayed overlapping, the distance from the moving body to the object, and the display position of the content relative to the respective field of view of the first display unit and the second display unit.

[0008] According to the first aspect of the present disclosure, the display control device can select a display unit suitable for each condition.

[0009] In the display control device of the first embodiment of this disclosure, the processor may be configured to cause the second display unit to display the content when the display position of the content is outside the display area of ​​the first display unit.

[0010] According to the first aspect of the present disclosure, the display control device can compensate for the display of content outside the display area of ​​the first display unit by means of the second display unit.

[0011] In the display control device of the first embodiment of this disclosure, the processor may be configured to cause the second display unit to display the content when the distance from the moving body to the object is greater than a predetermined value.

[0012] According to the first aspect of the present disclosure, the display control device can compensate for the display of content even when the viewing distance of the content in the first display unit is far from the display object.

[0013] The second aspect of the present disclosure includes a display system comprising: a display control device; a head-up display serving as the first display unit, mounted on a vehicle serving as the moving body; and smart contact lenses, included in a second display unit worn on the eyes of a passenger in the vehicle.

[0014] According to the second aspect of the present disclosure, the display system can provide easily visually recognizable displays to passengers by using display units with different characteristics in different parts of the vehicle.

[0015] The third embodiment of the present disclosure includes the following display method: a first step, in a first display unit fixed to a movable body or a second display unit worn on a human body, determining the content to be superimposed on the object, taking a real image visually recognized through the first display unit or the second display unit as the object; a second step, selecting which of the first display unit and the second display unit to display the content based on predetermined conditions; and a third step, superimposing the content on the object in the selected first display unit or the second display unit, wherein the processing of the first step, the second step and the third step is performed by a computer.

[0016] In the third aspect of the display method disclosed herein, a computer is configured to display content to a first display unit or a second display unit. Here, the configurations for the first and second display units are as described above. In this display method, the computer selects the display unit to display the content based on predetermined conditions, and in the selected display unit, the content is overlaid on a real image that is visually recognizable through the display unit. According to this display method, the characteristics of different display units can be compensated for to provide a display that is easily visually recognizable to the user.

[0017] The non-transitory storage medium of the fourth aspect of this disclosure stores a command executable by one or more processors, which causes the one or more processors to perform the following functions: determining, in a first display fixed to a mobile body or a second display worn on a human body, to determine, as an object, a real image visually recognized through the first display or the second display, to overlay content onto the object; selecting, based on predetermined conditions, which of the first display and the second display to display the content; and, in the selected first display or the second display, to overlay the content with the object in a manner that is appropriate.

[0018] The fourth solution disclosed herein non-transitory stores an instruction that causes the processor to perform a function to display content on a first display unit or a second display unit. Here, the first and second display units are as described above. This non-transitory storage medium stores an instruction that causes the processor to perform the following function: select a display unit to display content based on predetermined conditions, and in the selected display unit, overlay the content onto a real image that is visually recognizable through the display unit. According to this non-transitory storage medium, it is possible to compensate for the individual characteristics of different display units to provide a display that is easily visually recognizable to the user.

[0019] According to this disclosure, the characteristics of different display units can be compensated to provide a display that is easy for the user to visually recognize. Attached Figure Description

[0020] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, wherein the same reference numerals denote the same elements, wherein:

[0021] Figure 1 This is a diagram showing the schematic configuration of the display system according to the first embodiment.

[0022] Figure 2 This is a block diagram illustrating the hardware configuration of the vehicle according to the first embodiment.

[0023] Figure 3 This is a block diagram showing the configuration of the ROM in the display control device according to the first embodiment.

[0024] Figure 4 This is a block diagram illustrating the functional configuration of the CPU in the display control device of the first embodiment.

[0025] Figure 5 This is a diagram showing the hardware configuration of the smart contact lens according to the first embodiment.

[0026] Figure 6 This is a flowchart illustrating the display processing flow in the first embodiment.

[0027] Figure 7 This is a flowchart illustrating the selection process in the first embodiment.

[0028] Figure 8 This figure illustrates the display implemented via a head-up display in the first embodiment.

[0029] Figure 9 This is a diagram illustrating the display achieved through smart contact lenses in the first embodiment.

[0030] Figure 10 This is a diagram showing the schematic configuration of the display system according to the second embodiment.

[0031] Figure 11 This is a flowchart illustrating the selection process in the second embodiment. Detailed Implementation

[0032] Hereinafter, the display system 10, which is an embodiment of the present disclosure, will be described with reference to the accompanying drawings.

[0033] [First Implementation Method]

[0034] (Overall Composition)

[0035] like Figure 1As shown, the display system 10 of the first embodiment is configured to include a display control device 20, a head-up display 24, smart contact lenses 14, and a smartphone 16. The head-up display 24 and smart contact lenses 14 are provided as a display unit DS to display various functions of the vehicle 12 to the occupants of the vehicle 12. Furthermore, the head-up display 24 and smart contact lenses 14 of this embodiment can use AR (Augmented Reality) technology to overlay images, text, and other content with the scenery outside the vehicle. The head-up display 24 is an example of a first display unit, and the smart contact lenses 14 is an example of a second display unit.

[0036] Regarding the display control device 20 and the head-up display 24, their main components are housed in the instrument panel 12A (see reference). Figure 8 and Figure 9 Inside the vehicle. A smart contact lens 14 is worn on the passenger's eyes. A smartphone 16 is held by the passenger.

[0037] The head-up display 24 is electrically connected to the display control device 20. Furthermore, the smart contact lens 14 is connected to the display control device 20 via a smartphone 16 in a communicative manner. However, this is not a limitation; the smart contact lens 14 may also communicate directly with the display control device 20 without using the smartphone 16.

[0038] (vehicle)

[0039] like Figure 2 As shown, vehicle 12 is configured to include at least a display control unit 20, an ECU 22, and a head-up display 24. In addition, vehicle 12 is equipped with a GPS (Global Positioning System) device 30, an external camera 34, and a line-of-sight camera 36.

[0040] The ECU (Electronic Control Unit) 22 includes the vehicle navigation ECU 22A, the ADAS (Advanced Driver Assistance System) ECU 22B, and the body ECU 22C.

[0041] The vehicle navigation ECU 22A controls the vehicle navigation system. A GPS device 30 is connected to the vehicle navigation ECU 22A. The GPS device 30 is a device for determining the current position of the vehicle 12. The GPS device 30 includes an antenna (not shown) for receiving signals from GPS satellites. It should be noted that the GPS device 30 can also be directly connected to the display control device 20.

[0042] ADAS-ECU 22B provides overall control of the advanced driver assistance system. An external camera 34 is connected to ADAS-ECU 22B as an external sensor to capture images at least in front of the vehicle 12. In addition to the external camera 34, ADAS-ECU 22B is also connected to a sensor array for detecting the surrounding environment of the vehicle 12, including millimeter-wave radar that transmits probe waves and receives reflected waves, and laser radar (Laser Imaging Detection and Ranging) that scans in front of the vehicle 12. Furthermore, actuators (not shown) are connected to ADAS-ECU 22B to actuate each of the brakes and accelerator. It should be noted that the external camera 34 can also be directly connected to the display control unit 20.

[0043] The vehicle body ECU22C controls the windshield wipers and lights. For example, switches for headlights and turn signals are connected to the vehicle body ECU22C.

[0044] The head-up display 24 is configured to include a projection device 26 and an actuator 28. The actuator 28 is a drive device for adjusting the angle of a reflector and the distance between the reflector and the projection device 26, wherein the reflector is a reflector that reflects an image projected from the projection device 26.

[0045] The gaze camera 36 is a camera that captures at least the face of the occupant and detects the occupant's gaze. The gaze camera 36 is connected to the input / output I / F20F described later.

[0046] Regarding the head-up display 24, the projection surface PA (refer to) has a defined area on the front windshield 12B. Figure 8 and Figure 9 The head-up display 24 displays an image projected from the projection device 26. This projection surface PA can be considered as the display area of ​​the head-up display 24. It should be noted that the projection surface of the head-up display 24 is not limited to the windshield 12B; the combination unit (reflector) located on the instrument panel 12A can also be a projection surface.

[0047] In this embodiment, the display control device 20 sets the position of the content displayed on the head-up display 24 based on the image captured by the external camera 34. At this time, the three-dimensional coordinates of the occupant's eye point are acquired based on the image from the gaze camera 36, ​​and the shooting location of the captured image is corrected to the occupant's eye point based on the acquired three-dimensional coordinates. Therefore, in the head-up display 24, the content can be displayed in a way that overlaps with the real image of the displayed object. It should be noted that, alternatively, the driving road, other vehicles, and pedestrians can be arranged in an imaginary space based on the image captured by the external camera 34, and the position of the content can be set for the displayed object in the imaginary space. In this case, the content can also be displayed in a way that overlaps with the real image of the displayed object in the head-up display 24.

[0048] The display control device 20 is configured to include a CPU (Central Processing Unit) 20A, a ROM (Read Only Memory) 20B, a RAM (Random Access Memory) 20C, a wireless communication I / F (Interface) 20D, an in-vehicle communication I / F 20E, and an input / output I / F 20F. The CPU 20A, ROM 20B, RAM 20C, wireless communication I / F 20D, in-vehicle communication I / F 20E, and input / output I / F 20F are interconnected communicatively via an internal bus 20G.

[0049] CPU 20A is a central processing unit that executes various programs or controls other parts of the system. Specifically, CPU 20A reads programs from ROM 20B and uses RAM 20C as the working area to execute the programs. CPU 20A can be considered the processor of this disclosure.

[0050] ROM20B stores various programs and data. For example... Figure 3 As shown, the ROM 20B in this embodiment stores a processing program 100 and image data 110. It should be noted that the processing program 100 and image data 110 can also be stored in storage devices such as HDD (Hard Disk Drive) or SSD (Solid State Drive).

[0051] The processing program 100, which is a display program, is used to perform the display processing and selection processing described later. The image data 110 is data that stores the content of an image displayed on the head-up display 24.

[0052] like Figure 2As shown, RAM20C temporarily stores programs or data as a work area.

[0053] The Wireless Communication I / F20D is an interface for connecting to the smartphone 16 or the smart glasses 18 described later. Communication standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark) are used in the Wireless Communication I / F20D.

[0054] The in-vehicle communication I / F20E is an interface used to connect to each ECU 22. This interface uses a communication standard based on the CAN (Controller Area Network) protocol. The in-vehicle communication I / F20E connects to each ECU 22 via the external bus 20H.

[0055] The input / output I / F20F is an interface for communicating with the projection device 26 and actuator 28 that constitute the head-up display 24.

[0056] like Figure 4 As shown, in the display control device 20 of this embodiment, the CPU 20A functions as the acquisition unit 200, the decision unit 210, the selection unit 220, and the display control unit 230 by executing the processing program 100.

[0057] The acquisition unit 200 has the function of acquiring vehicle information such as captured images from the vehicle 12 and control information of the vehicle 12. For example, the acquisition unit 200 acquires images captured by the external camera 34 and the line-of-sight camera 36. In addition, for example, the acquisition unit 200 acquires the route to the destination from the vehicle navigation ECU 22A. In addition, for example, the acquisition unit 200 acquires the speed, acceleration, yaw rate, etc. of the vehicle 12 from the ADAS-ECU 22B.

[0058] The decision unit 210 has the function of determining the content displayed on the head-up display 24 and the smart contact lens 14. For example, the decision unit 210 determines images such as arrows used in vehicle navigation as content based on image data 110. In addition, for example, the decision unit 210 determines images of numbers corresponding to vehicle speed as content.

[0059] The selection unit 220 has the function of selecting which display unit DS of the head-up display 24 and the smart contact lens 14 should display the content determined by the decision unit 210 based on predetermined conditions. The selection unit 220 selects each of the content to be displayed by performing the selection process described later.

[0060] The display control unit 230 has the function of displaying the content selected by the selection unit 220 on the display unit DS.

[0061] (Smart contact lenses)

[0062] The smart contact lens 14 is a wearable device worn on the eyes of the occupants of vehicle 12 to display images to their eyes. For example... Figure 5 As shown, the smart contact lens 14 is configured to include a control unit 40, a display unit 42, a camera unit 44, a power supply unit 46, a communication unit 48, and a sensor 49.

[0063] The control unit 40 controls the display unit 42, camera unit 44, power supply unit 46, communication unit 48, and sensor 49 that constitute the smart contact lens 14. The control unit 40 is configured as a microcomputer with a CPU, ROM, and RAM.

[0064] The display unit 42 functions as a display unit for showing images, text, and other content to prompt passengers. In this embodiment, the display unit 42 includes multiple light-emitting elements that generate images. It should be noted that the display unit 42, for example, uses a method that combines an LED (Light-Emitting Diode) as a light-emitting element with a microlens to project an image onto the retina, or uses holographic optical elements as light-emitting elements, thereby functioning as a device less affected by viewing distance. Furthermore, the display unit 42 in this embodiment includes multiple light-receiving elements for detecting gaze distance by receiving light reflected from the surface of the passenger's eyeball.

[0065] The camera unit 44 functions as a camera unit for capturing images of the outside of the eye. The camera unit 44 is configured to include a camera lens and a drive unit for driving the camera lens.

[0066] The power supply unit 46 is the driving source for the control unit 40, display unit 42, camera unit 44, communication unit 48, and sensor 49. It should be noted that in the smart contact lens 14, wireless power supply is also possible without a power supply unit 46.

[0067] The communication unit 48 functions as a communication unit for communicating with an external smartphone 16 or display control device 20. For example, a short-range wireless communication standard such as Bluetooth (registered trademark) is used in the communication unit 48. The communication unit 48 receives data from the content displayed on the display unit 42. Furthermore, the communication unit 48 transmits data from images captured by the camera unit 44 and data acquired by the sensor 49 (described later). It should be noted that in this embodiment, the communication unit 48 is connected to the display control device 20 via the smartphone 16, but it is not limited to this; the communication unit 48 may also be directly connected to the display control device 20.

[0068] Sensor 49 measures the resistance of the tears in the occupant's eyes. By measuring the resistance of the tears, the amount of tears can be determined in the display control device 20.

[0069] The smart contact lens 14 of this embodiment sets the position of the content displayed on the display unit 42 based on the image captured by the camera unit 44. Therefore, in the smart contact lens 14, the content can be displayed in a way that overlaps with the real image of the object being displayed.

[0070] (Smartphone)

[0071] The smartphone 16 is a communication terminal held by the passenger. The smartphone 16 is connected wirelessly to both the smart contact lens 14 and the display control device 20.

[0072] (Control process)

[0073] use Figure 6 and Figure 7 The flow of display processing and selection processing, which are the display methods executed in the display control device 20 of this embodiment, will be described. The display processing and selection processing are performed by the CPU 20A as the acquisition unit 200, the decision unit 210, the selection unit 220, and the display control unit 230.

[0074] exist Figure 6 In step S100, CPU 20A acquires captured images from each camera and obtains vehicle information from each ECU 22. For example, CPU 20A acquires the route of vehicle 12 to its destination, the speed of vehicle 12, acceleration, yaw rate, etc., as vehicle information.

[0075] In step S101, CPU20A determines the content to be displayed on each display unit DS and its display position.

[0076] In step S102, CPU20A performs a selection process. Details of the selection process will be described later.

[0077] In step S103, CPU 20A displays the content on the selected display unit DS. Then, it returns to step S100.

[0078] Specifically, when the head-up display 24 displays content, the CPU 20A outputs the image information of the content to the projection device 26. Thus, in the projection surface PA, from the occupant's viewpoint, the content is superimposed on the real image seen through the windshield 12B. For example, as... Figure 8As shown, a bounding box C1 representing the content being sensed by the advanced driver assistance system is overlaid on the vehicle in front of the displayed object. Furthermore, for example, a line C2 representing the path of vehicle 12 is overlaid on the driving road of the displayed object.

[0079] Furthermore, when the smart contact lens 14 displays content, the CPU 20A outputs the image information and location information of the content to the smart contact lens 14 via the smartphone 16. Thus, in the smart contact lens 14, the control unit 40 controls the display unit 42, so that the content is overlaid on the real image visually recognized through the display unit 42. For example, as... Figure 9 As shown, arrow C3, indicating the route ahead, is overlaid on the road in front of the displayed object. Furthermore, for example, to the right of vehicle 12, which is the displayed object area (object), a warning display C4 indicating that a two-wheeled vehicle is approaching from the right rear is overlaid.

[0080] Next, use Figure 7 The selection process in step S102 will be explained.

[0081] exist Figure 7 In step S200, CPU20A obtains the content to be displayed that was determined in S101.

[0082] In step S201, CPU 20A determines whether the content is displayed within the projection surface PA, i.e., within the display area of ​​the head-up display 24. In other words, CPU 20A determines whether the content display falls within the field of view of the head-up display 24. If CPU 20A determines that the content is displayed within the projection surface PA (if "yes" in step S201), it proceeds to step S202. On the other hand, if CPU 20A determines that the content is not displayed within the projection surface PA (if "no" in step S201), it proceeds to step S205.

[0083] In step S202, the CPU 20A determines whether the display position of the content is below a predetermined distance from the vehicle 12. Specifically, the distance from the vehicle 12 refers to the distance to the imaginary position where the virtual image of the content is displayed. For example, in... Figure 8In this case, the bounding box C1 is superimposed on the real image of the vehicle in front in the projection plane PA. In this case, the virtual image of the bounding box C1 is displayed at the imaginary position of the vehicle in front. If the CPU 20A determines that the display position of the content is less than or equal to a predetermined distance from the vehicle 12 (if "yes" is true in step S202), it proceeds to step S203. On the other hand, if the CPU 20A determines that the display position of the content is not less than or equal to a predetermined distance from the vehicle 12, that is, exceeds the predetermined distance (if "no" is true in step S202), it proceeds to step S205.

[0084] In step S203, CPU 20A determines whether there is a content-following object. Here, a content-following object refers to a vehicle, pedestrian, road, or other structure that is followed and overlaid by content. If CPU 20A determines that there is a content-following object (if "yes" in step S203), it proceeds to step S204. On the other hand, if CPU 20A determines that there is no content-following object (if "no" in step S203), it proceeds to step S205.

[0085] In step S204, CPU 20A selects to display content on head-up display 24. Then, proceed to step S206.

[0086] In step S205, CPU20A selects to enable the smart contact lens 14 to display content. Then, proceed to step S206.

[0087] In step S206, CPU 20A determines whether there are no other options to select. If CPU 20A determines that there are no other options to select (if "yes" in step S206), it ends the selection process. On the other hand, if CPU 20A determines that there are other options to select (if "no" in step S206), it returns to step S200.

[0088] (Summarize)

[0089] In the display control device 20 of this embodiment, the CPU 20A is configured to display content for each of the head-up display 24 (as a first display unit) and the smart contact lens 14 (as a second display unit). Here, the head-up display 24 is fixed to the vehicle 12, and the smart contact lens 14 is worn on the eyes of the passenger. In this embodiment, the CPU 20A selects the display unit DS for display content based on predetermined conditions, and overlays the content onto the real image visible through the display unit DS. According to the display control device 20 of this embodiment, it is possible to compensate for the individual characteristics of different display units DS to provide a display that is easily visually recognizable to the user.

[0090] In addition, the head-up display 24 is superior to the smart contact lens 14 in terms of the accuracy of the display position when the content is superimposed on the real image. On the other hand, the smart contact lens 14 of this embodiment projects the content directly onto the retina, thereby avoiding the sense of disharmony that occurs when the viewing distance of the content is far away, as is the case with the head-up display 24. In this embodiment, the CPU 20A selects which display unit DS to display the content based on at least one of the following conditions: the object to which the content is superimposed, the distance from the vehicle 12 to the object to be displayed, and the display position of the content relative to the field of view of each display unit DS.

[0091] In particular, in the display control device 20 of this embodiment, when the object for which content is to be overlaid is a vehicle, pedestrian, road, or other structure, a head-up display 24 with superior positioning accuracy is selected.

[0092] Furthermore, in the display control device 20 of this embodiment, when the display position of content exceeds the field of view of the head-up display 24, that is, exceeds the display area, the smart contact lens 14, which has less limitation on the field of view, displays the content. Therefore, according to this embodiment, the display of content outside the display area of ​​the head-up display 24 can be compensated for by the smart contact lens 14.

[0093] Furthermore, in the display control device 20 of this embodiment, when the distance from the vehicle 12 to the display object is greater than a predetermined distance, the smart contact lens 14, which is not easily affected by the viewing distance, displays the content. Therefore, according to this embodiment, even if the viewing distance of the content in the head-up display 24 deviates from that of the distant display object, the smart contact lens 14 can compensate for the display of the content.

[0094] [Second Implementation]

[0095] In the first embodiment, the display unit DS includes a head-up display 24 and a smart contact lens 14, and the user can select which display unit DS displays content. In contrast, in the second embodiment, the display unit DS includes a smart lens 18 in addition to the head-up display 24 and the smart contact lens 14, and the user can select which display unit DS displays content. The differences from the first embodiment will be described below. It should be noted that the same reference numerals are used for the same components, and detailed descriptions are omitted.

[0096] like Figure 10As shown, in the display control device 20 of this embodiment, in addition to the smartphone 16, smart glasses 18 are also connected in a communicative manner. The smart glasses 18 are worn on the head of the passenger. The smart glasses 18 are an example of a second display unit.

[0097] (Smart Glasses)

[0098] The smart glasses 18 are wearable devices that use AR technology to overlay images, text, and other content onto the scenery outside the vehicle in a head-mounted display manner. The smart glasses 18 are worn by passengers of the vehicle 12. Although not shown, the smart glasses 18 include at least a camera for capturing images inside and outside the vehicle, a transmissive display, and a communication device for communicating with the display control unit 20.

[0099] (Control process)

[0100] use Figure 11 The selection process of this embodiment will be explained.

[0101] exist Figure 11 In step S250, CPU20A obtains the content to be displayed that was determined in S101.

[0102] In step S251, CPU 20A determines whether the content is displayed on the projection surface PA. If CPU 20A determines that the content is displayed on the projection surface PA (if "yes" is true in step S251), it proceeds to step S252. On the other hand, if CPU 20A determines that the content is not displayed on the projection surface PA (if "no" is true in step S251), it proceeds to step S255.

[0103] In step S252, CPU 20A determines whether the display position of the content is below a predetermined distance from vehicle 12. If CPU 20A determines that the display position of the content is below the predetermined distance from vehicle 12 (if "yes" in step S252), it proceeds to step S253. On the other hand, if CPU 20A determines that the display position of the content is not below the predetermined distance from vehicle 12, i.e., exceeds the predetermined distance (if "no" in step S252), it proceeds to step S255.

[0104] In step S253, CPU 20A determines whether there is a following object with content. If CPU 20A determines that there is a following object with content (if "yes" in step S253), it proceeds to step S254. On the other hand, if CPU 20A determines that there is a following object without content (if "no" in step S253), it proceeds to step S256.

[0105] In step S254, CPU 20A selects to display content on head-up display 24. Then, proceed to step S258.

[0106] In step S255, CPU 20A determines whether there is a following object with content. This determination is the same as in step S253. If CPU 20A determines that there is a following object with content (if "yes" in step S255), it proceeds to step S257. On the other hand, if CPU 20A determines that there is no following object with content (if "no" in step S255), it proceeds to step S256.

[0107] In step S256, CPU20A selects to enable the smart contact lens 14 to display content. Then, it proceeds to step S258.

[0108] In step S257, CPU20A selects to enable the smart glasses 18 to display content. Then, it proceeds to step S258.

[0109] In step S258, CPU 20A determines whether there are no other options to select. If CPU 20A determines that there are no other options to select (if "yes" in step S258), it ends the selection process. On the other hand, if CPU 20A determines that there are other options to select (if "no" in step S258), it returns to step S250.

[0110] In the second embodiment, the same effects as in the first embodiment can be achieved. In particular, by using three display units (DS) – the head-up display 24, the smart contact lens 14, and the smart glasses 18 – to display content to the passenger, the best display that leverages the advantages of each display unit (DS) can be provided to the passenger. Specifically, by using the smart glasses 18, it is less constrained by the display area compared to the head-up display 24, and compared to the smart contact lens 14, it ensures the accuracy of the display position.

[0111] [Remark]

[0112] It should be noted that the bounding box C1, line C2, arrow C3, and warning display C4 can also be displayed on any one of the head-up display 24, smart contact lens 14, and smart glasses 18. It should also be noted that various processors other than the CPU can execute the various processes read and executed by the CPU 20A in the above embodiment. Examples of processors in this case include FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices) whose circuit configuration can be changed after manufacturing, and ASICs (Application Specific Integrated Circuits), which are dedicated circuits with circuit configurations specifically designed for executing specific processes. Furthermore, the above-described processes can be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit obtained by combining circuit elements such as semiconductor elements.

[0113] Furthermore, in the above embodiments, the scheme in which each program is pre-stored (installed) on a computer-readable non-transitory recording medium has been described. For example, the processing program 100 in the display control device 20 is pre-stored in the ROM 20B. However, it is not limited to this; each program may also be provided by recording on a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. In addition, the program may also be downloaded from an external device via a network.

[0114] The processing flow described in the above embodiments is also an example. Unnecessary steps can be deleted, new steps can be added, or the processing order can be changed without departing from the main idea.

Claims

1. A display control device, characterized in that, Including processors, The processor is configured to: In a first display unit fixed to a mobile body or a second display unit worn on a human body, a real image that is visually recognized through the first display unit or the second display unit is taken as an object, and the content to be superimposed on the object is determined. Based on predetermined conditions, it is determined which of the first and second display units will display the content. as well as In the selected first or second display unit, the content is displayed in an overlapping manner to match the object. The processor is configured to: cause the second display unit to display the content when the distance from the moving body to the object is greater than a predetermined value, and cause the first display unit to display the content when the distance from the moving body to the object is less than or equal to the predetermined value. The second display unit includes smart glasses worn on the head and smart contact lenses worn on the eyes. The processor is also configured to: when the second display unit is selected for display, select which of the smart glasses and the smart contact lens to display the content based on whether there is a follower of the content.

2. The display control device according to claim 1, characterized in that, The processor is further configured to select which of the first display unit and the second display unit displays the content based on at least one of the following conditions: the display position of the content relative to the field of view of the first display unit and the second display unit, and the object to which the content is to be displayed overlapping with the content.

3. The display control device according to claim 1, characterized in that, The processor is further configured to cause the second display unit to display the content when the display position of the content is outside the display area of ​​the first display unit.

4. The display control device according to claim 2, characterized in that, The processor is further configured to cause the second display unit to display the content when the display position of the content is outside the display area of ​​the first display unit.

5. A display system, characterized in that, include: The display control device according to any one of claims 1 to 4; The head-up display, which serves as the first display unit, is mounted on the vehicle, which serves as the moving body. A smart contact lens, included in a second display portion worn on the eyes of a passenger in the vehicle; as well as Smart glasses, including a second display unit worn on the head of a passenger in the vehicle.

6. A display method, characterized in that, include: The first step is to determine the content to be superimposed on the object by taking the real image that is visually recognized through the first display unit or the second display unit fixed to the moving body or worn on the human body as the object. The second step is to select, based on predetermined conditions, which of the first and second display units should display the content; as well as The third step is to display the content in an overlapping manner with the object in the selected first or second display unit. The first, second, and third steps are executed by a computer. If the distance from the moving body to the object is greater than a predetermined value, the second display unit displays the content; if the distance from the moving body to the object is less than or equal to the predetermined value, the first display unit displays the content. The second display unit includes smart glasses worn on the head and smart contact lenses worn on the eyes. When the second display unit is selected for display, the selection of which of the smart glasses and smart contact lenses to display the content is based on whether there is a follower of the content.

7. A non-transitory storage medium storing an instruction executable by one or more processors, the instruction causing the one or more processors to perform a function, wherein the non-transitory storage medium is characterized in that the function includes: In a first display unit fixed to a mobile body or a second display unit worn on a human body, a real image that is visually recognized through the first display unit or the second display unit is taken as an object, and the content to be superimposed on the object is determined. Based on predetermined conditions, it is determined which of the first and second display units will display the content. as well as In the selected first or second display unit, the content is displayed in an overlapping manner to match the object. If the distance from the moving body to the object is greater than a predetermined value, the second display unit displays the content; if the distance from the moving body to the object is less than or equal to the predetermined value, the first display unit displays the content. The second display unit includes smart glasses worn on the head and smart contact lenses worn on the eyes. When the second display unit is selected for display, the selection of which of the smart glasses and smart contact lenses to display the content is based on whether there is a follower of the content.

Images