Information processing device, information processing method, and information processing system

Abstract

An information processing device includes processing circuitry configured to control display of a display target in a head-up display of a vehicle based on vehicle equipment information of the vehicle. Display is controlled by adjusting the size, layout, and the like of the display target without deterioration of operability due to a decrease in visibility of the display target based on an installation position of a steering wheel, and optical characteristics of a windshield on which the display target is projected by the head-up display.

Description

INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING SYSTEM

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing system, and particularly, to an information processing device, an information processing method, and an information processing system for realizing uniform operability in UI operations using an HUD in response to a difference in vehicle equipment between vehicle models.

[0002] In vehicles such as automobiles, an instrument panel is provided within a dashboard, and information such as a vehicle speed and the number of revolutions of an engine is usually displayed on this instrument panel. Furthermore, a display is often built into the dashboard or installed on the dashboard, and thus screens for car navigation and the like are displayed on this display.

[0003] However, when a driver visually recognizes information displayed on an instrument panel or a display, the driver needs to move the line of sight significantly.

[0004] Consequently, a head-up display (HUD) is known as a technology for reducing the amount of line-of-sight movement. The head-up display projects and displays information such as a vehicle speed and instructions related to car navigation onto a windshield or the like. Hereinafter, the head-up display will also be referred to simply as an HUD.

[0005] Technologies related to such an HUD have been proposed, for example, in PTLs 1 to 5.

[0006] [PTL 1] JP 2020-115158 A [PTL 2] Japanese Patent No. 7282174 [PTL 3] JP 2020-197564 A [PTL 4] JP 2020-034856 A [PTL 5] JP 2020-071415 A

[0007] Incidentally, a technology is assumed that, by using an HUD, implements user interface (UI) operations from estimation results obtained by projecting icons and the like required for the UI operations onto a windshield, capturing an image of a user and estimating an icon gazed at by the user from the line-of-sight direction, and estimating gestures.

[0008] However, there are various vehicle models, and vehicle equipment varies depending on the vehicle model.

[0009] For example, positions at which icons required for the UI operations are projected from a driver's perspective change depending on whether a steering wheel, which is one piece of vehicle equipment, is provided on the left seat side or the right seat side. For this reason, information of icons visually recognized by the driver changes depending on the position of the steering wheel being vehicle equipment.

[0010] Furthermore, for example, there are various projection methods for a projection device of an HUD being vehicle equipment, and the windshields which are vehicle equipment have various optical characteristics due to differences in curvature, front-rear tilt angle, glass thicknesses and materials, and the like among vehicle models. For this reason, information of icons visually recognized by the driver changes depending on the projection method of the projection device of the HUD being vehicle equipment, and the optical characteristics of the windshield.

[0011] That is, in the UI operations using the HUD, the visibility of the icons changes depending on vehicle equipment, and thus visibility may be reduced depending on vehicle equipment, resulting in insufficient operability.

[0012] The present disclosure has been made in consideration of such circumstances, and particularly therewith, in UI operations using an HUD, it is possible to realize uniform operability in response to a difference in vehicle equipment between vehicle models.

[0013] An information processing device and an information processing system according to an aspect of the present disclosure are an information processing device and an information processing system that include a display control unit configured to control projection display of a display target by a projection unit based on vehicle equipment information of a vehicle.

[0014] An information processing method according to an aspect of the present disclosure is an information processing method including performing display control processing for controlling projection display of a display target based on vehicle equipment information of a vehicle.

[0015] In an aspect of the present disclosure, projection display of a display target by a projection unit is controlled based on vehicle equipment information of a vehicle.

[0016] Fig. 1 is a diagram illustrating an operation principle of an HUD.Fig. 2 is a diagram illustrating a UI operation using an HUD.Fig. 3 is a diagram illustrating that it is difficult to gaze at a region far from a driver for a predetermined period of time or longer as compared to a region close to the driver.Fig. 4 is a diagram illustrating an overview of the present disclosure.Fig. 5 is a diagram illustrating an HUD system according to the present disclosure.Fig. 6 is a diagram illustrating an example configuration of hardware of an information processing device in Fig. 5.Fig. 7 is a diagram illustrating functions implemented by the information processing device in Fig. 5.Fig. 8 is a diagram illustrating an example of a table stored in a design adjustment unit according to a first embodiment of the present disclosure.Fig. 9 is a flowchart illustrating display control processing according to the first embodiment of the present disclosure.Fig. 10 is a diagram illustrating a first modification example of the first embodiment of the present disclosure.Fig. 11 is a diagram illustrating a second modification example of the first embodiment of the present disclosure.Fig. 12 is a diagram illustrating a second modification example of the first embodiment of the present disclosure.Fig. 13 is a diagram illustrating a third modification example of the first embodiment of the present disclosure.Fig. 14 is a diagram illustrating a third modification example of the first embodiment of the present disclosure.Fig. 15 is a diagram illustrating an overview of a second embodiment of the present disclosure.Fig. 16 is a diagram illustrating the second embodiment of the present disclosure.Fig. 17 is a diagram illustrating an example of a table stored in a design adjustment unit according to the second embodiment of the present disclosure.Fig. 18 is a flowchart illustrating display control processing according to the second embodiment of the present disclosure.Fig. 19 is a diagram illustrating an overview of a third embodiment of the present disclosure.Fig. 20 is a diagram illustrating an example of impressions given by a Gothic font and a Mincho font, respectively.Fig. 21 is a diagram illustrating a relationship when weights and sizes are changed.Fig. 22 is a diagram illustrating a difference in visibility for each font.Fig. 23 is a diagram illustrating an example of curbing a decrease in visibility using a font and a character color.Fig. 24 is a diagram illustrating an example of curbing a decrease in visibility using a font and a character color.Fig. 25 is a flowchart illustrating display control processing according to the third embodiment of the present disclosure.Fig. 26 is a diagram illustrating an overview of a fourth embodiment of the present disclosure.Fig. 27 is a diagram illustrating an icon group having three levels of fineness.Fig. 28 is a diagram illustrating icons having seven different levels of fineness.Fig. 29 is a diagram illustrating an example of a table stored in a design adjustment unit according to the fourth embodiment of the present disclosure.Fig. 30 is a flowchart illustrating display control processing according to the third embodiment of the present disclosure.Fig. 31 is a diagram illustrating an example configuration of a general-purpose computer.

[0017] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configurations are given by the same reference numerals, and thus repeated descriptions are omitted.

[0018] Hereinafter, embodiments for implementing the present technology will be described. The description will be given in the following order.

[0019] 1. Overview of Present Disclosure 2. First Embodiment 2-1. First Modification Example of First Embodiment 2-2. Second Modification Example of First Embodiment 2-3. Third Modification Example of First Embodiment 3. Second Embodiment 4. Third Embodiment 5. Fourth Embodiment 6. Example of Software Execution

[0020] 1. Overview of Present Disclosure Configuration and Operation Principle of HUD The present disclosure is intended to realize uniform operability in response to changes in vehicle equipment different for each vehicle model by controlling the display of images such as icons required for UI operations, particularly, in accordance with the vehicle equipment in the UI operation using an HUD. Consequently, first, an overview of the present disclosure will be described.

[0021] Fig. 1 is a diagram illustrating the configuration and operation principle of a head up display (HUD).

[0022] As illustrated in Fig. 1, an HUD 11 is configured with a projection unit 22 including a windshield, a combiner, and the like, and a projection device 52.

[0023] The projection device 52 projects an image 23 as projection light onto the projection unit 22 provided on the windshield side in front of a driver's viewpoint 24, thereby presenting the image 23 so as to be superimposed on the real scenery in a driving field of vision and visually recognizable from the driver's viewpoint 24.

[0024] That is, with such a configuration, the image 23 projected onto the projection unit 22 from the projection device 52 is visually recognized from the driver's viewpoint 24 as a virtual image that appears to exist far away on the real scenery.

[0025] The HUD 11 configured as described above can display a vehicle information presentation screen, such as a vehicle speed and the number of revolutions of an engine, and a navigation screen as the image 23, allowing the user to visually recognize a vehicle information presentation image, a navigation image, and the like without moving his or her line of sight to an instrument panel or a display.

[0026] In the following description, it is assumed that the projection unit 22 projects the image 23 onto a windshield equipped with the projection unit 22 including a windshield, a combiner, or the like and that the driver visually recognizes the image 23 projected onto the windshield and superimposed on the real scenery.

[0027] UI Operation System Using HUD Next, a user interface (UI) operation system using a head up display (HUD) will be described with reference to Fig. 2. In addition to the HUD 11 described above, the UI operation system using the HUD includes a camera C that images a line-of-sight direction EP from the driver's viewpoint E and gestures using a hand H.

[0028] That is, in the UI operation system in Fig. 2, the projection unit 22 projects images of icons M1 and M2 onto the windshield, and the icons M1 and M2 are visually recognized as being floating on the real scenery through the windshield from the driver's viewpoint E.

[0029] The camera C captures an image having an angle of view that includes the viewpoint E for estimating the driver's line-of-sight direction and the driver's hand H for estimating the driver's gesture.

[0030] With such a configuration, the line-of-sight direction is estimated from the driver's viewpoint E, based on the image captured by the camera C, and an icon that exists in the estimated line-of-sight direction is considered to be an icon selected by the driver. In Fig. 2, which of the icons M1 and M2 has been selected by the driver is identified.

[0031] In addition, an operation input specified in association with the selected icon is identified by the hand gesture estimated based on the image captured by the camera C.

[0032] That is, in the UI operation system using the HUD in Fig. 2, the driver selects a desired icon by directing his / her line of sight toward the icon and gazing at the icon for a predetermined period of time or longer, and performs, with his / her hand, a gesture which is set in association with the selected icon, thus allowing an operation input with a desired content to be realized.

[0033] With such a configuration, for example, when the driver wants to output, as a voice sound, business hours of a store indicated by a hamburger-shaped icon M2 as illustrated in Fig. 2, the following operation is performed.

[0034] That is, for example, it is assumed that the driver directs the line-of-sight direction EP from the viewpoint E toward the icon M2, and performs a gesture representing a so-called "OK" sign by bringing the pads of the thumb and index finger of the hand H together and extending the middle finger, ring finger, and little finger.

[0035] At this time, the camera C captures a video of the driver. Based on the video captured by the camera C, the driver's line-of-sight direction EP is first identified.

[0036] More specifically, for example, in the video captured by the camera C, an icon that exists in the line-of-sight direction EP for a predetermined period of time or longer is recognized as the icon selected by the driver.

[0037] That is, in Fig. 2, the icon M2 visually recognized by the driver exists in the line-of-sight direction EP, and thus, when it is recognized that the icon M2 is gazed at for a predetermined period of time or longer while the line-of-sight direction EP is directed toward the icon M2, the icon M2 is recognized as having been selected by the driver.

[0038] Next, a gesture is determined from the movement of the fingers of the hand H, based on the video captured by the camera C.

[0039] In Fig. 2, from the resulting image captured by the camera C, a gesture representing a so-called "OK" sign by bringing the pads of the thumb and index finger of the hand H together and extending the middle finger, ring finger, and little finger is recognized. Here, it is assumed that the gesture representing the "OK" sign performed by the hand H in Fig. 2 is defined as an operation input for giving an instruction for a voice sound output of business hours of a store corresponding to the selected icon.

[0040] That is, as illustrated in Fig. 2, when the driver wants to output business hours of the store indicated by the hamburger-shaped icon M2 as a voice sound, the driver gazes at the icon M2 of the desired store in the line-of-sight direction EP for a predetermined period of time or longer, and performs a gesture representing an "OK" sign for giving an instruction for a voice sound output of business hours of a store corresponding to the selected icon as a specific action defined by the icon M2, thereby implementing a desired operation input.

[0041] In other words, the driver can implement a UI operation by moving his or her line of sight to an icon corresponding to the desired store among the icons visually recognized through the HUD 11, gazing at the icon for a predetermined period of time or longer, and performing a gesture for specifying an operation input defined by the desired icon.

[0042] Further, the UI operation in Fig. 2 is merely an example, and other UI operations may be used. For example, after continuing gazing at a desired icon for a predetermined period of time or longer, a speech for specifying a predetermined action may be performed or a gesture other than that of the hand may be performed.

[0043] Changes in Operability of UI Operations According to Vehicle Equipment Next, changes in operability of UI operations according to vehicle equipment will be described with reference to Fig. 3.

[0044] As described above, in UI operations using an HUD, it is necessary to maintain a state where an operation target such as an icon is gazed at for a predetermined period of time or longer, but the farther an image of a target object in front of the line of sight is, the more difficult it is to continue gazing at the same position for a predetermined period of time or longer.

[0045] That is, when the size of icons projected on the windshield are the same, the size of an icon visually recognized in the driver's field of vision becomes smaller as the position where the icon is projected becomes farther from the driver.

[0046] For this reason, when attempting to gaze at the same icon for a predetermined period of time or longer, at the time of gazing at an icon displayed at a position close to the driver, the icon is visually recognizable as an icon having a certain size within the field of vision, and thus even when the line-of-sight direction changes slightly, it is easier to recognize that the state of gazing at the same icon has been maintained.

[0047] On the other hand, an icon displayed at a position farther from the driver will be visually recognized as a smaller size of icon in the field of vision than an icon displayed at a position closer to the driver, and thus unless the line-of-sight direction is in a state of being fixed within a narrow range, in other words, in a state resulting from staring, it will be difficult for the icon to be recognized as being gazed on.

[0048] That is, an icon close to the driver's viewpoint is easier to gaze at for a predetermined period of time or longer, but the farther an icon is from the driver's viewpoint position, the more difficult it is to gaze at the icon for a predetermined period of time or longer.

[0049] More specifically, for example, as illustrated in Fig. 3, when the steering wheel being vehicle equipment is provided on the left seat side (hereinafter, in the case of a left-hand drive), the driver can easily gaze at information displayed at a position being in front of the driver and relatively close to the driver in a region on the left side of the windshield for a predetermined period of time or longer, even when the information is in a relatively small range such as a range ZN.

[0050] However, it is difficult for the driver to gaze at information displayed at a position being on the right side of the driver and relatively far from the driver in a region on the right side of the windshield for a predetermined period of time or longer unless the information is in a range larger than the range ZN, as indicated by a range ZF.

[0051] Note that, although not illustrated in the drawing, when the steering wheel being vehicle equipment is provided on the right seat side (hereinafter, in the case of a right-hand drive), the left and right are reversed.

[0052] Thus, for example, when a uniform size of icon with the size of the range ZN in Fig. 3 is displayed without considering the position of the steering wheel being vehicle equipment, it will be difficult for the driver to gaze at an icon displayed on the right side of the windshield for a predetermined period of time or longer in the case of a left-hand drive. In contrast, in the case of a right-hand drive, it becomes difficult to gaze at the icon displayed on the left side of the windshield for a predetermined period of time or longer.

[0053] In addition, a plurality of icons are typically placed, and thus, when the layout is the same between a right-hand drive and a left-hand drive, it becomes difficult to visually recognize icons displayed on the right side of a windshield in the case of a left-hand drive, and it becomes difficult to visually recognize icons displayed on the left side of the windshield in the case of a right-hand drive. That is, icons that are easier to view and icons that are difficult to view change depending on the position of a steering wheel being vehicle equipment.

[0054] That is, as described with reference to Fig. 3, the visibility of an icon changes depending on vehicle equipment, and operability in a UI operation changes, and thus the operability is reduced in specific vehicle equipment, and there is a concern that operability as intended at the time of design cannot be exhibited.

[0055] Consequently, in the present disclosure, even when a position at which an icon is displayed is relatively far from the driver, all icons are displayed in a size that allows the driver to easily gaze at the icons for a predetermined period of time or longer, and the layout is also changed symmetrically, based on the position of the steering wheel being vehicle equipment.

[0056] More specifically, as illustrated in a left part in Fig. 4, when the vehicle equipment corresponds to a left-hand drive, an icon SL having a size ZFL that is easier to gaze at for a predetermined period of time or longer is also displayed in front of the driver even when the icon is displayed at a position relatively far from the driver on the right side of the windshield.

[0057] In this case, since a distance from the driver to the front position is relatively short, it is easy for the driver to gaze at an icon having a size ZNL smaller than the size ZFL for a predetermined period of time or longer.

[0058] However, even when an icon is displayed on the right side of the windshield at a position relatively far from the driver, a uniform size of icon SL with the size ZFL that allows for gazing for a predetermined period of time or longer, is also displayed at a position relatively close to the driver.

[0059] Furthermore, as illustrated in a right part in Fig. 4, when the vehicle equipment corresponds to a right-hand drive, an icon SR having a size ZFR that is easier to gaze at for a predetermined period of time or longer is also displayed in front of the driver, even when the icon is displayed on the left side of the windshield at a position relatively far from the driver.

[0060] In this case, since a distance from the driver to the front position is relatively short, it is easy for the driver to gaze at an icon having a size ZNR (= ZNL) for a predetermined period of time or longer.

[0061] However, a uniform size of icon SR, with the size ZFR (=ZFL) that allows for gazing for a predetermined period of time or longer at a position relatively far from the driver on the left side of the windshield, is also displayed at a position relatively close to the driver.

[0062] Thereby, it is possible to curb a deterioration of operability in response to changes in vehicle equipment and to curb a deterioration of design characteristics by using a uniform icon size.

[0063] In addition, as illustrated in Fig. 4, the layout of the icons is displayed in a horizontally flipped manner. That is, as illustrated in the left part of Fig. 4, in the case of a left-hand drive, an icon of a telephone mark is presented on the left side in the upper row, and an icon for giving an instruction to playback a content is presented on the right side.

[0064] On the other hand, as illustrated in the right part in Fig. 4, in the case of a right-hand drive, an icon of a telephone mark is presented on the right side in the upper row, and an icon for giving an instruction to playback a content is presented on the left side. As illustrated in the left part in Fig. 4, the layout of the icons is horizontally flipped, and the layouts of icons in the other rows are horizontally flipped in the same manner.

[0065] In this manner, for example, when the most frequently used icon is laid out in the uppermost row in front, it can be placed in the uppermost row in front of the driver regardless of whether the steering wheel being vehicle equipment is on the left or right side, and thus it is possible to maintain the uniformity of designed operability even when the vehicle equipment is in a different state.

[0066] Furthermore, regarding the position of the steering wheel being vehicle equipment, a distance from the driver changes between a wide vehicle and a narrow vehicle, and thus the size of a displayed icon is changed in accordance with information on the vehicle equipment.

[0067] That is, in the case of a large vehicle with a wide vehicle width as vehicle equipment, as illustrated in the left part in Fig. 4, in the case of a left-hand drive, an icon on the right side of the windshield is displayed at a position farther away than in a small vehicle with a narrow vehicle width, and thus a uniform size of icons with a larger size are displayed.

[0068] In contrast, in the case of a small vehicle with a narrow vehicle width as the vehicle equipment, as illustrated in the left part in Fig. 4, in the case of a left-hand drive, the icon on the right side of the windshield is displayed at a closer position than in the vehicle with a large vehicle width, and thus a uniform size of icons having a smaller size than the icons in a large vehicle with a wide vehicle width are displayed.

[0069] In this manner, according to the present disclosure, it is possible to curb a deterioration of operability by controlling the display of the HUD in response to changes in vehicle equipment to mitigate changes in operability in UI operations, thus making it possible to achieve uniform operability intended at the time of design, regardless of the vehicle equipment.

[0070] 2. First Embodiment Next, an example of the configuration of a UI operation system using an HUD of the present disclosure will be described with reference to Fig. 5.

[0071] A UI operation system 31 using an HUD in Fig. 5 is an in-vehicle system and is configured with an information processing device 51, the projection device 52, and an imaging unit 53.

[0072] The information processing device 51 controls the projection device 52 to project onto a windshield 41 various vehicle information, such as a vehicle speed and the number of revolutions of an engine, required for UI operations, map information, navigation information, and images corresponding to icons and operation buttons required for UI operations, allowing a driver (user) 40 to visually recognize the images superimposed on the real scenery of the outside world through the windshield 41.

[0073] The information processing device 51 stores design information of an image to be projected and vehicle equipment information. When receiving an operation input from the driver 40, the information processing device 51 reads out the design information of the required image in response to the received operation input, adjusts the design information, based on the vehicle equipment information, supplies it to the projection device 52, and projects an image, based on the design information.

[0074] The design information referred to here is information for specifying variations in an expression method of the information projected by the projection device 52, the information including information on whether the information projected by the projection device 52 is expressed as pictograms such as icons, or as characters or sentences.

[0075] Furthermore, when the information projected by the projection device 52 is expressed as pictograms such as icons, the design information includes information for specifying variations such as a shape pattern, color pattern, and size of the pictograms, and the layout of a plurality of icons.

[0076] Furthermore, when the information projected by the projection device 52 is expressed as characters or sentences, the design information includes information for specifying variations such as the layout, color pattern, font (weight), and size of the characters.

[0077] Furthermore, here, the vehicle equipment information is information related to the equipment of the vehicle that affects visibility when presenting information required for UI operations. More specifically, the vehicle equipment information is information related to display performance such as the specifications and resolution information of the projection device 52, the vehicle width of the vehicle in which the UI operation system 31 is installed, and information such as a distance to an icon displayed at a position farthest from the driver 40 in a state where the driver 40 is seated in the seat on the side where the steering wheel 42 is provided. Further, the distance to the icon displayed at a position farthest from the driver 40 in a state where the driver 40 is seated in the seat on the side where the steering wheel 42 is provided corresponds to the vehicle width, and the wider the vehicle width, the larger the distance is, and the narrower the vehicle width, the smaller the distance is.

[0078] Considering the above, the size and layout of the pictograms and characters, which are design information, are adjusted in accordance with the vehicle equipment information so as to reduce the influence on visibility.

[0079] More specifically, for example, as described with reference to Fig. 4, when an icon is projected as an operation target for implementing a UI operation, the information processing device 51 acquires information such as the vehicle width and the position of the steering wheel as vehicle equipment information.

[0080] Next, based on the vehicle equipment information, the information processing device 51 specifies the size of an icon that allows for gazing for a predetermined period of time or longer at a projection position farthest from the driver 40 when the driver 40 is seated in the seat where the steering wheel 42 is provided.

[0081] Here, the size of an icon that allows for gazing for a predetermined period of time or longer is a size that makes it easier to determine whether the driver has intentionally gazed at a desired icon for a predetermined period of time or longer, as described with reference to Fig. 4.

[0082] More specifically, the size of an icon that allows for gazing for a predetermined period of time or longer is, for example, a size that, when the driver intentionally gazes at a desired icon, the driver can continue gazing at the icon for a predetermined period of time or longer particularly without having to stare at it while fixing the driver’s line of sight to the icon.

[0083] In other words, the size of an icon that allows for gazing for a predetermined period of time or longer is, for example, a size that avoids a situation in which, due to a display position being far from the driver’s position and the icon being too small to be visually recognized, the driver's line of sight deviates from the icon even when the driver intentionally gazes at the icon, thereby making it impossible to continue gazing at the icon for a predetermined period of time or longer.

[0084] Then, based on the design information of the icons to be projected by the projection device 52 and the vehicle equipment information, the information processing device 51 sets the sizes of the icons to a uniform size that allows for gazing for a predetermined period of time or longer, adjusts the layout according to the right or left position of the steering wheel, and supplies the icons to the projection device 52 to project the icons onto the windshield 41.

[0085] By performing such processing, the icons projected by the projection device 52 are projected in a size that allows for gazing for a predetermined period of time or longer even at a projection position farthest from the driver 40, and thus the icons can be gazed at for a predetermined period of time or longer regardless of the position at which they are projected, making it possible to appropriately implement UI operations.

[0086] As a result, even when the UI operation system is installed in vehicles with various vehicle equipment, a decrease in visibility that changes depending on the vehicle equipment is curbed, and it is possible to maintain uniform operability as intended at the time of design, regardless of the vehicle equipment.

[0087] Further, with regard to the size and layout of icons, which are design information adjusted based on vehicle equipment information, information with optimal size and layout may be registered in advance as a table for each piece of vehicle equipment information, and selected in accordance with the vehicle equipment information. In addition, the size and layout of icons, which are design information adjusted based on vehicle equipment information, may be calculated each time based on the vehicle equipment information.

[0088] The information processing device 51 controls the imaging unit 53 to capture an image of the driver 40, and estimates the line-of-sight direction and gesture (hand gesture) of the driver 40, based on the resulting captured image.

[0089] In addition, based on the design information projected by controlling the projection device 52 and targeted for UI operations, and the line-of-sight direction, the information processing device 51 identifies, as an icon selected by the driver 40, an icon gazed at for a predetermined period of time or longer among icons targeted for UI operations, and identifies a gesture.

[0090] Furthermore, based on the specified icon and gesture, the information processing device 51 specifies an operation input that is set for each gesture in association with the specified icon, and executes processing according to the specified operation input.

[0091] Example Configuration of Hardware of Information Processing Device in Fig. 5 Next, an example configuration of the hardware of the information processing device 51 in Fig. 5 will be described with reference to Fig. 6.

[0092] The information processing device 51 is configured with a control unit 71, an input unit 72, an output unit 73, a storage unit 74, a communication unit 75, a drive 76, and a removable storage medium 77, which are connected to each other via a bus 78 and can transmit and receive data and programs to and from each other. As described with reference to Fig. 5, the information processing device 51 is also connected to the projection device 52 and the imaging unit 53, and both the projection device 52 and the imaging unit 53 are connected via the bus 78.

[0093] The control unit 71 is configured with a processor and a memory, and controls the overall operation of the information processing device 51. The control unit 71 also includes a gesture determination unit 91, a line-of-sight estimation unit 92, an operation input estimation unit 93, a vehicle equipment information acquisition unit 94, a design information acquisition unit 95, a design adjustment unit 96, and a display control unit 97.

[0094] Further, functions implemented by the control unit 71, the gesture determination unit 91, the line-of-sight estimation unit 92, the operation input estimation unit 93, the vehicle equipment information acquisition unit 94, the design information acquisition unit 95, the design adjustment unit 96, and the display control unit 97 will be described below in detail with reference to Fig. 7.

[0095] The input unit 72 is configured with input devices such as buttons, switches, an operation knob, a keyboard, a pointing device, and a touch panel for inputting various types of information, and supplies various signals corresponding to the input information to the control unit 71.

[0096] The output unit 73 is controlled by the control unit 71 and includes a display unit 111 and a sound output unit 112. The display unit 111 is configured with, for example, a liquid crystal display (LCD) or an organic electro luminescence (EL), and displays operation inputs of the input unit 72 and various processing results of the control unit 71.

[0097] The sound output unit 112 is configured with a sound output device such as a speaker, and outputs various voice sounds, music, sound effects, and the like as sounds.

[0098] The storage unit 74 is configured with a hard disk drive (HDD), a solid state drive (SSD), or semiconductor memory, and is controlled by the control unit 71 to write or read out various data and programs. Furthermore, the storage unit 74 stores vehicle equipment information 131 and design information 132, which are written or read out as necessary under the control of the control unit 71.

[0099] The communication unit 75 is controlled by the control unit 71, and implements wired or wireless communication via a means represented by a local area network (LAN) or Bluetooth (registered trademark), and transmits and receives various data and programs to and from other information processing devices via a network as necessary.

[0100] The drive 76 reads and writes data from and to removable storage media 77 such as a magnetic disk (including a flexible disk), an optical disk (including a compact disc-read only memory (CD-ROM)) and a digital versatile disc (DVD)), a magneto-optical disk (including a mini disc (MD)), or a semiconductor memory.

[0101] As described above, the projection device 52 is controlled by the display control unit 97 to project, onto the windshield 41, images corresponding to various vehicle information such as a vehicle speed and the number of revolutions of an engine, map information, navigation information, and icons and operation buttons required for UI operations. With such a configuration, the driver 40 can visually recognize, through the windshield 41, the images of the various vehicle information such as a vehicle speed and the number of revolutions of an engine, map information, navigation information, and icons and operation buttons required for UI operations so as to be superimposed on the real scenery of the outside world.

[0102] The imaging unit 53 is an image sensor configured with a complementary metal oxide semiconductor (CMOS) sensor or the like. As described above, the imaging unit 53 captures an image with an angle of view that includes the face and hands of the driver 40, and supplies the captured image to the control unit 71. The gesture determination unit 91 and the line-of-sight estimation unit 92 of the control unit 71 each estimate a gesture (hand gesture) and line-of-sight direction of the driver 40, based on the image.

[0103] Functions Implemented by Information Processing Device Next, functions implemented by the information processing device 51 will be described with reference to a functional block diagram of Fig. 7.

[0104] The gesture determination unit 91 estimates a gesture (hand gesture) of the driver 40, based on an image captured by the imaging unit 53, and supplies the estimated gesture to the operation input estimation unit 93.

[0105] The line-of-sight estimation unit 92 estimates the line-of-sight direction of the driver 40, based on an image captured by the imaging unit 53, and supplies the estimated line-of-sight direction to the operation input estimation unit 93.

[0106] The input unit 72 receives operation inputs from input devices such as buttons, switches, an operation knob, a keyboard, a pointing device, and a touch panel, and supplies information on the received operation inputs to the operation input estimation unit 93.

[0107] The operation input estimation unit 93 estimates an operation input intended by the driver 40, based on a gesture supplied by the gesture determination unit 91, line-of-sight direction information supplied by the line-of-sight estimation unit 92, and operation input information supplied by the input unit 72, and instructs the design adjustment unit 96 to display information corresponding to the estimated operation input.

[0108] More specifically, the operation input estimation unit 93 estimates which of the icons currently projected by the projection device 52 is selected based on the information on the line-of-sight direction, and supplies, as the operation input intended by the driver 40, an operation input identified by a gesture among operation inputs set for the estimated icon to the design adjustment unit 96. The operation input estimation unit 93 may also instruct the design adjustment unit 96 to display information corresponding to the operation input intended by the driver 40, the operation input being an operation input itself from the input unit 72.

[0109] The design adjustment unit 96 instructs the design information acquisition unit 95 to acquire design information to be projected by the projection device 52 in response to an operation input intended by the driver 40 and supplied from the operation input estimation unit 93, and acquires the design information supplied from the design information acquisition unit 95 in response to this instruction.

[0110] The design adjustment unit 96 instructs the vehicle equipment information acquisition unit 94 to acquire vehicle equipment information, for example, at the time of start-up, and acquires the vehicle equipment information supplied from the vehicle equipment information acquisition unit 94 in response to this instruction.

[0111] For example, when displaying an icon targeted for a UI operation, the design adjustment unit 96 adjusts, based on the vehicle equipment information, the design information related to the icon targeted for a UI operation, and supplied and acquired from the design information acquisition unit 95, and supplies the adjusted design information to the display control unit 97.

[0112] More specifically, as described with reference to Fig. 4, the design adjustment unit 96 identifies the size of an icon that allows for gazing for a predetermined period of time or longer at a projection position farthest from the driver 40 when the driver 40 is seated in the seat where the steering wheel 42 is provided, based on the vehicle equipment information, such as a vehicle width and an offset distance from the center position of the position where the steering wheel is provided, and adjusts the design information of the icon to the identified size.

[0113] In this case, when identifying the size of the icon, based on the vehicle equipment information, for each piece of information in the vehicle equipment information, such as the vehicle model, vehicle width, and the position of the icon projected farthest from the driver 40, the layout and size in the image of the icon to be projected by the projection device 52 may be prepared in advance as a table including adjustment parameters for adjusting the design information 132, and may be read out and used as necessary.

[0114] For example, in the case of a left-hand drive vehicle, the table may be stored in advance for each piece of vehicle equipment information as a table illustrated in a left part in Fig. 8, and the corresponding ones may be read out and used based on the vehicle equipment information 131.

[0115] In the left part in Fig. 8, it is defined that a display part displayed as an icon indicating "phone call" is displayed at a coordinate position (600, 50) with a size of 50 px x 50 px, and a display part displayed as an icon indicating "music playback" is displayed at a coordinate position (600, 150) with a size of 50 px x 50 px.

[0116] In addition, for example, in the case of a right-hand drive vehicle, the table may be stored in advance as a table as illustrated in a right part in Fig. 8, and may be read out and used.

[0117] In the right part in Fig. 8, it is defined that a display part displayed as an icon indicating "phone call" is displayed at a coordinate position (50, 50) with a size of 50 px x 50 px, and a display part displayed as an icon indicating "music playback" is displayed at a coordinate position (50, 150) with a size of 50 px x 50 px.

[0118] Note that, in a coordinate position of (x, y), x is a coordinate set in a leftward direction from a right edge of the windshield being the origin (0), and y is a coordinate set in a downward direction from an upper edge being the origin (0). In addition, this table may be stored in advance in the storage unit 74, and the design adjustment unit 96 may read out an appropriate table from the storage unit 74 and use it, based on the vehicle equipment information 131 as necessary.

[0119] The design information acquisition unit 95 accesses the storage unit 74 in response to an instruction from the design adjustment unit 96 to acquire design information 132 to be projected by the projection device 52, and supplies it to the design adjustment unit 96.

[0120] The vehicle equipment information acquisition unit 94 accesses the storage unit 74 in response to an instruction from the design adjustment unit 96 to acquire vehicle equipment information 131, and supplies it to the design adjustment unit 96.

[0121] Display Control Processing of First Embodiment Next, display control processing will be described with reference to a flowchart of Fig. 9.

[0122] In step S31, the design adjustment unit 96 acquires vehicle equipment information.

[0123] More specifically, the design adjustment unit 96 requests the vehicle equipment information acquisition unit 94 to acquire vehicle equipment information. In response to this request, the vehicle equipment information acquisition unit 94 accesses the storage unit 74, reads the vehicle equipment information 131, and supplies it to the design adjustment unit 96. In this manner, the design adjustment unit 96 acquires the vehicle equipment information 131 related to the vehicle in which the design adjustment unit 96 itself is installed.

[0124] That is, at this point, the design adjustment unit 96 has acquired vehicle maintenance information, and thus can recognize, for example, the vehicle width of the vehicle in which it is installed, and whether the seat to which the steering wheel is attached is on the right or left side.

[0125] In step S32, the design adjustment unit 96 is notified by the operation input estimation unit 93 that some operation input has been performed, and determines whether there is an instruction to update a display content.

[0126] More specifically, for example, in a state where an image displaying an icon or the like for implementing some UI operation is projected by the processing performed so far, when the driver 40 gazes at a desired icon for a predetermined period of time or longer and executes a gesture to specify an operation input set in accordance with the icon, the imaging unit 53 captures an image of this behavior and supplies the captured image to the gesture determination unit 91 and the line-of-sight estimation unit 92.

[0127] In response to this, the gesture determination unit 91 determines a gesture of the driver 40, based on the image from the imaging unit 53, and supplies gesture information resulting from determination to the operation input estimation unit 93. At the same time, the line-of-sight estimation unit 92 estimates the line-of-sight direction of the driver 40, based on the image from the imaging unit 53, and supplies information on the line-of-sight direction of the driver 40 resulting from estimation to the operation input estimation unit 93.

[0128] Then, the operation input estimation unit 93 estimates an operation input of the driver 40 from the line-of-sight direction estimated by the line-of-sight estimation unit 92 and the gesture determined by the gesture determination unit 91, and supplies the operation input resulting from estimation to the design adjustment unit 96.

[0129] Alternatively, for example, when the driver 40 operates an input unit 72 to input an operation input in a state where an image displaying icons or the like for implementing some UI operation is projected by the processing performed so far, the operation input estimation unit 93 supplies the operation input input by operating the input unit 72 to the design adjustment unit 96 as the result of estimation of the operation input of the driver 40.

[0130] When an operation input is supplied from the operation input estimation unit 93 by any processing described above, the design adjustment unit 96 determines whether the supplied operation input is an instruction to update the display content.

[0131] When it is determined in step S32 that the operation input is an instruction to update the display content, the processing proceeds to step S33.

[0132] In step S33, the design adjustment unit 96 acquires design information.

[0133] More specifically, the design adjustment unit 96 instructs the design information acquisition unit 95 to acquire design information 132 corresponding to the display content for which the instruction for updating has been performed. In response to the instruction from the design adjustment unit 96, the design information acquisition unit 95 accesses the storage unit 74, reads out the design information 132, and supplies it to the design adjustment unit 96. Thereby, the design adjustment unit 96 acquires the design information supplied from the design information acquisition unit 95 and corresponding to the display content for which the instruction for updating has been performed.

[0134] In step S34, the design adjustment unit 96 reads the vehicle width and the right or left installation position of the steering wheel from the acquired vehicle equipment information.

[0135] In step S35, the design adjustment unit 96 determines the size and layout in accordance with the vehicle width, the installation position of the steering wheel, and the attributes of a UI to be displayed, and adjusts the design information. At this time, the design adjustment unit 96 may read out a table including parameters for adjusting the design information in advance, based on the vehicle equipment information such as the vehicle width and the installation position of the steering wheel, and adjust the design information, based on the read-out parameters.

[0136] In step S36, the design adjustment unit 96 supplies the adjusted design information to the display control unit 97, and projects it from the projection device 52.

[0137] In step S37, when it is determined whether to end the processing, and an end instruction is not performed, the processing returns to step S32, and the subsequent processes are repeated.

[0138] Then, in step S37, when an instruction to end the processing is performed, the display control processing ends.

[0139] Note that, when it is determined in step S32 that the operation input is not an instruction to update the display content, the processes of steps S33 to S36 are skipped.

[0140] Through the above processing, the design information is adjusted based on the vehicle equipment information including the information on the vehicle width and the installation position of the steering wheel, and icons and the like which are display targets required for UI operations are projected in a size that allows for grazing for a predetermined period of time or longer even at a projection position farthest from the driver 40, and thus an icon can be gazed at for a predetermined period of time or longer regardless of the position at which the icon is projected. Thereby, the driver can appropriately implement a UI operation for selecting his or her desired icon.

[0141] In addition, the layout of icons is a layout in accordance with the right or left installation position of the steering wheel, making it possible to curb changes in operability in accordance with vehicle equipment indicating whether the steering wheel is installed on the right side or the left side.

[0142] Thereby, it is possible to curb a deterioration　of operability of UI operation that occurs due to a decrease in visibility occurring in response to changes in vehicle equipment with respect to icons and the like that are display targets required for UI operations and are projected by the projection device 52 in the UI operation system 31.

[0143] As a result, in UI operations using the UI operation system 31, it is possible to maintain uniform operability as intended at the time of design, even for various vehicle models with different vehicle equipment.

[0144] 2-1. First Modification Example of First Embodiment In the above, an example has been described in which the size and layout of an icon to be operated in a UI operation is adjusted and projected in accordance with vehicle equipment information, but instead of the size of the icon, the size of a hit area in which the driver can be considered to be gazing at the icon may be adjusted.

[0145] That is, for example, a case is considered in which, when the size of the icon is adjusted in accordance with the vehicle equipment information, the size is adjusted to a size SG as illustrated in a left part in Fig. 10.

[0146] In this case, since the size itself of the icon is adjusted, when the size of the icon to be displayed becomes too large, the area that blocks the real scenery may become large and block the field of vision of the real scenery.

[0147] Consequently, as illustrated in a right part in Fig. 10, the size of the icon may be kept as a default size SS, and only the hit area, which is not visually recognized by the driver 40 but where the driver can be considered to be gazing at the icon, may be set to a size HA1 similar to the size SG. Note that the hit area mentioned here refers to an area that includes an icon and is larger than the icon, and is an area where the driver can be considered to be gazing at the icon when the driver's line-of-sight is within the hit area larger than the icon even when the driver's line-of-sight deviates from the displayed icon.

[0148] With such display, an icon to be displayed may have the default size SS, and thus visual recognition of the real scenery is not blocked. When the driver 40 gazes at a desired icon for a predetermined period of time or longer, and even when the line-of-sight direction of the driver 40 deviates slightly from the display area, the driver is considered to be gazing at the icon since the hit area is set to be larger than the icon.

[0149] Further, in this case, in a table expressing the amount of adjustment to be applied to design information set for each piece of vehicle equipment information, the size of the icon in Fig. 8 can be substituted for the size of the hit area and used.

[0150] In the above, an example has been described in which the size of an icon and the size of a hit area are adjusted to make it easier for the driver to gaze at a desired icon for a predetermined period of time or longer, and the desired icon being continuously gazed at can be recognized.

[0151] This is because it is difficult for the driver to continue gazing at an icon that exists at a position far from the driver, and thus, in the above, the size of the icon and the size of the hit area are increased to make it easier to recognize that the driver is gazing at the icon.

[0152] However, as far as an icon projected at a position far from the driver and being gazed at can be easily recognized, a period of time during which the driver has to gaze at the icon, that is, a triggering time, may be shortened without changing the size of the icon or the size of the hit area to make it easier to recognize that the driver is gazing at the icon.

[0153] For example, for an icon projected at a position a predetermined distance or more away from the driver, a triggering time during which it is recognized that the driver is gazing at the icon may be adjusted to be shorter than that for an icon that exists at a position a predetermined distance or less away from the driver, making it easier to recognize that the driver is gazing at the icon.

[0154] 2-2. Second Modification Example of First Embodiment In the above, it has been assumed that icons to be operated in UI operations are projected in a predetermined layout over the entire surface of the windshield, but when the icons are placed only in an area near the front of the driver 40, the area having high visibility and operability, and information corresponding to the position of the real scenery is presented, a leader line from the position corresponding to the real scenery may be set and displayed.

[0155] That is, for example, in the case of a left-hand drive vehicle, as illustrated in a left part in Fig. 11, icons SNL having a default size in design information may be concentrated and displayed near the front of the driver 40, which is a left half area in the drawing having high visibility and operability, and when the icons are presented as information on a position PL corresponding to the real scenery, the icons may be displayed such that they are connected by a leader line DLL from the position PL.

[0156] Similarly, for example, in the case of a right-hand drive vehicle, as illustrated in a right part in Fig. 11, icons SNR having a default size in the design information may be concentrated and displayed near the front of the driver 40, which is a right half area in the drawing having high visibility and operability, and when the icons are presented as information on a position PR corresponding to the real scenery, the icons may be displayed such that they are connected by a leader line DLR from the position PR.

[0157] In this case, for example, in the case of a left-hand drive vehicle, as illustrated in a left part in Fig. 12, a table expressing the amount of adjustment to be applied to the design information set for each piece of vehicle equipment information can be used such that display coordinates of an icon expressing a restaurant on a map are not specified (floating) and set to be any coordinates in a free space in a left half area in front of the driver 40, and the size of the icon is set to be, for example, a default size (here, 20 px x 20 px).

[0158] Furthermore, for example, in the case of a right-hand drive vehicle, as illustrated in a right part in Fig. 12, a table expressing the amount of adjustment to be applied to the design information set for each piece of vehicle equipment information can be used such that display coordinates of an icon expressing a restaurant on the map are not specified (floating) and set to be any coordinates in a free space in a right half area in front of the driver 40, and the size of the icon is set to be, for example, a default size (here, 20 px x 20 px).

[0159] 2-3. Third Modification Example of First Embodiment In the above, an example has been described in which only icons to be operated in UI operations are projected, but a map image for navigation may also be projected. However, in this case, icons may be laid out at a position away from the driver 40, the icons, compared with a map image, having relatively high visibility and making it easy to identify a line-of-sight position, and a map image may be laid out at a position in front of the driver 40, the map image, compared with icons, having low visibility and making it difficult to identify a line-of-sight position regardless of the need of fine operations.

[0160] That is, for example, in the case of a left-hand drive vehicle, as illustrated in a left part in Fig. 13, a map image ML may be laid out near the front of the driver 40 in the left half area in the drawing, and icons may be laid out in the right half area relatively far from the driver 40.

[0161] Furthermore, for example, in the case of a right-hand drive vehicle, as illustrated in a right part in Fig. 13, a map image MR may be laid out near the front of the driver 40 in the right half area in the drawing, and icons may be laid out in the left half area relatively far from the driver 40.

[0162] In this case, as illustrated in a left part in Fig. 14, in a table for adjusting the design in the case of a left-hand drive vehicle, a display part displayed as a receiver and indicating “phone call” may be defined to be displayed in a size of 50 px x 50 px at a coordinate position (600, 50), a display part displayed as an icon and indicating “music playback” may be defined to be displayed in a size of 50 px x 50 px at a coordinate position (600, 150), and a map may be defined to be displayed at a coordinate position (50, 50).

[0163] In addition, as illustrated in a right part in Fig. 14, in a table for adjusting the design in the case of a right-hand drive vehicle, a display part displayed as an icon and indicating "phone call" may be defined to be displayed in a size of 50 px x 50 px at a coordinate position (50, 50), a display part displayed as an icon and indicating "music playback" may be defined to be displayed in a size of 50 px x 50 px at a coordinate position (50, 150), and a map may be defined to be displayed at a coordinate position (600, 50).

[0164] 3. Second Embodiment According to the above-mentioned principle in which the UI operation system 31 projects an image onto the windshield from the projection device 52 to cause the driver to visually recognize the image as a virtual image floating on the real scenery, various blurs occur in an image visually recognized by the driver due to the optical characteristics of the windshield and the like being vehicle equipment.

[0165] The image projected by the projection device 52 may be designed to be visually recognized as if a certain degree of blur occurs in the image.

[0166] However, in reality, as described above, due to the optical characteristics of the windshield being vehicle equipment, the image may be visually recognized in a state where blurring intended in the design stage is not applied, or in a state where the intended blurring is applied excessively.

[0167] Consequently, in such a case, the original image of the image projected by the projection device 52 may be projected by applying adjustment blurring to the original image in accordance with the optical characteristics of the windshield being vehicle equipment, and thus the blur that occurs in the image visually recognized by the driver may be adjusted to a state intended in the design stage.

[0168] For example, a case where a windshield has optical characteristics that cause blurring is considered, the blurring being approximately 2 px as illustrated in the third column from the left in Fig. 15 when an original image has a star sign in the upper row and a character in the lower row as illustrated in the leftmost column in Fig. 15 and is projected by the projection device 52 without any change.

[0169] In this case, for example, when it is desired to allow the driver to visually recognize the original image blurred by approximately 5 px as illustrated in the second column from the left in Fig. 15, and when the original image is projected by the projection device 52 with blurring of approximately 3 px as illustrated in the rightmost column in Fig. 15, the image is further blurred by approximately 2 px due to the optical characteristics of the windshield, and as a result, the driver can visually recognize an image blurred by approximately 5 px as illustrated in the second column from the left in Fig. 15.

[0170] More specifically, changes in blur according to the optical characteristics of the windshield are as illustrated in Fig. 16, for example.

[0171] In Fig. 16, an image V1 including the original images to be projected by the projection device 52 is illustrated in the upper row, and an image V2 including images to be projected by the projection device 52 onto the windshield and visually recognized by the driver is illustrated in the lower row. Furthermore, in each of the images V1 and V2, Gothic font characters, Mincho font characters, a rectangular graphic image, a star-shaped graphic image, and a color pattern (code: fff880ffa9a9a84a4ff) are illustrated in order from the top, and images obtained by blurring the original image with an amount of blurring of 0 px to px are illustrated from the left in the drawing. Note that an image with an amount of blurring of 0 px is the original image itself.

[0172] In Fig. 16, with respect to the image V1 projected by the projection device 52, blurring of approximately 2 px occurs in the image V2 visually recognized by the driver due to the optical characteristics of the windshield.

[0173] Thus, for example, when it is desired to allow the driver to visually recognize an image blurred by approximately 2 px with respect to the original image, an adjustment is performed such that an unblurred original image is projected by the projection device 52, in consideration of the optical characteristics of the windshield being vehicle equipment.

[0174] In addition, for example, when it is desired to allow the driver to visually recognize an image that is blurred by approximately 3 px with respect to the original image, an adjustment is performed such that an image blurred by 1 px with respect to the original image is projected by the projection device 52, in consideration of the optical characteristics of the windshield being vehicle equipment.

[0175] In this example, the design adjustment unit 96 adjusts the amount of blurring to be applied to the original image included in design information so that the amount of blurring, which is assumed in the design information in advance, occurs in the image desired to be visually recognized by the driver, in consideration of the amount of blurring known to occur, based on the optical characteristics of the windshield being vehicle equipment information.

[0176] More specifically, when it is desired to allow the driver to visually recognize an image blurred by approximately 2 px with respect to the original image, the design adjustment unit 96 causes the projection device 52 to project the original image without any change, when the amount of blurring known to occur is 2 px, based on the optical characteristics of the windshield being vehicle equipment information.

[0177] In addition, when it is desired to allow the driver to visually recognize an image blurred by approximately 3 px with respect to the original image, the design adjustment unit 96 adjusts the amount of blurring by blurring the original image by 1 px and causes the projection device 52 to project the adjusted image when the amount of blurring known to occur is 2 px, based on the optical characteristics of the windshield being vehicle equipment information.

[0178] The design adjustment unit 96 may obtain the amount of intrinsic blurring that occurs in accordance with the optical characteristics by calculation and the like from the curvature, angle, material, and the like of the windshield being vehicle equipment information. However, since the optical characteristics of the windshield are known in advance for each vehicle model, a list indicating the amount of intrinsic blurring may be stored for each vehicle model, that is, for each piece of vehicle equipment information, as illustrated in Fig. 17, and read out and used for each piece of vehicle equipment information.

[0179] Fig. 17 illustrates an example of a list in which a vehicle model is associated with the amount of intrinsic blurring, and illustrates a vehicle model A having the amount of intrinsic blurring of 2 px, and a vehicle model B having the amount of intrinsic blurring of 5 px.

[0180] The design adjustment unit 96 determines the amount of adjustment of blurring based on the amount of intrinsic blurring so that the amount of blurring desired to be visually recognized by the driver is set in advance, based on design information, and adjusts the design information by applying the amount of blurring equivalent to the amount of adjustment to the original image.

[0181] That is, for example, when the amount of blurring desired to be visually recognized by the driver, which is set in advance based on the design information, is 3 px, and the amount of intrinsic blurring of the vehicle specified by vehicle equipment information is 2 px, the design information is adjusted by applying blurring of 1 px to the original image as the amount of adjustment according to the amount of intrinsic blurring.

[0182] Thereby, since adjustment blurring of 1 px is applied to the original image, the amount of intrinsic blurring of 2 px is applied by the projection of the projection device 52, and thus the total amount of blurring is 3 px, thereby making it possible to implement the amount of blurring of 3 px desired to be visually recognized by the driver in advance based on the design information.

[0183] Display Control Processing of Second Embodiment Next, display control processing in the second embodiment will be described with reference to a flowchart in Fig. 18.

[0184] Note that the processing of steps S51 to S53 and step S57 in the flowchart of Fig. 18 is similar to the processing of steps S31 to S33 and step S37 in the flowchart of Fig. 9, and will thus be omitted as appropriate.

[0185] In steps S51 to S53, the design adjustment unit 96 acquires vehicle equipment information, acquires design information if an instruction to update a display content is determined to be present, and then the processing proceeds to step S54.

[0186] In step S54, the design adjustment unit 96 reads vehicle model information and information on the amount of intrinsic blurring which specifies the optical characteristics of the windshield from the acquired vehicle equipment information.

[0187] In step S55, the design adjustment unit 96 determines the amount of adjustment of blurring, based on the amount of intrinsic blurring specified from the vehicle model information and the optical characteristics of the windshield, and adjusts the blurring of the design information by applying blurring having the determined amount of adjustment to the design information. At this time, the design adjustment unit 96 may read out a list of the amounts of intrinsic blurring corresponding to the optical characteristics of the windshield in association with the vehicle equipment information, determine the amount of adjustment of blurring, based on the read-out amount of intrinsic blurring, and adjust the blurring of the design information by applying the determined amount of adjustment of blurring to the design information.

[0188] Then, in step S56, the design adjustment unit 96 supplies the adjusted design information to the display control unit 97, and projects the adjusted design information by the projection device 52. When an instruction to end the processing is performed in step S57, the display control processing ends.

[0189] Through the above processing, the amount of blurring in the design information is adjusted based on the amount of intrinsic blurring obtained from vehicle equipment information including the vehicle model information and the optical characteristics of the windshield, making it possible for the driver to appropriately visually recognize an image with the designed blurring applied.

[0190] As a result, in UI operations using the UI operation system 31, it is possible to maintain uniform operability as intended at the time of design, even for various vehicles with different vehicle equipment.

[0191] 4. Third Embodiment In the UI operation system 31, it may be difficult to visually recognize an image including an icon to be operated and projected onto the windshield due to a relationship between the design and the optical characteristics of the windshield.

[0192] For such an image, effects may be applied to make it easier to visually recognize the image while retaining the design of the original image as much as possible, thereby making it possible to curb a deterioration of operability of UI operations and maintain uniform operability.

[0193] For example, it is considered that an image as indicated by an image V11 in the uppermost row of Fig. 19 is projected onto a windshield 21 by the projection device 52.

[0194] In the upper right row of the image V11, "road surface freezing information" is written in a Gothic font in a character color, such as red, that has a relatively low brightness (lightness). In the middle row, "start navigation" which is a title is written in a Gothic font in a character color, such as yellow, that has medium brightness (lightness). In the lower row, a long sentence is written in a Mincho font in a white character color with high brightness (lightness).

[0195] Further, the red and yellow colors used to indicate "road surface freezing information" and "start navigation" in the image V11 illustrated in Fig. 19 are expressed only by brightness (lightness) in grayscale.

[0196] Here, a Gothic font, which is a type of font, has excellent visibility compared to other fonts and is used for headings and titles. As illustrated in the upper row of Fig. 20, an impression given by a Gothic font is considered to become more "contemporary", "feminine", "simple", and "modern" as the weight (thickness) becomes smaller. In contrast, as illustrated in the upper row of Fig. 20, an impression given by a Gothic font is considered to become more "strong", "masculine", "impactful", and "energetic" as the weight (thickness) becomes larger.

[0197] Furthermore, a Mincho font, which is a type of font, has excellent readability compared to other fonts and is used for long sentences. As illustrated in the lower row of Fig. 20, an impression given by a Mincho font is considered to become more "elegant", "feminine", "refined", and "delicate" as the weight (thickness) becomes smaller. In contrast, as illustrated in the lower row of Fig. 20, an impression given by a Mincho font is considered to become more "trustworthy", "masculine", "impactful", and "classical" as the weight (thickness) becomes larger.

[0198] For this reason, in the image V11, the titles such as "road surface freezing information" and "start navigation" for emphasis are expressed using a Gothic font having a relatively large weight (thickness) and excellent visibility to give an impact. On the other hand, a long sentence is expressed in a Mincho font having a relatively small weight (thickness) and excellent readability to convey a sense of elegance and refinement.

[0199] Further, while a Mincho font is a font having variations in weight (thickness) resembling brush strokes, a Gothic font has a generally uniform weight (thickness) and has almost no thin parts. That is, compared to a Mincho font, a Gothic font has a larger weight as a whole, gives a more "masculine" and "impactful" impression, and becomes thicker as a whole, and thus the Gothic font can be considered to be easier to visually recognize.

[0200] However, when the image V11 is projected by the projection device 52, a long sentence expressed in a font such as a Mincho font, which has a smaller weight than a Gothic font, may be difficult to visually recognize, depending on the optical characteristics of the windshield being vehicle equipment.

[0201] In such a case, for example, as illustrated in an image V12, visibility may be improved by making an adjustment for changing the font of the long sentence written in a Mincho font having a small weight (thickness) to a Gothic font having a large weight.

[0202] Furthermore, even when the image V12 is projected by the projection device 52, the titles such as "road surface freezing information" in red and "start navigation" in yellow for emphasis to call attention have low brightness (lightness) depending on the optical characteristics of the windshield being vehicle equipment, and thus it may be difficult to visually recognize the image when the image is projected by the projection device 52.

[0203] In such a case, for example, as illustrated in an image V13, the "road surface freezing information" in red for emphasis to call attention may be changed to a white color having the highest brightness (lightness), and its font size may be made larger. In addition, the "road surface freezing information" may be underlined to express an emphasis in a form other than a red color.

[0204] In addition, the title such as "start navigation" may be changed to have a white color with the highest brightness (lightness).

[0205] That is, when the visibility of an image projected by the projection device 52 is decreased due to the optical characteristics of the windshield being vehicle equipment, the effects on characters such as fonts may be adjusted to minimize the impairment of design characteristics while compensating for the visibility.

[0206] More specifically, the weight and size of a font, a character spacing, and a character color may be adjusted in accordance with the optical characteristics of the windshield being vehicle equipment, so as to minimize the impairment of design characteristics while compensating for the visibility.

[0207] Weight and Size of Font As for fonts, it is possible to adjust visibility by the weight and size thereof, and thus a font close to the original font and having a larger size may be used to keep the design characteristics from being impaired, while maintaining visibility.

[0208] For example, as illustrated in Fig. 21, when an original image V31 with changed font weight and size is projected by the projection device 52, it is visually recognized as illustrated, for example, in an image V32, and thus a decrease in visibility may be curbed by adjusting the font to a weight and size that are closest to a limit at which it is visually recognizable when projected by the projection device 52.

[0209] Note that, in Fig. 21, in the original image V31 in the upper row, character examples of corresponding to Black, Heavy, Bold, Medium, Regular, Light, and Thin are written in order of weight (thickness) from the top in the drawing, and the character examples are written in font sizes of 16 pt, 24 pt, 30 pt, 48 pt, 60 pt, and 72 pt in ascending order of font size from the left in the drawing.

[0210] In addition, an image V32 illustrates an example of character examples of that are visually recognized by the driver when the original image V31 is projected onto the windshield 21 by the projection device 52.

[0211] That is, in the images V31 and V32, as a font has a larger weight (thickness) and larger size, that is, as a font becomes closer to a font in the upper right part in the drawing, the font has higher visibility and becomes more suitable for a title or the like, but the readability thereof decreases. On the other hand, as a font has a smaller weight (thickness) and smaller size, that is, as a font becomes closer to a font in the lower left part in the drawing, the font has lower visibility, but has higher readability.

[0212] Here, a visibility limit curve Lp, which is represented by a dotted curve, represents the weights and sizes that are the limits of a character string that is visually recognizable by the driver in the image V32. Furthermore, a selection limit curve Lo in the image V31 is a curve that corresponds to the visibility limit curve Lp in the image V32.

[0213] That is, the visibility limit curve Lp in the image V32 represents the limit of visibility for fonts that are visually recognizable as an image projected by the projection device 52. For this reason, among the fonts displayed in the image V32, fonts that exist above the visibility limit curve Lp in the drawing are visually recognizable. In contrast, fonts that exist below the visibility limit curve Lp in the drawing cannot be visually recognized.

[0214] For this reason, for example, in the original image, when a font Fo in the image V31 has a weight of Thin and a size of 48 pt, it becomes a font Fpo in the image V32 when projected by the projection device 52.

[0215] That is, in this case, the font Fpo in the image V32 is located below the visibility limit curve Lp, and cannot be thus visually recognized.

[0216] Consequently, the design adjustment unit 96 performs adjustment of font so that the font Fo is changed to a font Fc that is above the selection limit curve Lo and has a weight and size close to the font in the original image, for example, a weight of Light and a size of 48 pt in the image V31.

[0217] As a result, when the font Fc in the image V31 is projected by the projection device 52, the font Fc is visually recognized as a font Vpc located above the visibility limit curve Lp in the image V32, and is thus projected in a visible state.

[0218] Note that the design adjustment unit 96 stores at least one of the selection limit curve Lo and the visibility limit curve Lp for the distributions of the weights and the sizes according to the optical characteristics of the windshield being vehicle equipment, and may adjust the weight and size of the characters in the original image so that the characters have a font that is located above at least one of the selection limit curve Lo and the visibility limit curve Lp in Fig. 21 and has a weight and size close to the font in the original image.

[0219] Font and Character Spacing Even for fonts having the same weight and size, visibility can be adjusted by a character spacing, that is, a spacing between characters, and thus by adjusting the character spacing, which is a spacing between characters, design information may be adjusted not to impair the design characteristics while maintaining visibility.

[0220] For example, as illustrated in Fig. 22, for fonts having the same weight and size, when an original image V41 with a different character spacing is visually recognized as illustrated in an image V42 when projected by the projection device 52, the font may be adjusted to have a character spacing that is visually recognizable.

[0221] Note that, in Fig. 22, in the original image V41 in the upper row, is written for each of fonts, that is, a Yu Gothic font, a HG Gothic font, an MS Gothic font, an HG Kyokasho font, a Yu Mincho font, and a Toppan Bunkyu Midashi Mincho font in descending order of weight (thickness) from the top in the drawing, and for each of spacings of 0, 50, 100, 150, and 200.

[0222] In addition, when the original image V41 is projected onto the windshield 21 by the projection device 52, the image V42 illustrates an example of a character string that is visually recognized by the driver.

[0223] That is, in the images V41 and V42, the larger the character spacing, that is, the wider the spacing, the higher the visibility, and the smaller the character spacing and the narrower the spacing, the lower the visibility.

[0224] For this reason, for example, when the original image has a font spacing Io of a character spacing 50 in the image V41, the font spacing becomes a font spacing Ipo in the image V42 when projected by the projection device 52.

[0225] That is, in this case, when the font is, for example, a Yu Mincho font of a character spacing 50 in the image V41, the font becomes almost visually unrecognizable in the image V42 when projected by the projection device 52. Consequently, in such a case, the design adjustment unit 96 adjusts the font so that it is changed from a Yu Mincho font to a recognizable font, for example, an MS Gothic font.

[0226] Thereby, when the MS Gothic font of the character spacing 50 in the image V41 is projected by the projection device 52, it will be projected as an MS Gothic font of a character spacing 50 that is visually recognizable in the image V42.

[0227] In addition, when it is desired to keep the font as a Yu Mincho font, for example, the design adjustment unit 96 may adjust the font so that it is changed to a recognizable Yu Mincho font of a character spacing 200.

[0228] In this case, when the Yu Mincho font of the character spacing 200 in the image V41 is projected by the projection device 52, it will be projected as a Yu Mincho font of a character spacing 200 that is visually recognizable in the image V42.

[0229] Thereby, when the Yu Mincho font of the character spacing 50 in the image V41 is projected by the projection device 52, the font is projected by either or at least one of using a font type visually recognizable in the image V42 or using a visible character spacing, thereby curbing a decrease in visibility.

[0230] The design adjustment unit 96 stores the visibility limit obtained from the relationship between a font and a character spacing as illustrated in Fig. 22, which corresponds to the optical characteristics of the windshield being vehicle equipment. For this reason, when it is expected that the visibility of the font and character spacing of characters in the original image will decrease below the visibility limit, the design adjustment unit 96 may adjust the design information, based on visibility limit information so that the font and character spacing become visually recognizable.

[0231] As described above, both font adjustment and character spacing adjustment may be performed, or either one of them may be performed. Furthermore, when there are a plurality of options related to the adjustment of a visually recognizable font or character spacing, one of them may be selected randomly, or it may be determined in advance that one of them will be selected.

[0232] Font and Character Color Visibility may be adjusted by using a font and a character color to keep the design characteristics　from being impaired, while maintaining visibility.

[0233] For example, as illustrated in a left part in Fig. 23, a case where the original image is an image V61 is considered, the image V61 indicating characters of "Please follow the actual traffic signs when driving" written in a highly readable Mincho font.

[0234] When the image V61 is projected by the projection device 52, the image is visually recognized by the driver as illustrated in an image V62. In this case, since the characters are written in a highly readable Mincho font, the weight is small, and visibility is decreased.

[0235] Consequently, when the visibility is below the visibility limit due to a decrease in visibility, the design adjustment unit 96 adjusts the font of the characters in the original image from a Mincho font to a Gothic font having a large weight (thickness) in order to curb a decrease in visibility.

[0236] As a result, when the characters in the image V61 are adjusted to have a Gothic font and projected by the projection device 52, they are visually recognized by the driver as Gothic characters with a larger weight, as illustrated in an image V63, and a decrease in visibility can be curbed.

[0237] In addition, for example, as illustrated in a left part in Fig. 24, a case where the original image is an image V71 is considered, the image V71 indicating characters "road surface freezing information" written in a red character color and a Gothic font. Note that the character color in the image V71 illustrated in the left part in Fig. 24 is gray for representing only the brightness (lightness) of red in the drawing.

[0238] When the image V71 is projected by the projection device 52, the image is visually recognized by the driver as illustrated in an image V72. In this case, the characters are written in a red character color with a low brightness (lightness), and thus the characters blend into the black background, thereby decreasing visibility.

[0239] Consequently, in order to curb such a decrease in visibility, the design adjustment unit 96 adjusts the character color of the characters in the original image from a red color to a white color having a high brightness (lightness) and adds an underline.

[0240] As a result, when the image V71 is projected by the projection device 52 with the characters in the image V71 changed from a red color to a white color having a high brightness (lightness) and with an underline added, the background stands out clearly against the black background, and the characters are visually recognized by the driver as underlined characters as illustrated in an image V73, making it possible to curb a decrease in visibility.

[0241] Note that the design adjustment unit 96 stores a visibility limit for each of the font and character color according to the optical characteristics of the windshield being vehicle equipment, and when it is expected that the visibility with respect to the font and character color of the characters in the original image will fall below the visibility limits, the font and character color may be adjusted to make visibility higher than the stored visibility limit.

[0242] That is, with regard to the font, as described above, the smaller the weight and the smaller the size, the lower the visibility, and thus, when it is expected that the visibility will fall below the limits of the weight and size, the design adjustment unit 96 increases both the weight and the size or increase either one of them, thereby curbing a decrease in visibility.

[0243] In addition, with regard to the character color, the effect of brightness (lightness) is large and the lower the brightness (lightness), the lower the visibility, and thus, when it is expected that the brightness (lightness) will fall below the visibility limit, the design adjustment unit 96 may adjust the character color so that the brightness (lightness) is higher than the visibility limit of brightness (lightness) to an extent that at least allows a decrease in visibility to be curbed. More specifically, for example, with regard to red and blue colors, the character color may be adjusted to, for example, a white color to increase the brightness (lightness), or the character color may be adjusted to a pink or light blue color to increase the lightness of each of the red and blue colors. The visibility limit of brightness (lightness) may be, for example, a lightness L in a CIELab color space of approximately 70 or more.

[0244] Display Control Processing of Third Embodiment Next, display control processing in a third embodiment will be described with reference to a flowchart of Fig. 25.

[0245] Note that the processing of steps S71 to S73 and step S77 in the flowchart of Fig. 25 is similar to the processing of steps S31 to S33 and step S37 in the flowchart of Fig. 9, and will thus be omitted as appropriate.

[0246] In steps S71 to S73, the design adjustment unit 96 acquires vehicle equipment information, acquires design information if an instruction to update a display content is determined to be present, and then the processing proceeds to step S74.

[0247] In step S74, the design adjustment unit 96 reads, from the acquired vehicle equipment information, information on visibility limits of a font (weight and size) and a character color, such as a visibility limit curve Lp (selection limit curve Lo), that is specified by vehicle model information and the optical characteristics of the windshield.

[0248] In step S75, based on the information on the visibility limits of the font and the character color, such as the visibility limit curve Lp (selection limit curve Lo), that is specified by the optical characteristics of the windshield being vehicle equipment, when it is expected that the visibility will fall below the visibility limits and that visual recognition will not be possible, the design adjustment unit 96 determines a font (weight and size) and a character color so as to exceed the visibility limits, and adjusts the design information with the determined font and character color.

[0249] Then, in step S76, the design adjustment unit 96 supplies the adjusted design information to the display control unit 97 and causes the projection device 52 to perform projection. If an instruction to end the processing is performed in step S57, the display control processing ends.

[0250] Through the above processing, based on the information on the visibility limits (visibility limits) of the font and the character color, such as the visibility limit curve Lp (selection limit curve Lo), that is specified from the optical characteristics of the windshield included in the vehicle equipment information, when it is expected that the visibility will fall below the visibility limits and that visual recognition will not be possible, the font (weight and size) and the character color are adjusted, and thus an image such as an icon to be operated based on the design information can be appropriately visually recognized by the driver.

[0251] As a result, in UI operations using the UI operation system 31, it is possible to maintain uniform operability as intended at the time of design, even for various vehicle models with different vehicle equipment.

[0252] 5. Fourth Embodiment Due to the optical characteristics of the projection device 52 and the windshield 21, which are vehicle equipment, horizontal and vertical modulation transfer functions (MTFs) are different from each other, and thus a spatial frequency is low and fine patterns may be unable to be expressed.

[0253] In such a case, when attempting to display a fine pattern, the pattern cannot be expressed despite the existence of information, and thus the driver will be unable to read it, which in effect results in a decrease in visibility.

[0254] Consequently, a plurality of icons and GUIs may be prepared, and those that can be expressed may be selectively displayed in accordance with a spatial frequency of each vehicle. In this case, when there are no appropriate icons and GUIs, the icons and GUIs themselves may simply be enlarged and displayed.

[0255] For example, as illustrated in an upper row in Fig. 26, when an original image V91, which is expressed by changing a line thickness and a line spacing, is projected onto the windshield 21 by the projection device 52, the image is visually recognized as illustrated in an image V92 by the driver.

[0256] The image V91 is the original image in which lines with horizontal thicknesses of 3, 5, 10, 15, 20, and 30 px are placed vertically in order from the top with line spacings of 9, 8, 7, 6, 5, 4, 3, 2, and 1 px.

[0257] In the image V92, for line spacings of up to 2 px, a pattern with any line thickness allows the presence of spacings to be appropriately visually recognized, while for patterns having line spacing of 1 px, patterns with some line thicknesses do not allow the presence of spacings to be appropriately visually recognized.

[0258] That is, due to the optical characteristics of the projection device 52 and the windshield 21, which are vehicle equipment in this vehicle, icons and GUIs with patterns having a line spacing of up to 2 px can be considered to be visually recognizable.

[0259] However, due to the optical characteristics of the projection device 52 and the windshield 21, which are vehicle equipment in this vehicle, when only icons and GUIs of patterns having a line spacing of up to 4 px are visually recognizable, and a high-resolution icon or GUI including a pattern having a line spacing of less than 4 px is displayed, the driver cannot visually recognize the presence of the line spacing, and there is a substantial decrease in visibility.

[0260] Consequently, for icons presenting the same information, due to the optical characteristics of the projection device 52 and the windshield 21, which are vehicle equipment, high-resolution icons used for vehicles that can represent a high-resolution pattern, medium-resolution icons used for vehicles that can represent a medium-resolution pattern, and low-resolution icons used for vehicles that can represent only a low-resolution pattern may be prepared, and the icons may be switched and displayed based on the vehicle equipment information.

[0261] More specifically, for example, as illustrated in Fig. 27, icon groups G1 and G11 including low-resolution patterns, icon groups G2 and G12 including medium-resolution patterns, and icon groups G3 and G13 including high-resolution patterns may be prepared, and the icons may be switched and displayed based on the vehicle equipment information.

[0262] In Fig. 27, icons representing the start of content playback, icons representing a phone call, icons representing a cart, and icons representing a position on a map are illustrated in order from the top.

[0263] In addition, the icon groups G1 to G3 are icon groups including patterns of lines only, while the icon groups G11 to G13 are icon groups including patterns cut out to make white parts transparent.

[0264] That is, the icon groups G3 and G13 are icon groups including patterns, directed to vehicles, that can be presented as information recognizable to the driver due to the optical characteristics of the projection device 52 and the windshield 21, which are vehicle equipment, even when high-resolution patterns are used, and thus the icon groups include fine patterns.

[0265] On the other hand, the icon groups G1 and G11 are icon groups including patterns, directed to vehicles, that cannot be presented as information recognizable to the driver due to the optical characteristics of the projection device 52 and the windshield 21, which are vehicle equipment, unless low-resolution patterns are used, and thus only simple patterns are present.

[0266] The icon groups G2 and G12 are icon groups including patterns with medium-resolution at an approximately intermediate level between the icon groups G1 and G11 and the icon groups G3 and G13.

[0267] Fig. 27 illustrates an example of icon groups classified into three levels of resolution, but icon groups classified into three or more levels of resolution may be generated.

[0268] For example, as illustrated in Fig. 28, icons may be classified into seven levels of resolution.

[0269] In Fig. 28, icons G31 to G37 representing circular meter gauges are illustrated in the upper rows, and icons G41 to G47 representing maps are illustrated in the lower rows.

[0270] Among the icons G31 to G37, the icon G31 is an icon having the lowest resolution, the resolution increases in the order of the icons G31, G32, G33,..., and G37, and the icon G37 is an icon having the highest resolution.

[0271] Similarly, in the icons G41 to G47, the icon G41 is an icon having the lowest resolution, the resolution increases in the order of the icons G41, G42, G43,..., and G47, and the icon G47 is an icon having the highest resolution.

[0272] The design adjustment unit 96 switches between and presents icon groups having different resolutions, based on distinguishable px amounts according to the optical characteristics of the windshield being vehicle equipment.

[0273] The design adjustment unit 96 may store the distinguishable px amount for each piece of vehicle equipment information and the corresponding icon set type, for example, as a table as illustrated in Fig. 29, and switch and display icons, based on the table.

[0274] In Fig. 29, a table for three levels of resolution is illustrated, with the distinguishable px amount written on the left and the icon set type written on the right.

[0275] More specifically, in Fig. 29, when the distinguishable px amount is less than 2 px, a high-resolution icon set is set, when the distinguishable px amount is 2 px or more and less than 6 px, a medium-resolution icon set is set, and when the distinguishable px amount is 6 px or more, a low-resolution icon set is set.

[0276] The design adjustment unit 96 stores the table as illustrated in Fig. 29, selects an icon set with a resolution according to the distinguishable px amount for each piece of vehicle equipment information item, adjusts the icon of the design information, and causes it to be projected by the projection device 52.

[0277] Thereby, the projection device 52 projects an icon with a resolution that can be displayed for each vehicle, and thus it is possible to curb a decrease in visibility that occurs when a high-resolution icon that cannot be displayed is presented in an incomplete state.

[0278] Display Control Processing of Fourth Embodiment Next, display control processing in a fourth embodiment will be described with reference to a flowchart of Fig. 30.

[0279] Note that the processing of steps S91 to S93 and step S97 in the flowchart of Fig. 30 is similar to the processing of steps S31 to S33 and step S37 in the flowchart of Fig. 9, and will thus be omitted as appropriate.

[0280] In steps S91 to S93, the design adjustment unit 96 acquires vehicle equipment information, acquires design information if an instruction to change a display content is determined to be present, and then the processing proceeds to step S94.

[0281] In step S94, the design adjustment unit 96 reads information on a distinguishable px amount specified by vehicle model information and the optical characteristics of the windshield from the acquired vehicle equipment information.

[0282] In step S95, the design adjustment unit 96 determines an icon set of a corresponding resolution, based on the information on the distinguishable px amount specified from the optical characteristics of the windshield being vehicle equipment, and adjusts design information with the determined icon set.

[0283] Then, in step S96, the design adjustment unit 96 supplies the adjusted design information to the display control unit 97, projects it by the projection device 52, and when an instruction to end the processing is performed in step S97, the display control processing ends.

[0284] Through the above processing, the design information is adjusted such that icons having a representable resolution specified from the optical characteristics of the windshield included in the vehicle equipment information are displayed, making it possible for the driver to appropriately visually recognize recognizable icons.

[0285] As a result, in UI operations using the UI operation system 31, it is possible to maintain uniform operability as intended at the time of design, even for various vehicle models with different vehicle equipment.

[0286] 6. Example of Execution by Software Incidentally, the above-described series of processing can be executed by hardware, but they can also be executed by software. When the series of processing is executed by software, programs that configure the software are installed from a recording medium into a computer built into dedicated hardware, or into, for example, a general-purpose computer that can execute various functions by installing various programs.

[0287] Fig. 31 illustrates an example of the configuration of a general-purpose computer. The computer has a built-in central processing unit (CPU) 1001. An input / output interface 1005 is connected to the CPU 1001 via a bus 1004. A read only memory (ROM) 1002 and a random access memory (RAM) 1003 are connected to the bus 1004.

[0288] An input unit 1006 configured with input devices such as a keyboard and a mouse through which a user inputs operation commands, an output unit 1007 which outputs a processing operation screen and an image of a processing result to a display device, a storage unit 1008 configured with a hard disk drive or the like for storing programs and various pieces of data, and a communication unit 1009, configured with a local area network (LAN) adapter or the like, for executing a communication processing via a network represented by the Internet are connected to an input / output interface 1005. In addition, a drive 1010 for reading and writing data from and to a removable storage medium 1011 such as a magnetic disk (including a flexible disk), an optical disk (including a compact disc-read only memory (CD-ROM), and a digital versatile disc (DVD), a magneto-optical disk (including a mini disc (MD)), or a semiconductor memory is connected.

[0289] The CPU 1001 executes various processing in accordance with a programs stored in the ROM 1002 or a program that is read out from the removable storage medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 1008, and loaded into the RAM 1003 from the storage unit 1008. The RAM 1003 also appropriately stores data necessary for the CPU 1001 to execute various processes.

[0290] In the computer configured as described above, the CPU 1001 loads the program stored in the storage unit 1008, for example, into the RAM 1003 via the input / output interface 1005 and the bus 1004, and executes the program, thereby performing the above-mentioned series of processing.

[0291] The program executed by the computer (CPU 1001) can be recorded, for example, on the removable storage medium 1011 which is a package medium or the like, and provided. The program can also be provided via a wired or wireless transmission medium, such as a local area network, the Internet, or digital satellite broadcasting.

[0292] In the computer, the program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable storage medium 1011 into the drive 1010. The program can also be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Other than the above manners, the program can be pre-installed in the ROM 1002 or the storage unit 1008.

[0293] The program executed by the computer may be a program in which time series processing is performed in the order described in the present specification, or may be a program in which processing is performed in parallel or at a required timing, such as at a time when called.

[0294] Further, the CPU 1001 of the general-purpose computer in Fig. 31 implements the functions of the gesture determination unit 91, the line-of-sight estimation unit 92, the operation input estimation unit 93, the vehicle equipment information acquisition unit 94, the design information acquisition unit 95, the design adjustment unit 96, and the display control unit 97 in the control unit 71 of the information processing device 51 in Fig. 6.

[0295] Additionally, in the present specification, a system refers to a collection of a plurality of components (devices, modules (parts), and the like), regardless of whether all of the components are in the same housing. Thus, a plurality of devices accommodated in separate housings and connected via a network, and a single device in which a plurality of modules are accommodated in a single housing, are each referred to as a system.

[0296] Furthermore, embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present disclosure.

[0297] For example, the present disclosure can adopt a configuration of cloud computing in which a single function is shared and processed jointly by a plurality of devices via a network.

[0298] Furthermore, the steps described in the above-described flowcharts can be executed by a single device, or can be shared and executed by a plurality of devices.

[0299] Furthermore, when a single step includes multiple processing tasks, the multiple processing tasks included in the single step can be executed by a single device, or can be shared and executed by a plurality of devices.

[0300] The present disclosure may also be configured as follows. <1> An information processing device including a display control unit configured to control projection display of a display target by a projection unit, based vehicle equipment information of a vehicle. <2> The information processing device according to <1>, in which the display control unit controls display of the display target by adjusting design information of the display target, based on the vehicle equipment information of the vehicle. <3> The information processing device according to <2>, in which the display target is an icon for implementing a user interface (UI) operation, the vehicle equipment information is information on an installation position of a steering wheel, and the display control unit controls the display of the display target by adjusting a size being the design information of the icon, in accordance with the installation position of the steering wheel. <4> The information processing device according to <3>, in which the display control unit controls the display of the display target by adjusting the icon projected at a position farthest from a driver of the vehicle to a size that allows for gazing by the driver for a predetermined period of time or longer, based on information on the installation position of the steering wheel. <5> The information processing device according to <3>, in which the display control unit controls the display of the display target by adjusting the size and a layout of the icon, based on information on the installation position of the steering wheel. <6> The information processing device according to <5>, in which the display control unit lays out the icon at a position near a front of the driver of the vehicle, based on the information on the installation position of the steering wheel. <7> The information processing device according to <5>, in which the display control unit stores information on the size and the layout of the icon for each installation position of the steering wheel as a table, and controls the display of the display target by reading out the table based on the information on the installation position of the steering wheel, and adjusting the size and the layout of the icon, based on the information of the read-out table. <8> The information processing device according to <3>, in which the display control unit controls the display of the display target by adjusting a size of a hit area which includes the icon and in which the driver of the vehicle is recognized as gazing at the icon, in accordance with the installation position of the steering wheel. <9> The information processing device according to <3>, in which the display target further includes map information in addition to the icon, and the display control unit controls the display of the display target by adjusting the size of the icon and performing adjustment so that the map information is laid out at a position near the front of the driver of the vehicle, in accordance with the installation position of the steering wheel. <10> The information processing device according to <2>, in which the display target is an icon and character information for implementing a user interface (UI) operation, the vehicle equipment information is optical characteristics related to projection of the display target, and the display control unit controls the display of the display target by determining an amount of blurring to be applied to the display target, based on an amount of intrinsic blurring specified in accordance with the optical characteristics, and adjusting the design information of the display target with the determined amount of blurring. <11> The information processing device according to <10>, in which the display control unit controls the display of the display target by storing the amount of intrinsic blurring for each piece of the vehicle equipment information, reading out the amount of intrinsic blurring, based on the vehicle equipment information, determining an amount of blurring to be applied to the display target, based on the read-out amount of intrinsic blurring, and adjusting the design information of the display target with the determined amount of blurring. <12> The information processing device according to <2>, in which the display target is character information, the vehicle equipment information is optical characteristics related to projection of the display target, and the display control unit controls the display of the display target by adjusting a font of the character information being the display target, based on the optical characteristics. <13> The information processing device according to <12>, in which the font includes a weight and a size, and the display control unit controls the display of the display target by storing, based on the optical characteristics, visibility limits of the weight and the size when the character information is projected by the projection unit, and adjusting at least one of the weight or the size so that the visibility limits of the weight and the size are exceeded when the character information being the display target is projected by the projection unit. <14> The information processing device according to <12>, in which the font includes a character spacing, and the display control unit controls the display of the display target by storing, based on the optical characteristics, a visibility limit of the character spacing when the character information is projected by the projection unit, and adjusting the character spacing so that the visibility limit of the character spacing is exceeded when the character information being the display target is projected by the projection unit. <15> The information processing device according to <12>, in which the font includes a character color, and the display control unit controls the display of the display target by storing a visibility limit of the character color when the character information is projected by the projection unit, based on the optical characteristics, and adjusting the character color so that the visibility limit of the character color is exceeded when the character information being the display target is projected by the projection unit. <16> The information processing device according to <2>, in which the display target is an icon for implementing a user interface (UI) operation, the vehicle equipment information is optical characteristics related to projection of the display target, and the display control unit controls the display of the display target by adjusting the design information by switching a resolution of the icon, based on the optical characteristics. <17> The information processing device according to <16>, in which the display control unit controls the display of the display target by storing a plurality of icon sets with different resolutions, and adjusting the design information by switching between the plurality of icon sets, based on the optical characteristics. <18> The information processing device according to <17>, in which the display control unit controls the display of the display target by storing a table indicating a relationship between each distinguishable pixel amount corresponding to the optical characteristics and a corresponding one of the plurality of icon sets with a usable resolution, and adjusting the design information by switching between the plurality of icon sets with the distinguishable pixel amount corresponding to the optical characteristics, based on the table. <19> An information processing method including performing display control processing for controlling projection display of a display target, based on vehicle equipment information of a vehicle. <20> An information processing system including a display control unit configured to control projection display of a display target by a projection unit, based on vehicle equipment information of a vehicle. <21> A program for causing a computer to function as a display control unit configured to control projection display of a display target by a projection unit, based on vehicle equipment information of a vehicle.

[0301] 31 UI operation system 41 Windshield 51 Information processing device 52 Projection device 53 Imaging unit 91 Gesture determination unit 92 Line-of-sight estimation unit 93 Operation input estimation unit 94 Vehicle equipment information acquisition unit 95 Design information acquisition unit 96 Design adjustment unit 97 Display control unit 131 Vehicle equipment information 132 Design information

Claims

1. An information processing device comprising: processing circuitry configured to control display of a display target in a head-up display of a vehicle based on vehicle equipment information of the vehicle.

2. The information processing device according to claim 1, wherein the processing circuitry controls display of the display target by adjusting design information of the display target, based on the vehicle equipment information of the vehicle.

3. The information processing device according to claim 2, wherein the display target is an icon for a user interface (UI) operation, the vehicle equipment information is information on an installation position of a steering wheel of the vehicle, and the processing circuitry controls the display of the display target by adjusting a size of the icon in accordance with the installation position of the steering wheel, the size of the icon being the design information.

4. The information processing device according to claim 3, wherein the processing circuitry controls the display of the display target by adjusting the size of the icon such that the driver looks at the icon for a predetermined period of time or longer without deterioration of operability, based on information on the installation position of the steering wheel.

5. The information processing device according to claim 3, wherein the processing circuitry controls the display of the display target by adjusting the size and a layout of the icon, based on information on the installation position of the steering wheel.

6. The information processing device according to claim 5, wherein the processing circuitry controls a layout the icon at a position near a front of the driver of the vehicle, based on the information on the installation position of the steering wheel.

7. The information processing device according to claim 5, wherein the processing circuitry is further configured to store information on the size and the layout of the icon for each installation position of the steering wheel as a table, control the display of the display target by reading the table based on the information on the installation position of the steering wheel, and adjust the size and the layout of the icon based on the information of the table.

8. The information processing device according to claim 3, wherein the processing circuitry controls the display of the display target by adjusting a size of a hit area, which includes the icon and in which the driver of the vehicle is recognized as looking at the icon, in accordance with the installation position of the steering wheel.

9. The information processing device according to claim 3, wherein the display target further includes map information, and the processing circuitry controls the display of the display target by adjusting the size of the icon and performing adjustment so that the map information is laid out at a position near the front of the driver of the vehicle, in accordance with the installation position of the steering wheel.

10. The information processing device according to claim 2, wherein the display target is an icon and character information for a user interface (UI) operation, the vehicle equipment information includes optical characteristics related to display of the display target, and the processing circuitry controls the display of the display target by: determining an amount of blurring for application to the display target, based on an amount of intrinsic blurring specified in accordance with the optical characteristics, and adjusting the design information of the display target with the determined amount of blurring.

11. The information processing device according to claim 10, wherein the processing circuitry controls the display of the display target by: storing an amount of intrinsic blurring for each piece of the vehicle equipment information, reading out the amount of intrinsic blurring based on the vehicle equipment information, determining the amount of blurring to be applied to the display target based on the read-out amount of intrinsic blurring, and adjusting the design information of the display target with the determined amount of blurring.

12. The information processing device according to claim 2, wherein the display target is character information, the vehicle equipment information includes optical characteristics related to display of the display target, and the processing circuitry controls the display of the display target by adjusting a font of the character information based on the optical characteristics.

13. The information processing device according to claim 12, wherein the font includes a weight and a size, and the processing circuitry controls the display of the display target by: storing, based on the optical characteristics, visibility limits of the weight and the size when the character information is displayed, and adjusting at least one of the weight or the size so that the visibility limits of the weight and the size are exceeded when the character information is displayed.

14. The information processing device according to claim 12, wherein the font includes a character spacing, and the processing circuitry controls the display of the display target by: storing, based on the optical characteristics, a visibility limit of the character spacing when the character information is displayed, and adjusting the character spacing so that the visibility limit of the character spacing is exceeded when the character information is displayed.

15. The information processing device according to claim 12, wherein the font includes a character color, and the processing circuitry controls the display of the display target by: storing a visibility limit of the character color when the character information is displayed based on the optical characteristics, and adjusting the character color so that the visibility limit of the character color is exceeded when the character information is displayed.

16. The information processing device according to claim 2, wherein the display target is an icon for a user interface (UI) operation, the vehicle equipment information includes optical characteristics related to display of the display target, and the processing circuitry controls the display of the display target by adjusting the design information by switching a resolution of the icon, based on the optical characteristics.

17. The information processing device according to claim 16, wherein the processing circuitry controls the display of the display target by: storing a plurality of icon sets with different resolutions, and adjusting the design information by switching between the plurality of icon sets, based on the optical characteristics.

18. The information processing device according to claim 17, wherein the processing controls the display of the display target by: storing a table indicating a relationship between each distinguishable pixel amount corresponding to the optical characteristics and a corresponding one of the plurality of icon sets with a usable resolution, and adjusting the design information by switching between the plurality of icon sets with the distinguishable pixel amount corresponding to the optical characteristics, based on the table.

19. An information processing method, comprising: performing display control processing based on vehicle equipment information of a vehicle; and controlling a display of the vehicle to display a display target based on the performed display control processing.

20. An information processing system in a vehicle, the information processing system comprising: a display; and processing circuitry configured to control the display to display of a display target based on vehicle equipment information of the vehicle.

Images