Information processing device, information processing method, program, and information processing system
Patent Information
- Application Number
- IN202427030621
- Authority / Receiving Office
- IN · IN
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-10-25
- Filing Date
- 2024-04-16
- Publication Date
- 2026-10-23
- Estimated Expiration
- 2042-09-06
AI Technical Summary
Conventional in-vehicle display systems face visibility issues due to frequent changes in drawing color to match the changing landscape, leading to potential deterioration in content visibility.
An information processing apparatus and method that predicts environmental transitions outside the vehicle and optimizes the drawing method of content, including position and color, based on statistical processing of environmental conditions to maintain stable visibility.
The solution effectively suppresses visibility deterioration by determining an optimal drawing method that considers environmental changes, ensuring the content remains visible and legible during vehicle travel.
Abstract
Description
Field
[0001] The present invention relates to an informationprocessing apparatus, an information processing method, aprogram, and an information processing system.Background
[0002] An in-vehicle display that displays a content ona windshield or the like is known. In this type of display,a content is superimposed on a traveling landscape outsidethe vehicle. Depending on a drawing position of the contentand a drawing color thereof, there is a possibility that thecontent is mixed in the traveling landscape. Therefore,Patent Literature 1 proposes that the drawing color of thecontent is appropriately controlled depending on thetraveling landscape so as to easily identify the contentfrom the background.Citation ListPatent Literature
[0003] Patent Literature 1: JP 2019-162932 ASummaryTechnical Problem
[0004] In the conventional method, a drawing color of acontent is appropriately changed depending on a changingtraveling landscape. However, there is a possibility thatvisibility of the content deteriorates due a frequent changein the drawing color.
[0005] Therefore, the present disclosure proposes aninformation processing apparatus, an information processingmethod, a program, and an information processing systemcapable of suppressing deterioration in visibility of acontent due to a change in environment outside a vehicle.Solution to Problem
[0006] According to the present disclosure, aninformation processing apparatus is provided that comprises:an environment information processing unit configured topredict a transition of an environment outside a vehicleduring viewing of a content; andan optimization processing unit configured to determinea drawing method of the content based on the predictedtransition of the environment outside the vehicle.According to the present disclosure, an informationprocessing method in which an information process of theinformation processing apparatus is executed by a computer,a program causing a computer to perform the informationprocess of the information processing apparatus, and aninformation processing system having a video display unitconfigured to draw the content based on the drawing methoddetermined by the information processing apparatus, areprovided.Brief Description of Drawings
[0007] FIG. 1 is a diagram illustrating an outline ofdisplay control of a content.FIG. 2 is a block diagram illustrating a configurationexample of a vehicle control system.FIG. 3 is a diagram illustrating an example of a sensingregion of an external recognition sensor.FIG. 4 is a diagram illustrating a configuration exampleof an in-vehicle sensor and a video display unit.FIG. 5 is a diagram illustrating a display example ofa video provided as a content.FIG. 6 is a diagram illustrating another configurationexample of the video display unit.FIG. 7 is a diagram illustrating a display example ofa video provided as a content.FIG. 8 is a diagram illustrating another configurationexample of the video display unit.FIG. 9 is a diagram illustrating an example of afunctional configuration related to video display of avehicle.FIG. 10 is a diagram illustrating an example of acontrol flow related to determination processing of a drawingmethod.FIG. 11 is a diagram illustrating a display example ofa content on a transparent screen.FIG. 12 is a diagram illustrating a display example ina case where the sun is directly viewed through thetransparent screen.FIG. 13 is a diagram illustrating a display example ofa content on a non-transparent screen.FIG. 14 is a diagram illustrating the display exampleof the content on the non-transparent screen.FIG. 15 is an explanatory diagram of a prediction periodduring which a transition of an environment outside thevehicle is predicted.FIG. 16 is a flowchart illustrating determinationprocessing of a drawing method.FIG. 17 is a diagram illustrating a table used for thedetermination processing of the drawing method.FIG. 18 is a flow illustrating a specific example of adetermination method of a drawing position for each time.FIG. 19 is a flow illustrating a specific example of adetermination method of a drawing color for each time.FIG. 20 is a flow illustrating a specific example ofoptimization by statistical processing.FIG. 21 is a diagram illustrating change processing ofthe drawing method.FIG. 22 is a diagram illustrating an example of changingthe drawing method according to a surrounding situation ofthe vehicle.FIG. 23 is a diagram illustrating an example of changingthe drawing method according to the surrounding situation ofthe vehicle.FIG. 24 is a diagram illustrating an example of a flowof a countermeasure against visually induced motion sickness.FIG. 25 is a diagram illustrating an example of changinga drawing method according to a viewing environment in thevehicle.FIG. 26 is a diagram illustrating an example of a flowof a countermeasure against opening and closing of a window.FIG. 27 is a diagram illustrating determinationprocessing of a drawing method without prediction of atransition of the environment outside the vehicle.FIG. 28 is a flowchart illustrating the determinationprocessing of the drawing method without prediction of thetransition of the environment outside the vehicle.FIG. 29 is a diagram illustrating a determination methodof a drawing color in consideration of a content.FIG. 30 is a diagram illustrating response to a lightsource other than the sun.FIG. 31 is a flowchart illustrating determinationprocessing of a drawing method based on a movement situationof a light source.Description of Embodiments
[0008] Hereinafter, embodiments of the present disclosurewill be described in detail with reference to the drawings.In the following embodiments, the same parts will be denotedby the same reference numerals, and a redundant descriptionthereof will be omitted.
[0009] It is noted that a description will be given inthe following order.[1. Outline of display control of content][2. Configuration example of vehicle control system][3. Sensing region of external recognition sensor][4. Configuration example of video display unit][5. Functional configuration related to video display][6. Control flow][7. Display control example][7-1. Display of content on transparent screen][7-2. Superimposition avoidance with sun][7-3. Display of content on non-transparent screen][7-4. Optimization of drawing method based onprediction of transition of environment outside vehicle][7-4-1. Prediction period][7-4-2. Determination of drawing method][7-4-3. Specific example of drawing position anddrawing color][7-5. Change of drawing method][7-5-1. Countermeasure against visually inducedmotion sickness][7-5-2. Countermeasure against shape change ofprojection target][7-6. Determination of drawing method withoutprediction][7-7. Determination of drawing color in considerationof content][7-8. Response to light source other than sun][8. Effects]
[0010] [1. Outline of display control of content]FIG. 1 is a diagram illustrating an outline of displaycontrol of a content CT.
[0011] A vehicle 1 includes a vehicle control system 11capable of performing display control of the content CT. Apassenger PA can enjoy various contents CT displayed on avideo display unit 93. The content CT includes not only acontent for driving assistance but also a content forentertainment intended to entertain the passenger PA.Examples of the content CT for entertainment include a movie,music, a game, news, a mail, a map / tourist guide, an SNS,and a chat.
[0012] For example, the video display unit 93 can switchbetween a display state and a transparent state. The videodisplay unit 93 is disposed so as to surround a vehicleinterior space along an inner wall or a partition wall ofthe vehicle 1. The video display unit 93 presents thecontent CT in the display state, and causes the passenger PAto see through the outside of the vehicle in the transparentstate.
[0013] The "display state" includes a semi-transparentstate in which information such as a video displayed on ascene outside the vehicle can be superimposed, such asaugmented reality (AR). The video display unit 93 canperform AR display by superimposing the video on the sceneoutside the vehicle. For example, the video display unit 93adjusts display brightness or transparency or haze of thevideo display unit 93 according to brightness outside thevehicle. Accordingly, visibility of the content CT can beenhanced.
[0014] The vehicle control system 11 acquires pathinformation on a travel path of the vehicle 1. The pathinformation includes a movement path, a map around themovement path, and weather information. Based on the pathinformation, the vehicle control system 11 predicts how theenvironment outside the vehicle transitions while thecontent CT is viewed. The vehicle control system 11calculates, based on a predetermined standard, a drawingcondition (an appropriate drawing condition) suitable forthe environment outside the vehicle for each time. Thedrawing condition includes a drawing position and a drawingcolor. The vehicle control system 11 determines a stabledrawing method (an optimum drawing condition) by performingstatistical processing on the appropriate drawing conditionfor each time.
[0015] [2. Configuration example of vehicle controlsystem]FIG. 2 is a block diagram illustrating a configurationexample of the vehicle control system 11.
[0016] The vehicle control system 11 is provided in thevehicle 1 and performs processing related to travelassistance and automatic driving of the vehicle 1.
[0017] The vehicle control system 11 includes a vehiclecontrol electronic control unit (ECU) 21, a communicationunit 22, a map information accumulation unit 23, a positioninformation acquisition unit 24, an external recognitionsensor 25, an in-vehicle sensor 26, a vehicle sensor 27, astorage unit 28, a travel assistance / automatic drivingcontrol unit 29, a driver monitoring system (DMS) 30, a humanmachine interface (HMI) 31, and a vehicle control unit 32.
[0018] The vehicle control ECU 21, the communication unit22, the map information accumulation unit 23, the positioninformation acquisition unit 24, the external recognitionsensor 25, the in-vehicle sensor 26, the vehicle sensor 27,the storage unit 28, the travel assistance / automatic drivingcontrol unit 29, the driver monitoring system (DMS) 30, thehuman machine interface (HMI) 31, and the vehicle controlunit 32 are communicably connected to each other via acommunication network 41. The communication network 41includes, for example, an in-vehicle communication network,a bus, or the like conforming to a digital bidirectionalcommunication standard such as a controller area network(CAN), a local interconnect network (LIN), a local areanetwork (LAN), FlexRay (registered trademark), or Ethernet(registered trademark). The communication network 41 may beselectively used depending on the type of data to betransmitted. For example, the CAN may be applied to datarelated to vehicle control, and the Ethernet may be appliedto large-capacity data. It is noted that each unit of thevehicle control system 11 may be directly connected not viathe communication network 41 but by using wirelesscommunication that assumes communication at a relativelyshort distance, such as near field communication (NFC) orBluetooth (registered trademark).
[0019] It is noted that, hereinafter, when each unit ofthe vehicle control system 11 performs communication via thecommunication network 41, a description of the communicationnetwork 41 will be omitted. For example, when the vehiclecontrol ECU 21 and the communication unit 22 performcommunication via the communication network 41, it is simplydescribed that the vehicle control ECU 21 and thecommunication unit 22 perform communication.
[0020] The vehicle control ECU 21 includes, for example,various processors such as a central processing unit (CPU)and a micro processing unit (MPU). The vehicle control ECU21 controls a whole or a part of the functions of the vehiclecontrol system 11.
[0021] The communication unit 22 communicates withvarious devices inside and outside the vehicle, othervehicles, servers, base stations, and the like, and transmitsand receives various data. At this time, the communicationunit 22 can perform communication using a plurality ofcommunication methods.
[0022] Communication with the outside of the vehicleexecutable by the communication unit 22 will be schematicallydescribed. The communication unit 22 communicates with aserver (hereinafter, referred to as an external server) orthe like existing on an external network via a base stationor an access point by a wireless communication method suchas a 5th generation mobile communication system (5G), longterm evolution (LTE), or dedicated short rangecommunications (DSRC). The external network with which thecommunication unit 22 performs communication is, for example,the Internet, a cloud network, a network unique to a company,or the like. The communication method performed by thecommunication unit 22 with respect to the external networkis not particularly limited as long as the communicationmethod is a wireless communication method capable ofperforming digital bidirectional communication at acommunication speed equal to or higher than a predeterminedspeed and at a distance equal to or longer than apredetermined distance.
[0023] Furthermore, for example, the communication unit22 can communicate with a terminal existing in the vicinityof the host vehicle using a peer to peer (P2P) technology.The terminal existing in the vicinity of the host vehicleis, for example, a terminal worn by a moving body moving ata relatively low speed such as a pedestrian or a bicycle, aterminal installed in a store or the like with a fixedposition, or a machine type communication (MTC) terminal.Furthermore, the communication unit 22 can also perform V2Xcommunication. The V2X communication refers to, for example,communication between a host vehicle and another vehicle,such as vehicle to vehicle communication, that is,communication between the host vehicle and another vehicle,vehicle to infrastructure communication, that is,communication between the host vehicle and a roadside deviceor the like, vehicle to home communication, that is,communication between the host vehicle and home, and vehicleto pedestrian communication, that is, communication betweenthe host vehicle and a terminal or the like possessed by apedestrian.
[0024] For example, the communication unit 22 can receivea program for updating software for controlling the operationof the vehicle control system 11 from the outside (Over TheAir). The communication unit 22 can further receive mapinformation, traffic information, information around thevehicle 1, and the like from the outside. Furthermore, forexample, the communication unit 22 can transmit informationregarding the vehicle 1, the information around the vehicle1, and the like to the outside. Examples of the informationon the vehicle 1 transmitted to the outside by thecommunication unit 22 include data indicating the state ofthe vehicle 1, a recognition result by a recognition unit73, and the like. Furthermore, for example, thecommunication unit 22 performs communication correspondingto a vehicle emergency call system such as an eCall.
[0025] For example, the communication unit 22 receives anelectromagnetic wave transmitted by a road trafficinformation communication system (vehicle information andcommunication system (VICS) (registered trademark)) such asa radio wave beacon, an optical beacon, or FM multiplexbroadcasting.
[0026] Communication with the inside of the vehicleexecutable by the communication unit 22 will be schematicallydescribed. The communication unit 22 can communicate witheach device in the vehicle using, for example, wirelesscommunication. The communication unit 22 can performwireless communication with an in-vehicle device by acommunication method capable of performing digitalbidirectional communication at a predetermined communicationspeed or higher by wireless communication, such as wirelessLAN, Bluetooth, NFC, or wireless USB (WUSB). The presentinvention is not limited thereto, and the communication unit22 can also communicate with each device in the vehicle usingwired communication. For example, the communication unit 22can communicate with each device in the vehicle by wiredcommunication via a cable connected to a connection terminal(not illustrated). The communication unit 22 cancommunicate with each device in the vehicle by acommunication method capable of performing digitalbidirectional communication at a predetermined communicationspeed or higher by wired communication, such as universalserial bus (USB), high-definition multimedia interface(HDMI) (registered trademark), or mobile high-definitionlink (MHL).
[0027] Here, the in-vehicle device refers to, for example,a device that is not connected to the communication network41 in the vehicle. As the in-vehicle device, for example,a mobile device or a wearable device possessed by thepassenger PA such as a driver, an information device broughtinto the vehicle and temporarily installed, or the like isassumed.
[0028] The map information accumulation unit 23accumulates one or both of a map acquired from the outsideand a map created by the vehicle 1. For example, the mapinformation accumulation unit 23 accumulates a threedimensionalhighly accurate map, a global map having loweraccuracy than the highly accurate map and covering a widearea, and the like.
[0029] The highly accurate map is, for example, a dynamicmap, a point cloud map, a vector map, or the like. Thedynamic map is, for example, a map including four layers ofdynamic information, semi-dynamic information, semi-staticinformation, and static information, and is provided to thevehicle 1 from an external server or the like. The pointcloud map is a map including point clouds (point cloud data).The vector map is, for example, a map in which trafficinformation such as a lane and a position of a traffic lightis associated with a point cloud map and adapted to anadvanced driver assistance system (ADAS) or autonomousdriving (AD).
[0030] The point cloud map and the vector map may beprovided from, for example, an external server or the like,or may be created by the vehicle 1 as a map for performingmatching with a local map to be described later based on asensing result by a camera 51, a radar 52, a LiDAR 53, orthe like, and may be accumulated in the map informationaccumulation unit 23. In addition, in a case where thehighly accurate map is provided from an external server orthe like, for example, map data of several hundred meterssquare regarding a planned path on which the vehicle 1travels from now is acquired from the external server or thelike in order to reduce the communication capacity.
[0031] The position information acquisition unit 24receives a global navigation satellite system (GNSS) signalfrom a GNSS satellite, and acquires position information ofthe vehicle 1. The acquired position information is suppliedto the travel assistance / automatic driving control unit 29.It is noted that the position information acquisition unit24 is not limited to the method using the GNSS signal, andmay acquire the position information using, for example, abeacon.
[0032] The external recognition sensor 25 includesvarious sensors used for recognizing a situation outside thevehicle 1, and supplies sensor data from each sensor to eachunit of the vehicle control system 11. The type and numberof sensors provided in the external recognition sensor 25are freely selected.
[0033] For example, the external recognition sensor 25includes the camera 51, the radar 52, the light detectionand ranging, laser imaging detection and ranging (LiDAR) 53,and an ultrasonic sensor 54. The present invention is notlimited thereto, and the external recognition sensor 25 maybe configured to include one or more types of sensors amongthe camera 51, the radar 52, the LiDAR 53, and the ultrasonicsensor 54. The numbers of the cameras 51, the radars 52,the LiDAR 53, and the ultrasonic sensors 54 are notparticularly limited as long as they can be practicallyinstalled in the vehicle 1. Furthermore, the type of sensorprovided in the external recognition sensor 25 is not limitedto this example, and the external recognition sensor 25 mayinclude another type of sensor. An example of a sensingregion of each sensor provided in the external recognitionsensor 25 will be described later.
[0034] It is noted that a photographing method of thecamera 51 is not particularly limited. For example, camerasof various photographing methods such as a time of flight(ToF) camera, a stereo camera, a monocular camera, and aninfrared camera, which are photographing methods capable ofperforming distance measurement, can be applied to the camera51 as necessary. The present invention is not limitedthereto, and the camera 51 may simply acquire a capturedimage regardless of distance measurement.
[0035] Furthermore, for example, the external recognitionsensor 25 can include an environment sensor for detectingthe environment of the vehicle 1. The environment sensor isa sensor for detecting an environment such as climate,weather, and brightness, and can include various sensorssuch as a raindrop sensor, a fog sensor, a sunshine sensor,a snow sensor, and an illuminance sensor.
[0036] Furthermore, for example, the external recognitionsensor 25 includes a microphone used for detecting a soundaround the vehicle 1, a position of a sound source, and thelike.
[0037] The in-vehicle sensor 26 includes various sensorsfor detecting information inside the vehicle, and suppliessensor data from each sensor to each unit of the vehiclecontrol system 11. The type and number of various sensorsincluded in the in-vehicle sensor 26 are not particularlylimited as long as they are types and numbers that can bepractically installed in the vehicle 1.
[0038] For example, the in-vehicle sensor 26 can includeone or more sensors among a camera, a radar, a seating sensor,a steering wheel sensor, a microphone, and a biologicalsensor. As a camera included in the in-vehicle sensor 26,for example, cameras having various photographing methodscapable of measuring a distance, such as a ToF camera, astereo camera, a monocular camera, and an infrared camera,can be used. The present invention is not limited thereto,and the camera included in the in-vehicle sensor 26 maysimply acquire a captured image regardless of distancemeasurement. The biological sensor included in the invehiclesensor 26 is provided, for example, on a seat, asteering wheel, or the like, and detects various types ofbiological information of a passenger such as a driver.
[0039] The vehicle sensor 27 includes various sensors fordetecting the state of the vehicle 1, and supplies sensordata from each sensor to each unit of the vehicle controlsystem 11. The type and number of various sensors includedin the vehicle sensor 27 are not particularly limited aslong as they are types and numbers that can be practicallyinstalled in the vehicle 1.
[0040] For example, the vehicle sensor 27 includes a speedsensor, an acceleration sensor, an angular velocity sensor(a gyro sensor), and an inertial measurement unit (IMU)integrating these sensors. For example, the vehicle sensor27 includes a steering angle sensor that detects a steeringangle of a steering wheel, a yaw rate sensor, an acceleratorsensor that detects an operation amount of an acceleratorpedal, and a brake sensor that detects an operation amountof a brake pedal. For example, the vehicle sensor 27includes a rotation sensor that detects the rotation speedof an engine or a motor, an air pressure sensor that detectsair pressure of a tire, a slip rate sensor that detects aslip rate of a tire, and a wheel speed sensor that detectsthe rotation speed of a wheel. For example, the vehiclesensor 27 includes a battery sensor that detects a remainingamount and a temperature of a battery, and an impact sensorthat detects an external impact. The vehicle sensor 27includes a sensor that detects a driving status DS.
[0041] The storage unit 28 includes at least one of anonvolatile storage medium and a volatile storage medium,and stores data and a program. The storage unit 28 is usedas, for example, an electrically erasable programmable readonly memory (EEPROM) and a random access memory (RAM), anda magnetic storage device such as a hard disc drive (HDD),a semiconductor storage device, an optical storage device,and a magneto optical storage device can be applied as astorage medium. The storage unit 28 stores various programsand data used by each unit of the vehicle control system 11.For example, the storage unit 28 includes an event datarecorder (EDR) and a data storage system for automateddriving (DSSAD), and stores information of the vehicle 1before and after an event such as an accident and informationacquired by the in-vehicle sensor 26.
[0042] The travel assistance / automatic driving controlunit 29 controls travel assistance and automatic driving ofthe vehicle 1. For example, the travel assistance / automaticdriving control unit 29 includes an analysis unit 61, anaction planning unit 62, and an operation control unit 63.
[0043] The analysis unit 61 performs analysis processingof a situation of the vehicle 1 and the surroundings. Theanalysis unit 61 includes a self-position estimation unit71, a sensor fusion unit 72, and a recognition unit 73.
[0044] The self-position estimation unit 71 estimates theself-position of the vehicle 1 based on sensor data from theexternal recognition sensor 25 and highly accurate mapaccumulated in the map information accumulation unit 23.For example, the self-position estimation unit 71 generatesa local map based on the sensor data from the externalrecognition sensor 25, and estimates the self-position ofthe vehicle 1 by matching the local map with the highlyaccurate map. The position of the vehicle 1 is based on,for example, the center of a rear wheel pair axle.
[0045] The local map is, for example, a three-dimensionalhighly accurate map created using a technology such assimultaneous localization and mapping (SLAM), an occupancygrid map, or the like. The three-dimensional highly accuratemap is, for example, the above-described point cloud map orthe like. The occupancy grid map is a map in which a threedimensionalor two-dimensional space around the vehicle 1 isdivided into grids (grids) having a predetermined size, andan occupancy state of an object is indicated in units ofgrids. The occupancy state of the object is indicated by,for example, presence or absence or existence probability ofthe object. For example, the local map is also used fordetection processing and recognition processing of asituation outside the vehicle 1 by the recognition unit 73.
[0046] It is noted that the self-position estimation unit71 may estimate the self-position of the vehicle 1 based onthe position information acquired by the positioninformation acquisition unit 24 and the sensor data from thevehicle sensor 27.
[0047] The sensor fusion unit 72 performs sensor fusionprocessing of combining a plurality of different types ofsensor data (for example, image data supplied from the camera51 and sensor data supplied from the radar 52) and obtainingnew information. Methods for combining different types ofsensor data include integration, fusion, association, andthe like.
[0048] The recognition unit 73 executes detectionprocessing for detecting a situation outside the vehicle 1and recognition processing for recognizing the situationoutside the vehicle 1.
[0049] For example, the recognition unit 73 performs thedetection processing and the recognition processing of thesituation outside the vehicle 1 based on information fromthe external recognition sensor 25, information from theself-position estimation unit 71, information from thesensor fusion unit 72, and the like.
[0050] Specifically, for example, the recognition unit 73performs the detection processing, the recognitionprocessing, and the like of an object around the vehicle 1.The object detection processing is, for example, processingof detecting the presence or absence, size, shape, position,movement, and the like of an object. The object recognitionprocessing is, for example, processing of recognizing anattribute such as a type of an object or identifying aspecific object. However, the detection processing and therecognition processing are not necessarily clearly divided,and may overlap each other.
[0051] For example, the recognition unit 73 detects anobject around the vehicle 1 by performing clustering toclassify point clouds based on the sensor data by the radar52, the LiDAR 53, or the like into clusters of point clouds.As a result, the presence or absence, size, shape, andposition of the object around the vehicle 1 are detected.
[0052] For example, the recognition unit 73 detects themovement of the object around the vehicle 1 by performingtracking that follows the movement of the cluster of thepoint cloud classified by clustering. As a result, the speedand the traveling direction (movement vector) of the objectaround the vehicle 1 are detected.
[0053] For example, the recognition unit 73 detects orrecognizes a vehicle, a person, a bicycle, an obstacle, astructure, a road, a traffic light, a traffic sign, a roadsign, and the like based on the image data supplied from thecamera 51. Furthermore, the recognition unit 73 mayrecognize the type of the object around the vehicle 1 byperforming recognition processing such as semanticsegmentation.
[0054] For example, the recognition unit 73 can performrecognition processing of traffic rules around the vehicle1 based on a map accumulated in the map informationaccumulation unit 23, an estimation result of the selfpositionby the self-position estimation unit 71, and arecognition result of an object around the vehicle 1 by therecognition unit 73. Through this processing, therecognition unit 73 can recognize the position and the stateof a traffic light, the contents of a traffic sign and aroad mark, the contents of a traffic regulation, a travelablelane, and the like.
[0055] For example, the recognition unit 73 can performrecognition processing of an environment around the vehicle1. As the surrounding environment to be recognized by therecognition unit 73, the presence or absence of a pedestrianand a surrounding vehicle, the weather, the temperature, thehumidity, the brightness, the state of the road surface, andthe like are assumed.
[0056] The action planning unit 62 creates an action planof the vehicle 1. For example, the action planning unit 62creates the action plan by performing processing of globalpath planning and path following.
[0057] It is noted that the global path planning isprocessing of planning a rough path from the start to thegoal. This global path planning is referred to as atrajectory planning, and includes processing of performinga local path planning that enables safe and smooth travelingin the vicinity of the vehicle 1 in consideration of movementcharacteristics of the vehicle 1 in the planned path.
[0058] The path following is processing of planning anoperation of enabling the vehicle to safely and accuratelytravel the path planned by the global path planning withina planned time. For example, the action planning unit 62can calculate the target speed and the target angularvelocity of the vehicle 1 based on the result of the pathfollowing processing.
[0059] The operation control unit 63 controls theoperation of the vehicle 1 in order to implement the actionplan created by the action planning unit 62.
[0060] For example, the operation control unit 63controls a steering control unit 81, a brake control unit82, and a drive control unit 83 included in the vehiclecontrol unit 32 to be described later, and performsacceleration / deceleration control and direction control soas to allow the vehicle 1 to travel on the trajectorycalculated by the trajectory planning. For example, theoperation control unit 63 performs cooperative control forthe purpose of implementing the functions of the ADAS suchas collision avoidance or impact mitigation, follow-uptraveling, vehicle speed maintaining traveling, collisionwarning of the host vehicle, lane deviation warning of thehost vehicle, and the like. For example, the operationcontrol unit 63 performs cooperative control for the purposeof automatic driving or the like in which a vehicleautonomously travels without depending on the operation ofa driver.
[0061] The DMS 30 performs authentication processing ofa driver, recognition processing of the state of a driver,and the like based on the sensor data from the in-vehiclesensor 26, input data input to the HMI 31 to be describedlater, and the like. As the state of the driver to berecognized, for example, a physical condition, a wakefulnesslevel, a concentration level, a fatigue level, a line-ofsightdirection, a drunkenness level, a driving operation,a posture, and the like are assumed.
[0062] It is noted that the DMS 30 may performauthentication processing of a passenger other than thedriver and recognition processing of the state of thepassenger. Furthermore, for example, the DMS 30 may performrecognition processing of a situation inside the vehiclebased on the sensor data from the in-vehicle sensor 26. Asthe situation inside the vehicle to be recognized, forexample, temperature, humidity, brightness, odor, and thelike are assumed.
[0063] The HMI 31 inputs various data, instructions, andthe like, and presents the various data to the passengerincluding the driver.
[0064] The data input by the HMI 31 will be schematicallydescribed. The HMI 31 includes an input device that enablea person to input data. The HMI 31 generates an input signalbased on data, an instruction, or the like input by the inputdevice, and supplies the input signal to each unit of thevehicle control system 11. The HMI 31 includes an operatorsuch as a touch panel, a button, a switch, and a lever asthe input device. The present invention is not limitedthereto, and the HMI 31 may further include an input devicecapable of inputting information by a method other thanmanual operation by voice, gesture, or the like. Furthermore,the HMI 31 may use, for example, a remote control deviceusing infrared rays or radio waves, or an external connectiondevice such as a mobile device or a wearable devicecorresponding to the operation of the vehicle control system11 as the input device.
[0065] Presentation of data by the HMI 31 will beschematically described. The HMI 31 generates visualinformation, auditory information, and tactile informationon the passenger or the outside of the vehicle. In addition,the HMI 31 performs output control for controlling an output,an output content, an output timing, an output method, andthe like of each piece of generated information. The HMI 31generates and outputs, for example, an operation screen, astate display of the vehicle 1, a warning display, an imagesuch as a monitor image indicating a situation around thevehicle 1, and information (the content CT) indicated bylight as the visual information. Further, the HMI 31generates and outputs, as the auditory information,information indicated by sounds such as voice guidance, awarning sound, and a warning message. Further, the HMI 31generates and outputs, as the tactile information,information given to a tactile sense of the passenger by,for example, force, vibration, movement, or the like.
[0066] As an output device in which the HMI 31 outputsthe visual information, for example, a display device thatpresents visual information by displaying an image by itselfor a projector device that presents visual information byprojecting an image can be applied. It is noted that thedisplay device may be a device that displays visualinformation in the field of view of the passenger, such asa head-up display, a transmissive display, or a wearabledevice having an augmented reality (AR) function, in additionto a display device having a normal display. Further, inthe HMI 31, a display device included in a navigation device,an instrument panel, a camera monitoring system (CMS), anelectronic mirror, a lamp, or the like provided in thevehicle 1 can also be used as an output device that outputsthe visual information.
[0067] As an output device in which the HMI 31 outputsthe auditory information, for example, an audio speaker, aheadphone, or an earphone can be applied.
[0068] As an output device in which the HMI 31 outputsthe tactile information, for example, a haptic element usinga haptic technology can be applied. The haptic element isprovided, for example, at a portion with which the passengerof the vehicle 1 comes into contact, such as a steering wheelor a seat.
[0069] The vehicle control unit 32 controls each unit ofthe vehicle 1. The vehicle control unit 32 includes thesteering control unit 81, the brake control unit 82, thedrive control unit 83, a body system control unit 84, a lightcontrol unit 85, and a horn control unit 86.
[0070] The steering control unit 81 detects and controlsthe state of the steering system of the vehicle 1. Thesteering system includes, for example, a steering mechanismincluding a steering wheel and the like, electric powersteering, and the like. The steering control unit 81includes, for example, a steering ECU that controls thesteering system, an actuator that drives the steering system,and the like.
[0071] The brake control unit 82 detects and controls thestate of a brake system of the vehicle 1. The brake systemincludes, for example, a brake mechanism including a brakepedal, an antilock brake system (ABS), a regenerative brakemechanism, and the like. The brake control unit 82 includes,for example, a brake ECU that controls the brake system, anactuator that drives the brake system, and the like.
[0072] The drive control unit 83 detects and controls thestate of a drive system of the vehicle 1. The drive systemincludes, for example, a driving force generation device forgenerating driving force such as an accelerator pedal, aninternal combustion engine, or a driving motor, a drivingforce transmission mechanism for transmitting the drivingforce to wheels, and the like. The drive control unit 83includes, for example, a drive ECU that controls the drivesystem, an actuator that drives the drive system, and thelike.
[0073] The body system control unit 84 detects andcontrols the state of a body system of the vehicle 1. Thebody system includes, for example, a keyless entry system,a smart key system, a power window device, a power seat, anair conditioner, an airbag, a seat belt, a shift lever, andthe like. The body system control unit 84 includes, forexample, a body system ECU that controls the body system, anactuator that drives the body system, and the like.
[0074] The light control unit 85 detects and controls thestates of various lights of the vehicle 1. As the light tobe controlled, for example, a headlight, a backlight, a foglight, a turn signal, a brake light, a projection, a displayof a bumper, and the like are assumed. The light controlunit 85 includes a light ECU that controls light, an actuatorthat drives the light, and the like.
[0075] The horn control unit 86 detects and controls thestate of a car horn of the vehicle 1. The horn control unit86 includes, for example, a horn ECU that controls a carhorn, an actuator that drives the car horn, and the like.
[0076] [3. Sensing region of external recognition sensor]FIG. 3 is a diagram illustrating an example of a sensingregion by the camera 51, the radar 52, the LiDAR 53, theultrasonic sensor 54, and the like of the externalrecognition sensor 25. It is noted that FIG. 3 schematicallyillustrates the vehicle 1 as viewed from above. When FIG.3 is viewed in a posture in which a reference sign isdisplayed in the correct direction, the left end side isdefined as the front end (front) side of the vehicle 1, andthe right end side is defined as the rear end (rear) side ofthe vehicle 1.
[0077] A sensing region 101F and a sensing region 101Billustrate examples of the sensing region of the ultrasonicsensor 54. The sensing region 101F covers the periphery ofthe front end of the vehicle 1 by the plurality of ultrasonicsensors 54. The sensing region 101B covers the periphery ofthe rear end of the vehicle 1 by the plurality of ultrasonicsensors 54.
[0078] Sensing results in the sensing region 101F and thesensing region 101B are used, for example, for parkingassistance of the vehicle 1.
[0079] Sensing regions 102F to 102B illustrate examplesof the sensing regions of the radar 52 for a short distanceor a middle distance. The sensing region 102F covers aposition farther than that of the sensing region 101F infront of the vehicle 1. The sensing region 102B covers aposition farther than that of the sensing region 101B behindthe vehicle 1. A sensing region 102L covers the rearperiphery of the left side surface of the vehicle 1. Asensing region 102R covers the rear periphery of the rightside surface of the vehicle 1.
[0080] A sensing result in the sensing region 102F isused, for example, to detect a vehicle, a pedestrian, or thelike existing in front of the vehicle 1. A sensing resultin the sensing region 102B is used, for example, to performa collision prevention function or the like behind thevehicle 1. Sensing results in the sensing region 102L andthe sensing region 102R are used, for example, to detect anobject existing in a blind spot on the side of the vehicle1.
[0081] Sensing regions 103F to 103B illustrate examplesof sensing regions by the camera 51. The sensing region103F covers a position farther than that of the sensingregion 102F in front of the vehicle 1. The sensing region103B covers a position farther than that of the sensingregion 102B behind the vehicle 1. A sensing region 103Lcovers the periphery of the left side surface of the vehicle1. A sensing region 103R covers the periphery of the rightside surface of the vehicle 1.
[0082] A sensing result in the sensing region 103F can beused for, for example, recognition of a traffic light or atraffic sign, a lane departure prevention assistance system,and an automatic headlight control system. A sensing resultin the sensing region 103B can be used for, for example,parking assistance and a surround view system. Sensingresults in the sensing region 103L and the sensing region103R can be used for, for example, a surround view system.
[0083] A sensing region 104 shows an example of a sensingregion of the LiDAR 53. The sensing region 104 covers aposition farther than that of the sensing region 103F infront of the vehicle 1. Meanwhile, the sensing region 104has a narrower range in the left-and-right direction thanthat of the sensing region 103F.
[0084] A sensing result in the sensing region 104 is used,for example, to detect an object such as a surroundingvehicle.
[0085] A sensing region 105 illustrates an example of asensing region of the radar 52 for long distance. Thesensing region 105 covers a position farther than that ofthe sensing region 104 in front of the vehicle 1. Meanwhile,the sensing region 105 has a narrower range in the left-andrightdirection than that of the sensing region 104.
[0086] A sensing result in the sensing region 105 is usedfor, for example, adaptive cruise control (ACC), emergencybraking, collision avoidance, and the like.
[0087] It is noted that the sensing regions of therespective sensors of the camera 51, the radar 52, the LiDAR53, and the ultrasonic sensor 54 included in the externalrecognition sensor 25 may have various configurations otherthan those in FIG. 3. Specifically, the ultrasonic sensor54 may also sense the side of the vehicle 1, or the LiDAR 53may sense the rear of the vehicle 1. In addition, theinstallation position of each sensor is not limited to eachexample described above. The number of sensors may be oneor more.
[0088] [4. Configuration example of video display unit]FIG. 4 is a diagram illustrating a configuration exampleof the in-vehicle sensor 26 and the video display unit 93.FIG. 5 is a diagram illustrating a display example of a videoIM provided as the content CT.
[0089] In the examples of FIGS. 4 and 5, a video displayunit 93 is disposed so as to partition front row seats(driver's seat and passenger's seat) and rear row seats. Acamera CM for monitoring the passenger PA on the rear rowseat is installed in the vehicle. The camera CM functionsas the in-vehicle sensor 26. In the example of FIG. 4, thetwo cameras CM monitor the entire vehicle interior space(rear space) behind the front row seat.
[0090] The DMS 30 acquires viewing state information VSof the passenger PA in the rear space based on a videocaptured by the camera CM. The viewing state information VSincludes, for example, information regarding the number ofpassengers PA, whether the passenger PA is awake, the fieldof view of the passenger PA, and whether the passenger PAgazes the content CT. In a case where there are a pluralityof passengers PA, it is determined, for each passenger PA,the field of view of the passenger PA, whether the passengerPA is awake and whether the passenger PA gazes the contentCT. The HMI 31 controls display of the content CT based onthe viewing state information VS.
[0091] The video display unit 93 is an optical see-throughtype display or a video see-through type display capable oftransmitting and displaying a background. In the presentdisclosure, as an example, the optical see-through typedisplay is used as the video display unit 93. As the videodisplay unit 93, a known transparent display or technologysuch as a head-up display (HUD), a transparent organic lightemitting diode (OLED), and a transparent light emitting diode(LED) is used. In the present disclosure, as the videodisplay unit 93, for example, a configuration for projectinga video on a transparent screen SCR is adopted.
[0092] The video display unit 93 includes, for example,the transparent screen SCR and a projector PJ. Thetransparent screen SCR includes an optical lens film thatreflects light projected from the projector PJ. A lightcontrol film capable of electrically controlling thetransmittance of external light is laminated on the opticallens film. The HMI 31 adjusts display brightness ortransparency or haze of the video display unit 93 (thetransparent screen SCR) based on brightness informationoutside the vehicle extracted from environment informationCE. When the content CT is not displayed on the videodisplay unit 93, the passenger PA can see the scene outsidethe vehicle through the transparent screen SCR from a gapbetween the driver seat and the front passenger seat.
[0093] FIG. 6 is a diagram illustrating anotherconfiguration example of the video display unit 93. FIG. 7is a diagram illustrating a display example of the video IMprovided as the content CT.
[0094] In the examples of FIGS. 6 and 7, the video displayunit 93 is installed on the side of the passenger PA in therear row seat. Side glass (window WD: refer to FIG. 8) ofthe vehicle 1 is replaced by the transparent screen SCR.Instead of using the side glass as the transparent screenSCR, a film-like transparent screen may be attached to thesurface of the side glass or sandwiched inside the side glass.When the video IM is not displayed, the passenger PA can seethe scene outside the vehicle through the transparent screenSCR.
[0095] In the examples of FIGS. 6 and 7, the side glassof the rear seat has a function of the transparent screenSCR, but the glass having the function of the transparentscreen SCR is not limited to the side glass of the rear seat.The windshield, the side glass of the front seat, or therear glass may have the function of the transparent screenSCR.
[0096] FIG. 8 is a diagram illustrating anotherconfiguration example of the video display unit 93.
[0097] The video display unit 93 may adopt a configurationin which the content CT is projected onto a non-transparentscreen NTS such as a window frame WF. In the example of FIG.8, the window frame WF, a pillars PL, and an arm rest AM areused as the non-transparent screen NTS. When a desk or thelike is installed in the vehicle, the desk may be used asthe non-transparent screen NTS.
[0098] [5. Functional configuration related to videodisplay]FIG. 9 is a diagram illustrating an example of afunctional configuration related to the video display of thevehicle 1.
[0099] The vehicle control system 11 is an informationprocessing system that integrally controls the vehicle 1.The vehicle control system 11 includes a processing unit 90,the camera CM, a GPS receiver RV, the video display unit 93,and the storage unit 28.
[0100] The processing unit 90 is an informationprocessing apparatus that processes various types ofinformation. The processing unit 90 receives user inputinformation and provides a function of displaying a graphicto the passenger PA. The processing unit 90 has a controllayer of a general OS that performs gesture detection suchas multi-content drawing control and a touch operation. Theprocessing unit 90 determines a drawing method of the contentCT based on navigation information NV and a reception signalof the GPS receiver RV. The processing unit 90 determinesthe operation of a device or an application by the passengerPA based on sensing information of the camera CM.
[0101] The navigation information NV refers toinformation from a general car navigation system. Thenavigation information NV includes information on adestination and a movement path. The GPS receiver RV refersto a general sanitary positioning system. The GPS receiverRV receives a signal indicating the current position of thevehicle 1. The GPS receiver RV is included in the positioninformation acquisition unit 24.
[0102] The camera CM analyzes an image of an image sensorIS acquired via a filter FL and senses a human body. Thecamera CM is included in the in-vehicle sensor 26. Thereare two types of filters FL including an infraredtransmission filter and a visible light transmission filter.The processing unit 90 determines which filter FL is used toperform sensing based on the environment outside the vehicle.A control unit CU drives an actuator AC based on a determinedsensing method, and changes the filter FL. It is noted that,when an infrared image is captured, infrared rays are emittedfrom a light emitting element LE.
[0103] The processing unit 90 includes an environmentinformation processing unit 91, an optimization processingunit 92, a sensor information processing unit 94, an outputinformation generation unit 95, and the communication unit22.
[0104] The environment information processing unit 91acquires the navigation information NV from the travelassistance / automatic driving control unit 29. Theenvironment information processing unit 91 specifies thecurrent position of the vehicle 1 based on the receptionsignal of the GPS receiver RV. The environment informationprocessing unit 91 specifies a movement path from the currentlocation based on the navigation information NV, and acquiresa map around the movement path from the map informationaccumulation unit 23. The environment informationprocessing unit 91 accesses information on the Internet viathe communication unit 22 as necessary, and acquires weatherinformation for each traveling position. As a result, pathinformation (movement path, map around movement path,weather information for each traveling position) for thedestination is acquired. The environment informationprocessing unit 91 predicts a transition of the environmentoutside the vehicle during viewing of the content CT basedon the path information.
[0105] The optimization processing unit 92 determines thedrawing method of the content CT based on the predictedtransition of the environment outside the vehicle. Forexample, the environment information processing unit 91generates environment information indicating a predictionresult of the environment outside the vehicle for each time,and transmits the environment information to theoptimization processing unit 92. The environmentinformation includes, for example, a traveling position ofthe vehicle 1, weather of the traveling position, a travelingdirection, a peripheral map of the outside of the vehicle,and information regarding brightness (illuminance) of theenvironment outside the vehicle. The optimizationprocessing unit 92 determines a drawing condition (anappropriate drawing condition) corresponding to thepredicted environment outside the vehicle for each time.The optimization processing unit 92 determines a drawingmethod (an optimum drawing condition) based on a statisticalresult of the drawing condition for each time. Thus, thedrawing method is optimized. A drawing method determinationalgorithm is defined in a drawing rule accumulation unit DR.
[0106] The optimization processing unit 92 determines asensing method of the camera CM based on the predictedtransition of the environment outside the vehicle. Forexample, the optimization processing unit 92 determines, foreach time, a sensing method (an appropriate sensing method)according to the predicted environment outside the vehicle.The optimization processing unit 92 determines, for eachtime, a sensing method (an optimum sensing condition) basedon a statistical result of the appropriate sensing method.As a result, the sensing method is optimized.
[0107] For example, in a case where a bright environment(an environment with many infrared rays) is predicted,sensing of the passenger PA is performed based on a visiblelight image. In a case where a dark environment is predicted,sensing of the passenger PA is performed based on an infraredimage. A sensing method determination algorithm is definedin a sensing method accumulation unit SM.
[0108] The sensor information processing unit 94determines an operation of a device or an application thatis performed by the passenger PA based on the sensinginformation of the camera CM. The output informationgeneration unit 95 generates drawing information (thecontent CT) according to the operation of the passenger PAand outputs the drawing information to the video displayunit 93. The video display unit 93 draws the drawinginformation by the drawing method determined by theoptimization processing unit 92.
[0109] The optimization processing unit 92 can change,depending on the environment outside the vehicle, the drawingmethod and the sensing method optimized according to theprediction result. A sensing method change algorithm isdefined in a sensing change rule definition unit SCM. Adrawing method change algorithm is defined in a drawingchange rule definition unit DCR.
[0110] For example, in a case where a bright environmentis predicted, but the environment is actually dark, thesensing method of the passenger PA is changed from a methodbased on the visible light image to a method based on theinfrared image. In addition, in a case where drawing on thewindow WD is not appropriate in practice due to an influenceof a street lamp or the like although it is planned to drawon the window WD because a dark landscape is expected, adrawing position is changed to a position other than thewindow WD (for example, the arm rest AM, a desk installed inthe vehicle, or the like).
[0111] The storage unit 28 stores a productcharacteristic definition unit PC, the sensing methodaccumulation unit SM, the sensing change rule definitionunit SCM, the drawing rule accumulation unit DR, and thedrawing change rule definition unit DCR. The productcharacteristic definition unit PC defines a characteristic(a product characteristic) of a product on which the vehiclecontrol system 11 is mounted. Product characteristicincludes information on a mobility and a display type. The"mobility" indicates whether the product is a moving objectsuch as a vehicle or a fixed object such as a digital signage.The "display type" indicates whether the display istransparent.
[0112] [6. Control flow]FIG. 10 is a diagram illustrating an example of acontrol flow related to determination processing of thedrawing method.
[0113] When the display control of the content CT isstarted, the environment information processing unit 91acquires path information from a current position toward adestination (Step SA1). Based on the path information, theenvironment information processing unit 91 predicts a changein an environment outside a vehicle during viewing of thecontent CT (for example, a period from the current time toT seconds later) (Step SA2).
[0114] The optimization processing unit 92 optimizes adrawing method of the content CT based on a transition ofthe environment outside the vehicle during the viewing ofthe content CT (Step SA3). The output information generationunit 95 generates drawing information (the content CT)according to an operation of the passenger PA (Step SA4).The video display unit 93 draws the drawing information bythe drawing method determined by the optimization processingunit 92 (Step SA5). The processing unit 90 repeats theabove-described processing until end processing is performedby the operation of the passenger PA.
[0115] [7. Display control example]Hereinafter, a specific example of the display controlof the content CT will be described. The content CT to besubjected to the display control is a content capable ofchanging at least one of a drawing position and a drawingcolor. In the following example, an icon group that performsreproduction, stop, volume adjustment, setting change, andthe like of a moving image content such as a movie isexemplified as a target of the display control. Since acolor of a main video of the moving image content cannot bechanged, only the drawing position is a target of the displaycontrol.
[0116] [7-1. Display of content on transparent screen]FIG. 11 is a diagram illustrating a display example ofthe content CT on the transparent screen SCR.
[0117] Upon determining that the sun does not enter thefield of view of the passenger PA through the transparentscreen SCR based on environment information, theoptimization processing unit 92 determines, as the drawingposition of the content CT, the transparent screen SCR.
[0118] The optimization processing unit 92 detects alandscape LS outside the vehicle, in which the landscape LSis seen through the transparent screen SCR serving as adrawing target of the content CT, as a background BG of thecontent CT. The optimization processing unit 92 determinesa drawing color COL2 of the content CT based on a color COL1of the background BG. For example, the optimizationprocessing unit 92 determines a complementary color of thecolor COL1 of the background BG as the drawing color COL2.In the example of FIG. 11, the content CT is superimposed ontrees with green leaves. Therefore, purple, which is acomplementary color of green, is determined as the drawingcolor COL2 of the content CT.
[0119] [7-2. Superimposition avoidance with sun]FIG. 12 is a diagram illustrating a display example ina case where the sun SN is directly viewed through thetransparent screen SCR.
[0120] Upon determining that the sun SN enters the fieldof view of the passenger PA through the transparent screenSCR based on environment information, the optimizationprocessing unit 92 determines, as a drawing position of apart or all of the content CT, a position on the transparentscreen SCR that does not overlap the sun SN or the nontransparentscreen NTS. In the example of FIG. 12, thecontent CT is drawn at the position on the transparent screenSCR that does not overlap the sun SN, but the content CT maybe drawn on the non-transparent screen NTS such as the windowframe WF or the pillar PL.
[0121] [7-3. Display of content on non-transparentscreen]FIGS. 13 and 14 are diagrams illustrating a displayexample of the content CT on the non-transparent screen NTS.
[0122] In the example of FIG. 13, the rising sun or thesetting sun directly reaches the non-transparent screen NTS.Upon determining that the rising sun or the setting sundirectly reaches a content drawing position of the nontransparentscreen NTS based on environment information, theoptimization processing unit 92 determines an orange coloras the color COL1 of the background BG of the content CT.
[0123] The example of FIG. 14 is an example in which thecontent CT is drawn at a position on the non-transparentscreen NTS into which sunlight is not introduced. Theoptimization processing unit 92 detects the non-transparentscreen NTS, which is a drawing target of the content CT, asthe background BG of the content CT, and determines a drawingcolor based on the color of the background BG. For example,upon determining that sunlight does not directly reach thecontent drawing position of the non-transparent screen NTSbased on the environment information, the optimizationprocessing unit 92 determines, as the color COL1 of thebackground BG of the content CT, a color having luminancelower than luminance of a color of a material of the nontransparentscreen NTS at the content drawing position.
[0124] In the example of FIG. 14, the content CT is drawnin the pillar PL. When sunlight is not introduced, the colorof the non-transparent screen NTS serving as a projectiondestination of the content CT is recognized as a color darkerthan the original color of the material of the non30transparent screen NTS. Therefore, the optimizationprocessing unit 92 determines, as the background color COL1,the color having luminance lower than luminance of the colorof the material of the projection destination. When thecolor of the pillar PL is a gray color, the background colorCOL1 is a dark gray color. The optimization processing unit92 determines a complementary color of the dark gray color(for example, a whitish orange color) as the drawing colorCOL2 of the content CT.
[0125] [7-4. Optimization of drawing method based onprediction of transition of environment outside vehicle]A specification description will be given as tooptimization of a drawing method based on prediction of atransition of an environment outside a vehicle with referenceto FIGS. 15 to 20.
[0126] [7-4-1. Prediction period]FIG. 15 is an explanatory diagram of a prediction periodduring which the transition of the environment outside thevehicle is predicted.
[0127] A prediction period (T seconds) is defined foreach content CT. For example, the prediction period is setbased on a standard time at which the content CT is gazed.In the example of FIG. 15, the prediction period of anoperation-related content CT that does not require the gaze,such as an air conditioner and a music application, is setto 0. Regarding the content CT having a relatively shortbrowsing time, such as an SNS and an e-mail, the predictionperiod is set to 15 x 60 seconds. Regarding the content CThaving a relatively long browsing time such as Youtube, abrowser, and a game, the prediction period is set to 60 x 60seconds. Regarding the content CT, the reproduction time ofwhich is known in advance, such as a movie and a drama, theprediction period is set to the reproduction time of thecontent CT.
[0128] [7-4-2. Determination of drawing method]FIG. 16 is a flowchart illustrating determinationprocessing of a drawing method. FIG. 17 is a diagramillustrating a table used for the determination processingof the drawing method.
[0129] The processing unit 90 sets a variable i to 0 (StepSB1). The environment information processing unit 91 setsa prediction time Ti to Ti = Tc + Δt x i (Step SB2). Theenvironment information processing unit 91 acquires atraveling position at the prediction time Ti based on thenavigation information NV (Step SB3). The environmentinformation processing unit 91 acquires the weather at thetraveling position at the prediction time Ti based on weatherinformation or the like provided by the Meteorological Agency(Step SB4). The environment information processing unit 91acquires a peripheral map of the traveling position at theprediction time Ti from the map information accumulationunit 23 (Step SB5).
[0130] The optimization processing unit 92 detects alandscape LS outside the vehicle, the landscape LS servingas the background BG of the content CT, based on thetraveling position, the traveling direction, and theperipheral map of the outside of the vehicle 1 at theprediction time Ti. The optimization processing unit 92determines the drawing position and the drawing color COL2of the content CT at the prediction time Ti based on thebackground BG, and fills the table in FIG. 17 (Steps SB6 andSB7). The processing unit 90 increases a value of thevariable i by 1 (Step SB8) and determines whether theprediction period (T seconds) is smaller than Δt x i (StepSB9).
[0131] When the prediction period (T seconds) is smallerthan Δt x i (Step SB9: Yes), the optimization processingunit 92 performs statistical processing such as calculationof a mode value and calculation of an average value on theplurality of drawing conditions calculated for each time.The optimization processing unit 92 determines the drawingmethod obtained by the statistical processing of theplurality of drawing conditions as an optimum drawing method(Step SB10). When the prediction period (T seconds) is Δtx i or longer (Step SB9: No), the processing returns to StepSB2, and the processing from Step SB2 to Step SB9 isrepeatedly performed.
[0132] [7-4-3. Specific example of drawing position anddrawing color]FIG. 18 is a flow illustrating a specific example ofthe determination method of the drawing position for eachtime.
[0133] The optimization processing unit 92 determineswhether there is the sun in the field of view of the passengerPA based on environment information (Step SC1). If there isno sun in the field of view of the passenger PA (Step SC1:No), the optimization processing unit 92 sets a drawingposition in the window WD (Step SC2).
[0134] When the sun is in the field of view of thepassenger PA (Step SC1: Yes), the optimization processingunit 92 determines whether the displayed content CT is acontent having a wide drawing range, such as a moving imageor a browser (Step SC3). When the content CT having a widedrawing range is displayed (Step SC3: Yes), the optimizationprocessing unit 92 sets the drawing position in the windowframe WF (Step SC4).
[0135] When the content CT having a narrow drawing rangeis displayed (Step SC3: No), the optimization processingunit 92 determines whether the content CT is an operationrelatedapplication such as an air conditioner or music (StepSC5). When the operation-related application is displayedas the content CT (Step SC5: Yes), the optimizationprocessing unit 92 sets the drawing position to the arm restAM (Step SC6). When the content CT is not the operationrelatedapplication (Step SC5: No), the optimizationprocessing unit 92 sets the pillar PL as the drawing position(Step SC7).
[0136] FIG. 19 is a flow illustrating a specific exampleof the determination method of the drawing color COL2 foreach time.
[0137] The optimization processing unit 92 determineswhether a drawing position is the window WD (the transparentscreen SCR) (Step SD1). When the drawing position is thewindow WD (Step SD1: Yes), the optimization processing unit92 determines the background color COL1 from the travelingposition of the vehicle 1 (Step SD2).
[0138] When the drawing position is the non-transparentscreen NTS (Step SD1: No), the optimization processing unit92 determines whether sunlight is introduced into the drawingposition of the content CT (Step SD3). When sunlight is notintroduced to a content drawing position (Step SD3: No), theoptimization processing unit 92 determines, as thebackground color COL1, a color having luminance lower thanluminance of a color of a material of the non-transparentscreen NTS at the content drawing position (Step SD4).
[0139] When sunlight is introduced into the contentdrawing position (Step SD3: Yes), the optimizationprocessing unit 92 determines whether the altitude of thesun SN is low (sunrise or sunset) (Step SD5). When thealtitude of the sun SN is high (Step SD5: No), theoptimization processing unit 92 determines, as thebackground color COL1, a color brighter than the color ofthe material of the non-transparent screen NTS (Step SD6).That is, upon determining that daylight directly reaches thecontent drawing position of the non-transparent screen NTSbased on the environment information, the optimizationprocessing unit 92 determines, as the color COL1 of thebackground BG, a color having luminance higher than luminanceof the color of the material of the non-transparent screenNTS at the content drawing position. When the altitude ofthe sun SN is low (Step SD5: Yes), the optimizationprocessing unit 92 determines, as the background color COL1,an orange color (Step SD7).
[0140] When the background color COL1 is determined, theoptimization processing unit 92 determines a complementarycolor of the background color COL1 as the drawing color COL2(Step SD8). For example, when the traveling position is ina forest or a mountain, the background color COL1 isdetermined as a green color, and the drawing color COL2 isdetermined as a purple color. When the traveling positionis a seashore or an open place (a place at which the sky isvisible), the background color COL1 is determined as a bluecolor, and the drawing color COL2 is determined as a yellowcolor. When the traveling position is along an expresswayor in town, the background color COL1 is determined as alight gray color, and the drawing color COL2 is determinedas a dark gray color. When the traveling position is in aresidential area, the background color COL1 is determined asa white color, and the drawing color COL2 is determined asa black color.
[0141] FIG. 20 is a flow illustrating a specific exampleof optimization by statistical processing.
[0142] The optimization processing unit 92 aggregates aplurality of drawing positions calculated for eachprediction time Ti and calculates an expression frequencyfor each drawing position (Step SE1). The optimizationprocessing unit 92 determines a drawing position having thehighest expression frequency as an optimum drawing position(Step SE2).
[0143] The optimization processing unit 92 extracts oneor more prediction times Ti at which the drawing position isthe optimum drawing position (Step SE3). The optimizationprocessing unit 92 aggregates the drawing colors at theextracted one or more prediction times Ti and calculates theexpression frequency of each drawing color (Step SE4). Theoptimization processing unit 92 determines the drawing colorhaving the highest expression frequency as the optimumdrawing color (Step SE5). The optimization processing unit92 determines, as the drawing method of the content CT, adrawing method having the optimum drawing position and theoptimum drawing color as an optimum drawing condition.
[0144] [7-5. Change of drawing method]FIG. 21 is a diagram illustrating change processing ofthe drawing method.
[0145] The optimization processing unit 92 cantemporarily change a drawing condition of the content CTaccording to a situation inside and outside the vehicle.The situation inside and outside the vehicle is monitored bythe monitor unit MU. The monitor unit MU includes, forexample, the external recognition sensor 25 and the invehiclesensor 26. The environment information processingunit 91 detects a surrounding situation of the vehicle 1 anda viewing environment in the vehicle based on a monitoringresult of the monitor unit MU. The optimization processingunit 92 temporarily changes the drawing method performedunder the optimum drawing condition based on the surroundingsituation of the vehicle 1 and the viewing environment inthe vehicle. Specific examples will be described below.
[0146] [7-5-1. Countermeasure against visually inducedmotion sickness]FIGS. 22 and 23 are diagrams illustrating examples ofchanging the drawing method according to the surroundingsituation of the vehicle 1.
[0147] When the content CT is superimposed on thetransparent screen SCR, visibility of the content CT ischanged depending on a situation around the vehicle 1, inwhich the situation serves as the background BG. For example,when a crowd CW exists in the background BG of the contentCT, the passenger PA simultaneously observes the movement ofthe crowd passing through the transparent screen SCR whilegazing at the content CT. Therefore, autonomic nerves maybe disordered, and symptoms similar to carsickness (visuallyinduced motion sickness) may appear.
[0148] Therefore, when the crowd CW is detected at aposition serving as the background BG while drawing is beingperformed at the optimum drawing position, the drawingposition is changed to a position avoiding the crowd CW.For example, upon determining that the crowd CW outside thevehicle enters the field of view of the passenger PA via thespecific transparent screen SCR, the optimization processingunit 92 determines, as a drawing position of a part or allof the content CT, the transparent screen SCR other than thespecific transparent screen SCR or the non-transparentscreen NTS.
[0149] In the example of FIG. 22, the crowd CW exists onthe left side of the vehicle 1. Therefore, the drawingposition is changed to the window WD on the right side ofthe vehicle 1 at a position where the crowd CW is not seenthrough the window. In the example of FIG. 23, the crowd CWexists on both sides of the vehicle 1. Therefore, thedrawing position is changed to the transparent screen SCR atthe center of the vehicle 1 illustrated in FIG. 4 or thenon-transparent screen NTS such as the arm rest AM.
[0150] FIG. 24 is a diagram illustrating an example of aflow of the countermeasure against the visually inducedmotion sickness.
[0151] The monitor unit MU monitors a surroundingsituation of the vehicle 1 using the camera 51 facing theoutside of the vehicle (Step SF1). The environmentinformation processing unit 91 detects one or more personsat a position serving as the background BG of the content CTbased on a monitoring result, and supplies information onthe number of persons and the positions of the persons tothe optimization processing unit 92 (Step SF2).
[0152] The optimization processing unit 92 determineswhether the number of people existing in the background BGis equal to or greater than a threshold value (Step SF3).When the number of people is less than the threshold value(Step SF3: No), the optimization processing unit 92determines the window WD as the drawing position (Step SF4).
[0153] When the number of people is equal to or greaterthan the threshold value (Step SF3: Yes), the optimizationprocessing unit 92 determines whether there are people onboth sides (right side and left side) of the vehicle 1 (StepSF5). When there is a person only on one side of the vehicle1 (Step SF5: No), the optimization processing unit 92determines the window WD on the side where there is no personas the drawing position (Step SF6). When there are personson both sides of the vehicle 1 (Step SF5: Yes), theoptimization processing unit 92 determines the arm rest AMor a desk installed in the vehicle as the drawing position(Step SF7).
[0154] [7-5-2. Countermeasure against shape change ofprojection target]FIG. 25 is a diagram illustrating an example of changingthe drawing method according to the viewing environment inthe vehicle.
[0155] When the content CT is projected by the projectorPJ, the maximum drawing size of the content CT is changeddepending on a change in the shape of a projection target.For example, in a state where the window WD is half-opened,only the content CT having a half size as compared with astate in which the window WD is closed can be displayed onthe window WD.
[0156] Therefore, when the shape of the projection targetis changed while drawing is being performed at the optimumdrawing position, the drawing position is changed dependingon the amount of change in the shape of the projection target.For example, the optimization processing unit 92 determines,as the drawing position of a part or all of the content CTto be drawn on the transparent screen SCR, the non15transparent screen NTS based on the opening / closing amountof the window WD used as the transparent screen SCR.
[0157] In the example of FIG. 25, a cross media bar CMBis displayed as the content CT. The cross media bar CMB isa user interface (UI) that displays the icon group in whichicons are arranged in the horizontal direction and thevertical direction separately for each function.
[0158] When the window WD is completely closed, a firstdrawing method of drawing the content CT on the window WD isadopted. When the window WD is not completely closed, asecond drawing method of drawing the content CT on the windowWD in the closed portion and / or the non-transparent screenNTS is adopted. When the window WD is fully open, a thirddrawing method of drawing the content CT on the nontransparentscreen NTS is adopted.
[0159] In the example of FIG. 25, the content CT includesan icon group in which icons are arranged in the horizontaldirection and the vertical direction. When the window WD iscompletely closed, all the icons of the icon group arrangedin the horizontal direction and the vertical direction aredrawn on the window WD. When the window WD is not completelyclosed, the icons of the icon group arranged in thehorizontal direction are drawn in the horizontally longwindow WD in the half-open state, and the icons of the icongroup arranged in the vertical direction are drawn in thevertically long pillar PL. When the window WD is fully open,the icons of the icon group arranged in the horizontaldirection are drawn in the window frame WF in the horizontaldirection, and the icons of the icon group arranged in thevertical direction are drawn in the vertically long pillarPL.
[0160] FIG. 26 is a diagram illustrating an example of aflow of a countermeasure against the opening and closing ofthe window WD.
[0161] The monitor unit MU monitors the open / close stateof the window WD using the camera CM directed toward theinside of the vehicle. The environment informationprocessing unit 91 acquires information on a degree ofopening of the window WD based on a monitoring result andsupplies the acquired information to the optimizationprocessing unit 92 (Step SG1).
[0162] The optimization processing unit 92 determineswhether the window WD is completely closed (Step SG2). Whenthe window WD is completely closed (Step SG2: Yes), theoptimization processing unit 92 adopts the first drawingmethod (Step SG3).
[0163] When the window WD is open (Step SG2: No), theoptimization processing unit 92 determines whether thewindow WD is fully open (Step SG4). When the window WD isnot fully opened (Step SG4: No), the optimization processingunit 92 adopts the second drawing method (Step SG5). Whenthe window WD is fully open (Step SG4: Yes), the optimizationprocessing unit 92 adopts the third drawing method (StepSG6).
[0164] [7-6. Determination of drawing method withoutprediction]FIG. 27 is a diagram illustrating determinationprocessing of a drawing method without prediction of atransition of the environment outside the vehicle.
[0165] The optimization processing unit 92 can alsodetermine the drawing method at the current position basedon the environment outside the vehicle at the currentposition without predicting the transition of theenvironment outside the vehicle. When the prediction doesnot need to be performed, the landscape LS outside thevehicle or a color of a material of the non-transparentscreen NTS is monitored in real time by a camera (the cameraCM, the camera 51) installed inside or outside the vehicle,whereby the background color COL1 can be accurately detectedand the drawing color COL2 having high distinguishabilitycan be determined.
[0166] FIG. 28 is a flowchart illustrating thedetermination processing of the drawing method withoutprediction of the transition of the environment outside thevehicle.
[0167] The optimization processing unit 92 determineswhether a prediction period is 0 (Step SH1). When theprediction period is not 0 (Step SH1: No), the optimizationprocessing unit 92 predicts a transition of an environmentoutside a vehicle based on path information, and determinesa drawing method based on a prediction result (Step SH8).When the prediction period is 0 (Step SH1: Yes), theoptimization processing unit 92 determines whether a drawingposition is the window WD (Step SH2).
[0168] When the drawing position is not the window WD(Step SH2: No), the optimization processing unit 92 acquiresa captured image of the non-transparent screen NTS to beprojected from the camera CM (Step SH3). The optimizationprocessing unit 92 detects a color of a material of the nontransparentscreen NTS from the captured image of the cameraCM. The optimization processing unit 92 sets the detectedcolor of the material as the background color COL1 (StepSH4), and sets a complementary color of the background colorCOL1 as a drawing color (Step SH7).
[0169] When the drawing position is the window WD (StepSH2: Yes), the optimization processing unit 92 acquires acaptured image of the landscape LS outside the window WDfrom the camera 51 (Step SH5). The optimization processingunit 92 detects a color of the landscape LS of a portionsuperimposed on the content CT from the captured image ofthe camera 51. The optimization processing unit 92 sets thedetected color of the landscape LS as the background colorCOL1 (Step SH6), and sets the complementary color of thebackground color COL1 as the drawing color (Step SH7).
[0170] [7-7. Determination of drawing color inconsideration of content]FIG. 29 is a diagram illustrating the determinationmethod of the drawing color COL2 in consideration of thecontent CT.
[0171] There is the content CT such as a movie in whichthe intention of a content creator may be changed when thedrawing color COL2 is changed. Regarding the content CT,distinguishability from the background BG can be enhanced bydrawing a frame FLM without changing the drawing color.
[0172] FIG. 29 illustrates a moving image content such asa movie as an example of the above-described content CT.The moving image content includes a main video CT-A and anicon group CT-B. The icon group CT-B is used to performreproduction, stop, volume adjustment, setting change, andthe like of the main video CT-A.
[0173] A color of the main video CT-A cannot be changed.Therefore, the optimization processing unit 92 superimposesthe frame FLM surrounding the main video CT-A on thebackground BG and then displays the frame FLM. A color ofthe frame FLM may be any color as long as the color has highdistinguishability from the background BG. The frame FLMmay be displayed in a single color or in multiple colors.The shape of the frame FLM is also set according to the shapeof an object to be surrounded, and any shape such as a circleor a square is possible. In the example of FIG. 29, themonochromatic rectangular frame FLM that rims the main videoCT-A is displayed.
[0174] A color of the icon group CT-B can be changed.Therefore, the optimization processing unit 92 enhancesdistinguishability from the background BG by setting theappropriate drawing color COL2 depending on the backgroundcolor COL1. In the example of FIG. 29, the icon group CT-Bis superimposed on a tree (green leaf). Therefore, theoptimization processing unit 92 recognizes the backgroundcolor COL1 as a green color and sets the drawing color COL2as a pink color (a complementary color of the green color).
[0175] [7-8. Response to light source other than sun]FIG. 30 is a diagram illustrating response to a lightsource other than the sun SN.
[0176] Similar optimization can be performed even in thepresence of a light source (such as a tunnel light or astreet lamp) other than the sun SN. That is, upondetermining that the light source enters the field of viewof the passenger PA through the transparent screen SCR, theoptimization processing unit 92 can determine, as a drawingposition of a part or all of the content CT, a position onthe transparent screen SCR, in which the position does notoverlap the light source, or the non-transparent screen NTS.Accordingly, visibility of the content CT is enhanced.
[0177] However, when the light source frequently moveswithin the field of view of the passenger PA, the drawingposition of the content CT is also frequently changed inaccordance with the movement of the light source. Therefore,there is a possibility that usability deteriorates orvisually induced motion sickness occurs. Therefore, theoptimization processing unit 92 varies the optimizationmethod of the drawing method depending on a movementsituation of the light source. For example, when the lightsource is stopped in the field of view of the passenger PA,the optimization processing unit 92 optimizes both thedrawing position and the drawing color by the above-describedmethod. When the light source is moving within the field ofview of the passenger PA, the optimization processing unit92 fixes the drawing position and optimizes the drawing color.Whether the light source is moving is determined based on,for example, a moving situation of a shadow inside thevehicle caused by the light source.
[0178] FIG. 31 is a flowchart illustrating determinationprocessing of the drawing method based on the movementsituation of the light source.
[0179] First, the shadow inside the vehicle generated bythe light source is photographed for t seconds by the cameraCM (Step SI1). A photographing time (t seconds) is freelyand selectively set as a time at which the movement situationof the shadow can be determined. The environment informationprocessing unit 91 calculates a moving speed of the shadowbased on a video of the shadow (a visible light video)acquired from the camera CM and supplies the moving speed tothe optimization processing unit 92.
[0180] The optimization processing unit 92 determineswhether the position of the shadow is changing based on themoving speed of the shadow (Step SI2). For example, whenthe moving speed of the shadow is equal to or greater thana threshold value, it is determined that the position of theshadow is changing. When the moving speed of the shadow isless than the threshold value, it is determined that theposition of the shadow is not changing.
[0181] When the position of the shadow is not changing(Step SI2: No), the optimization processing unit 92determines that the light source is stopped, and optimizesonly the drawing color of the content CT by the abovedescribedmethod (Step SI3). When the position of the shadowis changing (Step SI2: Yes), the optimization processingunit 92 determines that the light source is moving, andoptimizes both the drawing position and the drawing color ofthe content CT by the above-described method (Step SI4).
[0182] [8. Effects]The processing unit 90 includes the environmentinformation processing unit 91 and the optimizationprocessing unit 92. The environment information processingunit 91 predicts the transition of the environment outsidethe vehicle while the content CT is being viewed. Theoptimization processing unit 92 determines the drawingmethod of the content CT based on the predicted transitionof the environment outside the vehicle. In the informationprocessing method of the present disclosure, the processingof the processing unit 90 is executed by a computer. Theprogram of the present disclosure causes the computer toimplement processing of the processing unit 90. The vehiclecontrol system 11 of the present disclosure includes thevideo display unit 93 that draws the content CT based on thedrawing method determined by a processing unit 90.
[0183] According to this configuration, an appropriatedrawing method in consideration of a change in theenvironment outside the vehicle is determined. Therefore,the visibility of the content CT is less likely todeteriorate due to a change in the environment outside thevehicle.
[0184] The environment information processing unit 91generates the environment information indicating aprediction result of the environment outside the vehicle foreach time. The optimization processing unit 92 determinesthe drawing condition according to the predicted environmentoutside the vehicle for each time, and determines the drawingmethod based on a statistical result of the drawing conditionfor each time.
[0185] According to this configuration, the drawingmethod is less likely to be frequently changed due to thechange in the environment outside the vehicle.
[0186] The environment information includes informationon a traveling position, weather at the traveling position,a traveling direction, and a peripheral map outside thevehicle.
[0187] According to this configuration, the drawingmethod is optimized based on the traveling landscape and theweather transition.
[0188] The drawing condition includes the drawingposition and the drawing color COL2.
[0189] According to this configuration, the drawingposition and the drawing color COL2 are prevented from beingfrequently changed due to the change in the environmentoutside the vehicle.
[0190] The optimization processing unit 92 aggregates aplurality of drawing positions calculated for each time andcalculates the expression frequency for each drawingposition. The optimization processing unit 92 determinesthe drawing position having the highest expression frequencyas the optimum drawing position. The optimizationprocessing unit 92 extracts one or more times at which thedrawing position is the optimum drawing position. Theoptimization processing unit 92 aggregates the drawingcolors COL2 at the extracted one or more times and calculatesthe expression frequency of each drawing color COL2. Theoptimization processing unit 92 determines the drawing colorCOL2 having the highest expression frequency as the optimumdrawing color. The optimization processing unit 92determines, as the drawing method of the content CT, adrawing method having the optimum drawing position and theoptimum drawing color as an optimum drawing condition.
[0191] According to this configuration, the optimumdrawing method is determined according to the drawingposition and the expression frequency of the drawing colorCOL2 calculated for each time.
[0192] The optimization processing unit 92 detects, asthe background BG of the content CT, the non-transparentscreen NTS, which is a drawing target of the content CT, orthe landscape LS outside the vehicle seen through thetransparent screen SCR, which is a drawing target of thecontent CT. The optimization processing unit 92 determinesthe drawing color COL2 based on the color of the backgroundBG.
[0193] According to this configuration, the content CT isbetter identified from the background BG.
[0194] Upon determining that sunlight does not directlyreach the content drawing position of the non-transparentscreen NTS based on the environment information, theoptimization processing unit 92 determines, as the color ofthe background BG, a color having luminance lower thanluminance of a color of a material of the non-transparentscreen NTS at the content drawing position.
[0195] According to this configuration, the appropriatedrawing color COL2 is determined in consideration of thecolor of the material of the non-transparent screen NTSserving as a drawing target.
[0196] Upon determining that daylight directly reachesthe content drawing position of the non-transparent screenNTS based on the environment information, the optimizationprocessing unit 92 determines, as the color of the backgroundBG, a color having luminance higher than luminance of thecolor of the material of the non-transparent screen NTS atthe content drawing position.
[0197] According to this configuration, the appropriatedrawing color COL2 is determined in consideration of thebrightness of direct sunlight (daylight) at the contentdrawing position.
[0198] Upon determining that the rising sun or the settingsun directly reaches the content drawing position of thenon-transparent screen NTS based on the environmentinformation, the optimization processing unit 92 determinesan orange color as the color of the background BG.
[0199] According to this configuration, the appropriatedrawing color COL2 is determined in consideration of thecolor (the orange color) of the rising sun or the settingsun.
[0200] The optimization processing unit 92 detects thelandscape LS outside the vehicle to be the background BG ofthe content CT based on the traveling position, the travelingdirection, and the peripheral map outside the vehicle.
[0201] According to this configuration, the landscape LSoutside the vehicle is better estimated.
[0202] The optimization processing unit 92 determines acomplementary color of the color of the background BG as thedrawing color COL2.
[0203] According to this configuration, the content CT isbetter identified from the background BG.
[0204] Upon determining that the sun SN does not enterthe field of view of the passenger PA through the transparentscreen SCR based on the environment information, theoptimization processing unit 92 determines the transparentscreen SCR as the drawing position of the content CT.
[0205] According to this configuration, deterioration invisibility due to direct view of the sun SN is suppressed.
[0206] Upon determining that the sun SN enters the fieldof view of the passenger PA through the transparent screenSCR based on environment information, the optimizationprocessing unit 92 determines, as a drawing position of apart or all of the content CT, a position on the transparentscreen SCR that does not overlap the sun SN or the nontransparentscreen NTS.
[0207] According to this configuration, the content CT ispresented at a position where the sun SN is not directlyviewed.
[0208] For example, upon determining that the crowd CWoutside the vehicle enters the field of view of the passengerPA through the specific transparent screen SCR, theoptimization processing unit 92 determines, as a drawingposition of a part or all of the content CT, the transparentscreen SCR other than the specific transparent screen SCR orthe non-transparent screen NTS.
[0209] According to this configuration, deterioration invisibility due to the content CT being superimposed on thecrowd CW is suppressed.
[0210] The optimization processing unit 92 determines, asa drawing position of a part or all of the content CT to bedrawn on the transparent screen SCR, the non-transparentscreen NTS based on the opening / closing amount of the windowWD used as the transparent screen SCR.
[0211] According to this configuration, the visibility ofthe content CT hardly deteriorates due to the opening andclosing of the window WD.
[0212] A prediction period during which the transition ofthe environment outside the vehicle is predicted is definedfor each content CT.
[0213] According to this configuration, an appropriateprediction period according to the content CT is set.
[0214] The prediction period is set based on a standardtime at which the content CT is gazed.
[0215] According to this configuration, the drawingmethod is hardly changed during the viewing of the contentCT.
[0216] It is noted that the effects described in thepresent specification are merely examples and are not limited,and other effects may be obtained.
[0217] [Supplementary note]It is noted that the present technology can also havethe following configurations.(1)An information processing apparatus comprising:an environment information processing unit configuredto predict a transition of an environment outside a vehicleduring viewing of a content; andan optimization processing unit configured to determinea drawing method of the content based on the predictedtransition of the environment outside the vehicle.(2)The information processing apparatus according to (1),wherein:the environment information processing unit isconfigured to generate environment information indicating aprediction result of the environment outside the vehicle foreach time; andthe optimization processing unit is configured todetermine, for each time, a drawing condition according tothe predicted environment outside the vehicle, and todetermine the drawing method based on a statistical resultof the drawing condition for each time.(3)The information processing apparatus according to (2),wherein the environment information includes information ona traveling position, weather at the traveling position, atraveling direction, and a peripheral map outside the vehicle.(4)The information processing apparatus according to (2)or (3), wherein the drawing condition includes a drawingposition and a drawing color.(5)The information processing apparatus according to (4),wherein the optimization processing unit is configured to:aggregate a plurality of the drawing positions, each ofthe drawing positions being calculated for each time, andcalculate an expression frequency of each of the drawingpositions;determine, as an optimum drawing position, the drawingposition having the highest expression frequency;extract one or more times at which the drawing positionis the optimum drawing position;aggregate the drawing colors at the extracted one ormore times and calculate the expression frequency of each ofthe drawing colors;determine, as an optimum drawing color, the drawingcolor having the highest expression frequency; anddetermine, as the drawing method of the content, thedrawing method having the optimum drawing position and theoptimum drawing color as optimum drawing conditions.(6)The information processing apparatus according to (4)or (5), wherein the optimization processing unit isconfigured to detect, as a background of the content, a nontransparentscreen serving as a drawing target of the contentor a landscape outside the vehicle, wherein the landscape isseen through a transparent screen serving as a drawing targetof the content, and to determine the drawing color based ona color of the background.(7)The information processing apparatus according to (6),wherein the optimization processing unit is configured todetermine, upon determining that sunlight does not directlyreach a content drawing position of the non-transparentscreen based on the environment information, a color havingluminance lower than luminance of a color of a material ofthe non-transparent screen at the content drawing positionas the color of the background.(8)The information processing apparatus according to (6)or (7), wherein the optimization processing unit isconfigured to determine, upon determining that daylightdirectly reaches a content drawing position of the nontransparentscreen based on the environment information, acolor having luminance higher than luminance of a color ofa material of the non-transparent screen at the contentdrawing position as the color of the background.(9)The information processing apparatus according to anyone of (6) to (8), wherein the optimization processing unitis configured to determine, upon determining that rising sunor setting sun directly reaches a content drawing positionof the non-transparent screen based on the environmentinformation, an orange color as the color of the background.(10)The information processing apparatus according to anyone of (6) to (9), wherein the optimization processing unitis configured to detect, based on a traveling position, atraveling direction, and a peripheral map outside the vehicle,the landscape outside the vehicle, wherein the landscapeserves as the background of the content.(11)The information processing apparatus according to anyone of (6) to (10), wherein the optimization processing unitis configured to determine, as the drawing color, acomplementary color of the color of the background.(12)The information processing apparatus according to anyone of (6) to (11), wherein the optimization processing unitis configured to determine, upon determining that sun doesnot enter a field of view of a passenger through thetransparent screen based on the environment information, thetransparent screen as the drawing position of the content.(13)The information processing apparatus according to anyone of (6) to (12), wherein the optimization processing unitis configured to determine, upon determining that sun entersa field of view of a passenger through the transparent screenbased on the environment information, a position on thetransparent screen, the position not overlapping the sun, orthe non-transparent screen as the drawing position of a partor all of the content.(14)The information processing apparatus according to anyone of (6) to (13), wherein the optimization processing unitis configured to determine, upon determining that a crowdoutside the vehicle enters a field of view of a passengerthrough a specific transparent screen, the transparentscreen other than the specific transparent screen or thenon-transparent screen as the drawing position of a part orall of the content.(15)The information processing apparatus according to anyone of (6) to (14), wherein the optimization processing unitis configured to determine, based on an opening / closingamount of a window used as the transparent screen, the non15transparent screen as the drawing position of a part or allof the content to be drawn on the transparent screen.(16)The information processing apparatus according to anyone of (1) to (15), wherein a prediction period during whichthe transition of the environment outside the vehicle ispredicted is defined for each of the contents.(17)The information processing apparatus according to (16),wherein the prediction period is set based on a standardtime at which the content is gazed.(18)An information processing method executed by a computer,the information processing method comprising:predicting a transition of an environment outside avehicle during viewing of a content; anddetermining a drawing method of the content based onthe predicted transition of the environment outside thevehicle.(19)A program configured to cause a computer to implement:predicting a transition of an environment outside avehicle during viewing of a content; anddetermining a drawing method of the content based onthe predicted transition of the environment outside thevehicle.(20)An information processing system comprising:the information processing apparatus according to anyone of (1) to (17); anda video display unit configured to draw the contentbased on the drawing method determined by the informationprocessing apparatus.Reference Signs List
[0218] 11 VEHICLE CONTROL SYSTEM (INFORMATION PROCESSINGSYSTEM)90 PROCESSING UNIT (INFORMATION PROCESSING APPARATUS)91 ENVIRONMENT INFORMATION PROCESSING UNIT92 OPTIMIZATION PROCESSING UNIT93 VIDEO DISPLAY UNITBG BACKGROUNDCOL1 BACKGROUND COLORCOL2 DRAWING COLORCT CONTENTLS LANDSCAPENTS NON-TRANSPARENT SCREENPA PASSENGERSCR TRANSPARENT SCREENSN SUNWD WINDOW
Claims
1. An information processing apparatus comprising: an environment information processing unit configured to predict a transition of an environment outside a vehicle during viewing of a content; and an optimization processing unit configured to determine a drawing method of the content based on the predicted transition of the environment outside the vehicle.
2. The information processing apparatus according to claim 1, wherein: the environment information processing unit is configured to generate environment information indicating a prediction result of the environment outside the vehicle for each time; and the optimization processing unit is configured to determine, for each time, a drawing condition according to the predicted environment outside the vehicle, and to determine the drawing method based on a statistical result of the drawing condition for each time.
3. The information processing apparatus according to claim 2, wherein the environment information includes information on a traveling position, weather at the traveling position, a traveling direction, and a peripheral map outside the vehicle.
4. The information processing apparatus according to claim 2, wherein the drawing condition includes a drawing position and a drawing color.
5. The information processing apparatus according to claim 4, wherein the optimization processing unit is configured to: aggregate a plurality of the drawing positions, each of the drawing positions being calculated for each time, and calculate an expression frequency of each of the drawing positions; determine, as an optimum drawing position, the drawing position having the highest expression frequency; extract one or more times at which the drawing position is the optimum drawing position; aggregate the drawing colors at the extracted one or more times and calculate the expression frequency of each of the drawing colors; determine, as an optimum drawing color, the drawing color having the highest expression frequency; and determine, as the drawing method of the content, the drawing method having the optimum drawing position and the optimum drawing color as optimum drawing conditions.
6. The information processing apparatus according to claim 4, wherein the optimization processing unit is configured to detect, as a background of the content, a nontransparent screen serving as a drawing target of the content or a landscape outside the vehicle, wherein the landscape is seen through a transparent screen serving as a drawing target of the content, and to determine the drawing color based on a color of the background.
7. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, upon determining that sunlight does not directly reach a content drawing position of the nontransparent screen based on the environment information, a color having luminance lower than luminance of a color of a material of the non-transparent screen at the content drawing position as the color of the background.
8. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, upon determining that daylight directly reaches a content drawing position of the nontransparent screen based on the environment information, a color having luminance higher than luminance of a color of a material of the non-transparent screen at the content drawing position as the color of the background.
9. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, upon determining that rising sun or setting sun directly reaches a content drawing position of the non-transparent screen based on the environment information, an orange color as the color of the background.
10. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to detect, based on a traveling position, a traveling direction, and a peripheral map outside the vehicle, the landscape outside the vehicle, wherein the landscape serves as the background of the content.
11. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, as the drawing color, a complementary color of the color of the background.
12. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, upon determining that sun does not enter a field of view of a passenger through the transparent screen based on the environment information, the transparent screen as the drawing position of the content.
13. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, upon determining that sun enters a field of view of a passenger through the transparent screen based on the environment information, a position on the transparent screen, the position not overlapping the sun, or the non-transparent screen as the drawing position of a part or all of the content.
14. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, upon determining that a crowd outside the vehicle enters a field of view of a passenger through a specific transparent screen, the transparent screen other than the specific transparent screen or the non-transparent screen as the drawing position of a part or all of the content.
15. The information processing apparatus according to claim 6, wherein the optimization processing unit is configured to determine, based on an opening / closing amount of a window used as the transparent screen, the nontransparent screen as the drawing position of a part or all of the content to be drawn on the transparent screen.
16. The information processing apparatus according to claim 1, wherein a prediction period during which the transition of the environment outside the vehicle is predicted is defined for each of the contents.
17. The information processing apparatus according to claim 16, wherein the prediction period is set based on a standard time at which the content is gazed.
18. An information processing method executed by a computer, the information processing method comprising: predicting a transition of an environment outside a vehicle during viewing of a content; and determining a drawing method of the content based on the predicted transition of the environment outside the vehicle.
19. A program configured to cause a computer to implement: predicting a transition of an environment outside a vehicle during viewing of a content; and determining a drawing method of the content based on the predicted transition of the environment outside the vehicle.
20. An information processing system comprising: the information processing apparatus according to claim 1; and a video display unit configured to draw the content based on the drawing method determined by the information processing apparatus.
