Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, information processing method, and program, which can carry out a presentation of information with high usability. On the basis of presentation location information relating to a state change of a presentation location at which presentation information is presented, and at least any one of process information relating to a process for generating the presentation information, user information relating to a user response to the process, and / or environment information relating to an environment state of the presentation location, a control is carried out of a presentation state of the presentation information and / or a state of the process. The present technology may be applied as a projection device for a vehicle interior, a conference room, or the like.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that present information with high usability. [Background technology]

[0002] Patent Document 1 discloses a technique for changing the size of a projected image (window size) in accordance with the size of a projection surface on a table. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 069575 Summary of the Invention [Problem to be solved by the invention]

[0004] When the state of the presentation part where the presentation information is presented, such as the position, changes, there are various situations. When the state of the presentation part changes, it is desirable to take measures taking the situation into consideration.

[0005] The present technology has been developed in light of such circumstances, and makes it possible to present information with high usability. [Means for solving the problem]

[0006] According to one aspect of the present technology, an information processing device or a program includes a control unit that controls at least one of a presentation state of the presentation information and a state of the processing based on at least one of presentation part information on a state change of a presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part. When the presentation part information indicates that a state change has occurred in the presentation part while the presentation information is being presented at least on the presentation part, the control unit changes the presentation part on which the presentation information is presented.An information processing device, or a program for causing a computer to function as such an information processing device.

[0007] In an information processing method according to one aspect of the present technology, a control unit of an information processing device having a control unit controls at least one of a presentation state of the presentation information and a state of the processing based on at least one of presentation part information on a state change of a presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part. and when the presentation part information indicates that a state change has occurred in the presentation part while the presentation information is being presented at least in the presentation part, the presentation part on which the presentation information is presented is changed. It is an information processing method.

[0008] In an information processing device, an information processing method, and a program according to one aspect of the present technology, at least one of a presentation state of the presentation information and a state of the processing is controlled based on at least one of presentation part information on a state change of a presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part. When the presentation part information indicates that a state change has occurred in the presentation part while the presentation information is being presented at least on the presentation part, the presentation part on which the presentation information is presented is changed. . [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration example of an embodiment of an information processing system to which the present technology is applied. [Diagram 2] FIG. 11 is a diagram illustrating an example of an action definition table. [Diagram 3] FIG. 11 is a diagram illustrating an example of an application state definition table. [Figure 4] FIG. 11 is a diagram illustrating an example of an urgency definition table. [Diagram 5] 11 is a flowchart illustrating an example of processing performed by an information processing device. [Figure 6] FIG. 1 is a plan view of an automobile in which an information processing device is applied as a projection device inside the automobile. [Figure 7]FIG. 1 is a side view of an automobile in which an information processing device is applied as a projection device inside the automobile. [Figure 8] 1 is a diagram illustrating an example of a position of a presentation portion to which presentation information is projected inside a car. [Figure 9] 13A-13C are diagrams illustrating the operation of a rear seat slidable armrest. [Figure 10] 11 is a diagram illustrating a state in which presentation information of a weather forecast application is projected onto an armrest. FIG. [Figure 11] 2 is a diagram illustrating an example of a record in which the “application type” field is a weather forecast application in the action definition table stored in the action definition storage unit of FIG. 1. FIG. [Figure 12] 12 is a diagram showing the state of the armrest and the projected image when the action of record R11 in FIG. 11 is executed. FIG. [Figure 13] 12 is a diagram illustrating an example of the state of the armrest and the projected image when the action of record R13 in FIG. 11 is executed. FIG. [Figure 14] 11 is a diagram illustrating a state in which presentation information of a music application is projected onto an armrest. FIG. [Figure 15] 1. FIG. 5 is a diagram illustrating an example of a record in which the "application type" field is a music application in the action definition table of FIG. 2 stored in the action definition storage unit of FIG. [Figure 16] 16 is a diagram illustrating an example of the state of the armrest and the projected image when the action of record R31 in FIG. 15 is executed. FIG. [Figure 17] 16 is a diagram illustrating an example of the state of the armrest and the projected image when the action of record R33 in FIG. 15 is executed. FIG. [Figure 18] 16 is a diagram illustrating an example of the state of the armrest and the projected image when the action of record R34 in FIG. 15 is executed. FIG. [Figure 19] 11 is a diagram illustrating an example of a state in which presented information of a car navigation application is projected onto a dashboard of a passenger seat. [Figure 20]1. FIG. 5 is a diagram illustrating an example of a record in which the “application type” field is a car navigation application and a gourmet application in the action definition table of FIG. 2 stored in the action definition storage unit of FIG. [Figure 21] 21 is a diagram illustrating an example of the state of the dashboard and the projected image when the action of record R51 in FIG. 20 is executed. FIG. [Figure 22] 21 is a diagram illustrating an example of the state of the dashboard and the projected image when the action of record R52 in FIG. 20 is executed. FIG. [Figure 23] 11 is a diagram illustrating an example of a state in which presented information of a gourmet app is projected onto a dashboard of a passenger seat. [Figure 24] 21 is a diagram illustrating an example of the state of the dashboard and the projected image when the action of record R53 in FIG. 20 is executed. FIG. [Diagram 25] 11A to 11C are diagrams illustrating an example of the operation of a headrest of a driver's seat. [Figure 26] 26 is a diagram illustrating a state in which presentation information of a video application is projected onto a headrest in state A of FIG. 25. FIG. [Figure 27] 1. FIG. 5 is a diagram illustrating an example of a record in which the “application type” field is a video application or a weather forecast application in the action definition table of FIG. 2 stored in the action definition storage unit of FIG. [Figure 28] 28 is a diagram illustrating an example of the state of the headrest and the projected image when the action of record R71 in FIG. 27 is executed. FIG. [Figure 29] 28 is a diagram illustrating an example of the state of the headrest and the projected image when the action of record R72 in FIG. 27 is executed. FIG. [Diagram 30] 11 is a diagram illustrating an example of the line of sight of a user viewing a headrest projection image onto which presentation information of a video application is projected. [Diagram 31] 13 is a diagram illustrating a state in which presentation information of a video application is projected onto a right door glass of a rear seat; FIG. [Diagram 32]1. FIG. 5 is a diagram illustrating an example of a record in which the “application type” field is a video application in the action definition table of FIG. 2 stored in action definition storage unit 52 of FIG. [Diagram 33] 33 is a diagram illustrating an example of the state of the right door glass and the projected image when the action of record R91 in FIG. 32 is executed. FIG. [Diagram 34] 11 is a diagram illustrating a state in which presented information of a map application is projected onto a ceiling. FIG. [Diagram 35] 3 is a diagram illustrating an example of a record in which the "application type" field is a map application in the action definition table of FIG. 2 stored in the action definition storage unit of FIG. 1. FIG. [Diagram 36] 36 is a diagram illustrating an example of the state of the interior of the vehicle and the projected image when record R111 or record R112 in FIG. 35 is executed. FIG. [Figure 37] 11 is a diagram illustrating the arrangement of a table in a room such as a conference room when the table top is used as a movable presentation part. FIG. [Figure 38] FIG. 13 is a diagram illustrating an example of a state in which presentation information of a discussion application is projected onto the tabletops of two joined tables. [Figure 39] 1. FIG. 5 is a diagram illustrating an example of a record in which the “application type” field is a discussion application in the action definition table of FIG. 2 stored in the action definition storage unit of FIG. [Diagram 40] 40 is a diagram illustrating an example of the state of the tabletop and the projected image when the action of record R121 in FIG. 39 is executed. [Diagram 41] 40 is a diagram illustrating an example of the state of the tabletop and the projected image when the action of record R122 in FIG. 39 is executed. [Diagram 42] FIG. 40 is a diagram illustrating another example 1 of the action of record R122 in FIG. [Diagram 43] FIG. 40 is a diagram illustrating another example 2 of the action of record R122 in FIG. [Diagram 44] FIG. 38 is a diagram illustrating another example 3 of the action of record R122 in FIG. [Diagram 45] FIG. 2 is a block diagram showing an example of the hardware configuration of a computer that executes a series of processes according to a program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0011] <<An embodiment of an information processing device to which the present technology is applied>> FIG. 1 is a block diagram showing an example of the configuration of an embodiment of an information processing device to which the present technology is applied.

[0012] In FIG. 1, an information processing device 11 is a device that projects presentation information generated by a predetermined process onto a movable presentation part by a projector and presents the information to a user in a room such as the interior of an automobile or a conference room.

[0013] The presented information is information generated by processing of an OS (Operating System) or a specific application (hereinafter, simply referred to as an application). The types of applications (processing) are not limited to specific types, such as a weather forecast application, a music application, a car navigation application, a gourmet application, a video application, a map application, and a discussion application.

[0014] The presentation part being movable means that the presentation part on which the presentation information is presented (projected) is the surface of a movable part. For example, the door glass or sunroof (sliding panel) of an automobile, or a movable table (top) in a conference room, etc., can be a movable presentation part. Note that there may be a plurality of presentation parts on which the presentation information is projected, and some of these presentation parts may be the surface of a fixed part (immovable presentation parts).

[0015] The information processing device 11 includes an input unit 21, a calculation processing unit 22, a storage unit 23, and an output unit 24.

[0016] The input unit 21 acquires “information about the presentation part” onto which the presentation information is projected, “information about the user”, and “information about the environment” of the presentation part, and supplies these to the calculation processing unit 22 as input information.

[0017] As specific components of the input unit 21, an appropriate sensor or the like is appropriately selected and used depending on the environment in which the presentation portion is provided.

[0018] For example, the sensor for acquiring the "information about the presentation part" and the "information about the user" may be one or more image sensors or depth sensors for acquiring images of the presentation part or the user. The image sensor may be a visible light camera or an infrared camera. The depth sensor may be a stereo camera for acquiring three-dimensional information about the presentation part or the user, or may be a ToF (time of flight) sensor.

[0019] Furthermore, the sensor or the like that obtains the "information regarding the presentation part" may be a position sensor that detects the position of a member that includes the presentation part.

[0020] It should be noted that a member including a presentation portion refers not only to a member that is integral with the member that becomes the presentation portion, but also to a member that works in conjunction with the member that becomes the presentation portion, that is, a member that is related to the state change of the presentation portion.

[0021] Furthermore, in the case where the device has an operation unit such as a touch sensor or an operation switch for electrically or otherwise moving a member including the presentation portion (the operation unit includes the operation unit of a remote controller; the same applies below), or in the case where the device has an operation unit for issuing predetermined instructions to the processing of an app or the like in the arithmetic processing unit 22, information indicating the action of the user operating the operation unit corresponds to "information about the user." In the case where the device has such an operation unit, the device may include the operation unit (a mechanism for detecting the state of the operation unit) as a sensor or the like for acquiring "information about the user."

[0022] The sensor etc. that acquires the "information about the environment" may be an acceleration sensor, an angular velocity sensor, a GPS (Global Positioning System), etc. For example, when the environment in which the member that serves as the presentation portion is provided is an automobile, the sensor etc. that acquires the "information about the environment" may be an acceleration sensor or a GPS for detecting whether the automobile is moving or stopped.

[0023] It should be noted that information obtained from a device other than the information processing device 11 may be supplied to the arithmetic processing unit 22 as input information from the input unit 21 .

[0024] The calculation processing unit 22 acquires necessary information (described later) based on the input information from the input unit 21 and the like.

[0025] Furthermore, the arithmetic processing unit 22 executes the OS program, executes one or more applications designated by the user, and supplies the output unit 24 with presentation information generated by the OS and the applications.

[0026] The storage unit 23 stores data and the like required for executing programs such as the OS and various applications executed by the arithmetic processing unit 22.

[0027] The output unit 24 has one or more projectors that project the presentation information supplied from the calculation processing unit 22 onto the presentation site. The projector may be a fixed projector whose projection direction is fixed, or a movable projector whose projection direction can be changed.

[0028] Here, the calculation processing unit 22 supplies the presentation information to the projector as an image of a window area corresponding to the position and size of the presentation part within the area of ​​the largest projection image that the projector can project (total projection area).

[0029] Furthermore, the calculation processing unit 22 supplies the projector with an image of the presentation information in which projection distortion has been corrected based on the inclination of the presentation portion (projection surface) relative to the projector, or the like.

[0030] Furthermore, by controlling (changing) the position and size of the window within the entire projection area using the calculation processing unit 22, it is possible to project images of presentation information onto presentation areas at multiple positions using one projector, and the position of the presentation area can be changed.

[0031] Furthermore, by setting a plurality of windows in the entire projection area by the calculation processing unit 22, it is possible to project an image of the same presentation information or a plurality of images of different presentation information onto a plurality of presentation sites simultaneously using one projector.

[0032] In addition, in the case of a movable projector, the calculation processing unit 22 may control the projection direction of the projector so that one projector projects images of presentation information onto presentation parts at multiple positions.

[0033] The output unit 24 also includes a speaker as a component for outputting music or voice as necessary. The arithmetic processing unit 22 supplies sound information such as music or voice generated by processing of an application or the like to the speaker of the output unit 24.

[0034] (Details of Processing by Arithmetic Processing Unit 22) Next, the details of the processing of the calculation processing unit 22 will be described. The calculation processing unit 22 has, as functional blocks, a graphics display processing unit 31 and a detection processing unit 32 that are constructed by executing a program. The detection processing unit 32 includes a database stored in the storage unit 23.

[0035] The graphics display processing unit 31 includes an input information analysis unit 41, an application execution unit 42, and a display control unit 43.

[0036] The input information analysis unit 41 analyzes the input information from the input unit 21 to obtain “presentation part information” relating to changes in the state of the presentation part, “user information” relating to the user's response to processing of an app, etc., and “environmental information” indicating the state of the environment of the presentation part.

[0037] "Presentation part information" is "information indicating a change in the state of the presentation part" when the movable presentation part that projects the presentation information moves, or "information indicating a user's action related to a change in the state of the presentation part."

[0038] The “information indicating a change in the state of the presentation part” is acquired by the input information analysis section 41 analyzing the “information on the presentation part” which is input information from the input section 21.

[0039] "Information indicating a user's action related to a change in the state of the presentation part" is information indicating a user's action of operating an operating unit to electrically or otherwise move a member including the presentation part, or information indicating a user's action of manually moving a member including the presentation part.

[0040] The “information indicating the user's action related to the change in state of the presentation part” is acquired by the input information analysis unit 41 analyzing the “information about the user” which is the input information from the input unit 21.

[0041] The "user information" includes information indicating the user's state with respect to the presented information (whether or not the user is viewing the presented information).

[0042] Furthermore, the "user information" includes "information indicating a user's operation for a process."

[0043] The "information indicating a user operation" further includes information indicating the content of the user operation for the process, and information indicating the presence or absence of the user operation for the process (whether or not an operation is being performed).

[0044] The “user information” is acquired by the input information analysis unit 41 analyzing the “information about the user” that is input information from the input unit 21.

[0045] For example, when the environment in which the presentation portion is provided is the interior of a car, the "environment information" is information indicating that the car is moving or stopped.

[0046] The “environmental information” is acquired by the input information analysis unit 41 analyzing the “information on the environment” that is input information from the input unit 21 .

[0047] The input information analysis unit 41 supplies the "presentation part information," "user information," and "environment information" acquired by analyzing the input information to the detection processing unit 32. The "user information" supplied to the detection processing unit 32 is specifically "information indicating the user's state (whether or not the user is browsing)" and "information indicating the presence or absence of a user operation on a process (whether or not the user is operating)."

[0048] Furthermore, the input information analysis unit 41 supplies "information indicating the user's operation content for the process" to the application execution unit 42 as "user information".

[0049] The application execution unit 42 starts an application designated by the user based on the “information indicating the user's operation content for the process” from the input information analysis unit 41, and supplies presentation information generated by the application to the display control unit 43.

[0050] Furthermore, the application execution unit 42 supplies to the detection processing unit 32 "information indicating the type of process (application)" that it is executing, and "information indicating the status (contents) of the process."

[0051] The "information indicating the type of process (application)" and the "information indicating the status (contents) of the process" are "processing information" related to the process (application) that generates the presentation information.

[0052] Furthermore, during execution of the application, the application execution unit 42 executes processing corresponding to the user's operation content based on the "user information" ("information indicating the user's operation content for processing") from the input information analysis unit 41.

[0053] Furthermore, the application execution unit 42 executes a process (described later) instructed by the detection processing unit 32.

[0054] The display control unit 43 controls the presentation state relating to the presentation part (area) and presentation form when the presentation information from the application execution unit 42 is projected onto the presentation part.

[0055] That is, the display control unit 43 supplies an image of the presentation information to the projector of the output unit 24 as a projection image of a window area corresponding to the position and size of the presentation part specified by the user based on the "information indicating the user's operation content for the processing" from the input information analysis unit 41.

[0056] Furthermore, the display control unit 43 controls the presentation form (image configuration, etc.) of the presentation information when the presentation information from the application execution unit 42 is projected onto a presentation part.

[0057] Furthermore, in accordance with an instruction from the detection processing unit 32, the display control unit 43 controls the presentation portion onto which the presentation information is projected and the presentation form of the presentation information.

[0058] The detection processing unit 32 includes an action execution unit 51, an action definition storage unit 52, an urgency definition storage unit 53, and an application state definition storage unit .

[0059] The action execution unit 51 acquires the “presentation part information”, “processing information”, “user information”, and “environment information” from the input information analysis unit 41 and the application execution unit 42 of the graphics display processing unit 31 .

[0060] The “user information” that the action execution unit 51 acquires from the input information analysis unit 41 is, specifically, “information indicating the user's state with respect to the presented information (whether or not the user is viewing the information)” and “information indicating whether or not the user has performed an operation on the processing (whether or not the user is performing an operation).”

[0061] Furthermore, the action execution unit 51 detects an action to be executed from the database stored in the storage unit 23 based on the acquired information.

[0062] Furthermore, the action execution unit 51 instructs the graphics display processing unit 31 (the application execution unit 42 and the display control unit 43) to execute processing of the action detected from the database.

[0063] The action definition storage unit 52 stores data (records) of an action definition table, which is a database for detecting actions to be executed by the action execution unit 51 (to be executed by the graphics display processing unit 31).

[0064] 2 is a diagram illustrating an example of an action definition table. The action definition table has "application type", "application status", "trigger", "urgency", and "action" as items (attributes) of each column. Each record (row) of the action definition table has a field for storing data corresponding to each column item.

[0065] That is, each record has fields for storing the contents of an "application type", an "application state", a "trigger", an "urgency", and an "action".

[0066] Hereinafter, the fields that store the data of each record's "application type", "application status", "trigger", "urgency", and "action" will be referred to as the "application type" field, the "application status" field, the "trigger" field, the "urgency" field, and the "action" field.

[0067] The “application type” field indicates the type of application that can be executed by the graphics display processing unit 31 (application execution unit 42). The “application type” field stores data indicating the type of application, such as a weather forecast application, a music application, a car navigation application, a video application, a map application, and a discussion application.

[0068] Based on the "processing information" from the graphics display processing unit 31, the action execution unit 51 detects a record in which the type of application being executed by the graphics display processing unit 31 is set as data in the "application type" field.

[0069] More specifically, the action execution unit 51 detects a record in which the type of application obtained from the "information indicating the type of process (application)" supplied from the application execution unit 42 is set as data in the "application type" field.

[0070] The "application status" field indicates at least one of the following: the status of the application's processing, the status of the user's response to the application's processing, and the status of the environment of the presentation part.

[0071] The processing state of an application, for example, in the case of a music application or a video application, indicates the processing state such as playing or stopping the content.

[0072] The state of the user's response to the process indicates the state of the user with respect to the presented information (projected image) (whether or not the user is viewing the image) and whether or not the user is performing an operation on the process (whether or not the user is performing an operation).

[0073] For example, when the presentation part is a member provided inside an automobile, the environmental state of the presentation part indicates an environmental state such as whether the automobile is moving or not.

[0074] The "application state" field stores one of the application states defined in an application state definition table in FIG. 3, which will be described later.

[0075] The action execution unit 51 detects, among the records in the action definition table, a record whose “application status” field data corresponds to the status of the processing, the status regarding the user's response to the processing, and the environmental status of the presentation part obtained from the “processing information,” “user information,” and “environmental information” from the graphics display processing unit 31 (application execution unit 42).

[0076] The "trigger" field indicates an event that triggers the execution of an action, such as a change in the state of a presentation part or a user action related to a change in the state of a presentation part.

[0077] The state change of the presentation part indicates, for example, a state change such as an armrest of a car serving as a movable presentation part being slid (the position being changed).

[0078] The user's action related to the state change of the presentation part of the presentation information represents, for example, the action of the user placing a hand on an armrest of a car as a movable presentation part in order to slide the armrest.

[0079] Based on the "presented part information" from the graphics display processing unit 31 (input information analysis unit 41), the action execution unit 51 detects a record in which a change in the state of the presented part, or a user action related to a change in the state of the presented part, is set as data in the "trigger" field.

[0080] The state change of the presentation part or the user's action represented by the data in the "trigger" field is simply called a trigger.

[0081] The "urgency" field indicates the degree to which the state of the presentation part after the change is related to physiology if no action is taken when the state of the presentation part is changed by a trigger represented by the data in the "trigger" field.

[0082] The "action" field indicates the process that the detection processing unit 32 executes (the detection processing unit 32 causes the graphics display processing unit 31 to execute) when the trigger indicated by the data in the "trigger" field occurs.

[0083] The "action" field indicates at least one of the following control contents: control of the presentation part (projection area) such as movement or change of the presentation part, control of the presentation form of the presentation information such as the configuration (layout) of the projection image of the presentation information or addition of warning information, and control of the application processing state. Note that the contents of each control may include a case where nothing is done, and when the contents of all controls are to do nothing, the "action" field indicates that nothing is done.

[0084] As described above, each record in the action definition table defines the execution condition of an action, that is, when the graphics display processing unit 31 is executing the application in the “Application Type” field and the application state in the “Application State” field is such that a trigger in the “Trigger” field occurs, the action in the “Action” field is executed.

[0085] In addition, an application represented by data in the “application type” field of any record in the action definition table is referred to as an application defined in the action definition storage unit 52 (for which an action is defined), and a trigger represented by data in the “trigger” field of any record in the action definition table is referred to as a trigger defined in the action definition storage unit 52 (for which an action is defined).

[0086] In FIG. 1, the application state definition storage unit 54 stores data of an application state definition table that defines the type of data in the "application state" field of the action definition table in FIG.

[0087] 3 is a diagram illustrating an example of an application state definition table. The application state definition table has "application state" and "meaning" as column items (attributes). Each record (row) of the application state definition table has a field that stores data corresponding to each column item.

[0088] That is, each record in the application state definition table has an "application state" field that stores the contents of the "meaning" and a "meaning" field.

[0089] In the "application status" field, for example, four types of application statuses, "operating", "viewing", "playing", and "normal", are stored. The meanings of "operating", "viewing", "playing", and "normal" are stored in the "meaning" field. According to the application status definition table in FIG. 3, "operating", "viewing", "playing", and "normal" in the "application status" field have the following meanings:

[0090] "In operation" represents a state in which a user is performing an operation on an app.

[0091] "Viewing" represents a state in which the user is viewing the presented information (projected image) of the app.

[0092] "Playing" represents the state in which the app is playing content.

[0093] "Normal" represents a state that does not fall into any other type of application state.

[0094] It should be noted that the "Meaning" field and data in the application state definition table do not necessarily need to be stored as table data.

[0095] In addition, although only four types of application states are illustrated in the application state definition table in Fig. 3, more types of application states may be defined. In the "Application State" field in the action state tables in Fig. 15, Fig. 20, and Fig. 39 illustrated below, application states further subdivided into "Operating", "Viewing", "Playing", and "Normal" are shown in parentheses.

[0096] In FIG. 1, an urgency definition storage unit 53 stores (preserves) an urgency definition table that defines the “urgency” of the application state definition table in FIG.

[0097] 4 is a diagram illustrating an example of an urgency definition table. The urgency definition table has "urgency" and "meaning" as items (attributes) of fields (columns). Each record (row) of the urgency definition table stores data corresponding to the items of each field.

[0098] In the "urgency" field, four types of data are exemplified: "Critical", "Major", "Minor", and "Trivial". The meanings of "Critical", "Major", "Minor", and "Trivial" are stored in the "Meaning" field. According to the application state definition table in FIG. 4, "Critical", "Major", "Minor", and "Trivial" in the "urgency" field have the following meanings:

[0099] "Critical" indicates that the condition is physiological and requires immediate action as indicated by the data in the "Action" field in Figure 2.

[0100] "Major" indicates that if the same condition continues for a long period of time, it may be related to menstruation.

[0101] "Minor" indicates a condition that causes inconvenience to the user.

[0102] "Trivial" represents a state in which no action needs to be executed.

[0103] Here, when the action execution unit 51 detects an action to be executed using the action definition table of FIG. 2, the data in the "urgency" field is not referenced. Therefore, it is possible not to provide a "urgency" column in the action definition table of FIG. 2, and not to define urgency using the urgency definition table of FIG. 4.

[0104] However, for example, even if the records have different application types in the "Application Type" field of the action definition table in Figure 2, if the data in the "Application Status" field, the "Trigger" field, and the "Urgency" field are common, the data in the "Action" field can be common.

[0105] Therefore, for example, when adding a new application to the action definition table, the urgency of the "Urgency" field when a trigger in the "Trigger" field occurs in the application state of the "Application State" field can be selected from the urgency defined in the urgency definition table and set, so that the data in the "Action" field can be automatically determined.

[0106] The data of the "action" field that is automatically determined may be data of the "action" field of another application that has common data in the "application status" field, the "trigger" field, and the "urgency" field, or may be data of the "action" field that is determined in advance for each combination of data in the "application status" field, the "trigger" field, and the "urgency" field. This reduces the effort required to create an action definition table by considering actions in the "action" field for each application, and makes it easy to add a new application to the applications in the action definition table.

[0107] It should be noted that the "Meaning" field and data in the urgency definition table of FIG. 4 do not necessarily need to be stored as table data.

[0108] FIG. 5 is a flowchart illustrating an example of processing performed by the information processing device 11.

[0109] In step S11, the graphics display processing unit 31 starts projection of the presentation information by supplying the presentation information to the output unit 24. The process proceeds from step S11 to step S12.

[0110] Here, the graphics display processing unit 31 starts projecting the projection image when it detects that the user has performed a switch operation, gesture, or the like to instruct projection of the projection image onto a predetermined presentation part based on input information from the input unit 21. In addition, when starting the projection, the graphics display processing unit 31 projects, for example, a guide image for the user to select an application to be started onto the presentation part as an initial projection image.

[0111] In step S12, the graphics display processing unit 31 starts the application selected by the user based on the user's operation ("information indicating the user's operation content") acquired from the input information ("information about the user") from the input unit 21.

[0112] Then, the graphics display processing unit 31 supplies the presentation information generated by the started application to the projector of the output unit 24, and projects the presentation information onto the presentation part. The process proceeds from step S12 to step S13.

[0113] In step S13, the graphics display processing unit 31 determines, based on the user's operation, whether or not the user has performed an operation to end the projection of the projection information.

[0114] If it is determined in step S13 that the end operation has been performed, the process of this flowchart ends.

[0115] If it is determined in step S13 that the end operation has not been performed, the process proceeds from step S13 to step S14.

[0116] In step S14, the graphics display processing unit 31 acquires input information from the input unit 21. Then, the graphics display processing unit 31 analyzes the acquired input information to acquire "presentation part information", "user information", and "environment information".

[0117] The graphics display processing unit 31 supplies the “presentation part information,” “user information,” and “environmental information” obtained by analyzing the input information, as well as information indicating the type of processing (application) being executed by the graphics display processing unit 31 and information indicating the status of the processing being executed by the graphics display processing unit 31, to the detection processing unit 32 as “processing information.”

[0118] That is, the graphics display processing unit 31 supplies the "presentation part information", "processing information", "user information", and "environment information" to the detection processing unit 32. The process proceeds from step S14 to step S15.

[0119] In step S15, the detection processing unit 32 determines whether or not the graphics display processing unit 31 is executing any of the applications defined in the action definition storage unit 52.

[0120] That is, based on the “processing information” supplied from the graphics display processing unit 31 in step S14, the detection processing unit 32 determines whether or not the application being executed by the graphics display processing unit 31 exists as data in the “application type” field of any record in the action definition table of FIG. 2.

[0121] If it is determined in step S15 that the graphics display processing unit 31 is not executing the application defined in the action definition storage unit 52, the process returns from step S15 to step S13, and steps S13 to S15 are repeated.

[0122] On the other hand, if it is determined in step S15 that the graphics display processing unit 31 is executing an application defined in the action definition storage unit 52, the process proceeds from step S15 to step S16.

[0123] In step S16, the detection processing unit 32 determines whether or not any of the triggers defined in the action definition storage unit 52 has been detected.

[0124] That is, the detection processing unit 32 acquires a state change of the presentation part or a user action related to the state change of the presentation part based on the "presentation part information" acquired in step S14. Then, the detection processing unit 32 determines whether or not the acquired state change of the presentation part or the user action exists as a trigger represented by data in the "trigger" field in any record for the application executed by the graphics display processing unit 31 in the action definition table in FIG.

[0125] If it is determined in step S16 that the trigger defined in the action definition storage unit 52 has not occurred, the process returns from step S16 to step S13, and the processes from step S13 to step S16 are repeated.

[0126] On the other hand, if it is determined in step S16 that a trigger defined in the action definition storage unit 52 has been detected, the process proceeds from step S16 to step S17.

[0127] In step S17, the detection processing unit 32 detects, from among the records in the action definition table of FIG. 2 for the application being executed by the graphics display processing unit 31 and the trigger detected in step S16, a record in which the application state obtained from the “processing information,” “user information,” and “environmental information” acquired in step S14 is set as data in the “application state” field.

[0128] Then, the detection processing unit 32 causes the graphics display processing unit 31 to execute the action in the "action" field of the detected record as an action according to the application state. The process returns from step S17 to step S13, and steps S13 to S17 are repeated.

[0129] If it is determined in step S13 that a termination operation has been performed, the graphics display processing unit 31 ends the processing of this flowchart.

[0130] <<Application example 1 of information processing device 11>> As a first application example of the information processing device 11 in FIG. 1, a case will be described in which the information processing device 11 is applied as a projection device that projects presentation information of various applications onto a movable presentation portion inside an automobile.

[0131] 6 and 7 are a plan view and a side view of an automobile 71 in which the information processing device 11 is applied as a projection device inside the automobile.

[0132] 6 and 7, projectors 81, 82, 83, and 84, which are the output unit 24 of the information processing device 11 in FIG.

[0133] Projector 81 is installed near the center in the vertical and horizontal directions at the rear of the vehicle. Projector 82 is installed at the upper part of the right side at the rear of the vehicle. Projector 83 is installed at the upper part of the left side at the rear of the vehicle. Projector 84 is installed at the front part of the ceiling of the vehicle.

[0134] FIG. 8 is a diagram illustrating the positions of presentation sites (projection surfaces) in the interior 91 of the automobile 71 onto which presentation information is projected.

[0135] In the interior 91 of the automobile 71, a ceiling 92 is used as a presentation part, and a projected image 93 is projected onto the ceiling 92 from a projector 81. If the automobile 71 is equipped with a sunroof, a sliding panel that blocks the sunroof (hereinafter simply referred to as a sunroof) is used as a movable presentation part.

[0136] Further, a left door glass 94 in the rear seat is used as a movable presentation portion, and a projection image 95 is projected onto the left door glass 94 from the projector 82 .

[0137] In addition, a right door glass 96 in the rear seat is used as a movable presentation portion, and a projection image 97 is projected onto the right door glass 96 from a projector 83 .

[0138] In addition to the ceiling 92, and the left and right door windows 94 and 96 of the rear seats, an armrest 98 in the center of the rear seats, a headrest 99 of the driver's seat, a headrest 100 of the passenger seat, a door glass 101 of the passenger seat, and a dashboard (door of the passenger seat glove box) 102 are used as movable presentation parts. For example, a projection image from any of the projectors 81 to 84 is projected onto the armrest 98.

[0139] An image is projected from any one of the projectors 81 to 83 onto a headrest 99 of the driver's seat and a headrest 100 of the passenger's seat.

[0140] An image is projected from the projector 84 onto the passenger door glass 101 and the dashboard 102.

[0141] <When the sliding armrest 98 of the rear seat is used as the movable presentation part> (When weather forecast app information is projected) FIG. 9 illustrates the operation of the rear seat slidable armrest 98.

[0142] 9, the armrest 98 is provided to be slidable in the front-rear direction on an upper portion of a rectangular base 122 installed in the center of the rear seat 121. In state A, the armrest 98 is disposed in a standard position at the front end of its movable range relative to the base 122. In state B, the armrest 98 is slid toward the rear of its movable range relative to the base 122, and the storage section 123 of the base 122 is opened.

[0143] FIG. 10 is a diagram illustrating a state in which presentation information of a weather forecast application is projected onto an armrest 98. As shown in FIG.

[0144] When the user performs an operation to project presentation information of a weather forecast application onto the armrest 98 as a presentation part, a projection image 141 of the presentation information of the weather forecast application is projected onto the armrest 98 by, for example, the projector 82 in Fig. 6. The projection image 141 is projected onto, for example, the front end side of the upper surface of the armrest 98.

[0145] When the user then slides the armrest 98 rearward, a trigger is generated.

[0146] FIG. 11 is a diagram illustrating an example of a record in which the "application type" field is a weather forecast application in the action definition table of FIG. 2 stored in action definition storage unit 52 of FIG.

[0147] When a user is operating a weather forecast app and slides (shifts) armrest 98 backward as shown in state B of Figure 9, the app type is a weather forecast app, the app state is operating, and the trigger is the user's action of shifting armrest 98 or a change in the position of armrest 98.

[0148] In this case, the situation corresponds to record R11 in FIG. 11, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R11 in FIG.

[0149] FIG. 12 is a diagram illustrating the state of the armrest 98 and the projected image when the action of record R11 in FIG. 11 is executed.

[0150] When the action of record R11 is executed, the projected image 141, which is the presentation information of the weather forecast application, slides backward together with the armrest 98, as shown in FIG.

[0151] That is, the area of ​​the window onto which the presentation information of the weather forecast application is projected (hereinafter referred to as the application window) slides backward together with the armrest 98, and the projection image 141 of the presentation information slides backward together with the armrest 98. As a result, the entire presentation information of the weather forecast application is projected onto the front end side of the surface of the armrest 98 without any loss.

[0152] When the user is operating a weather forecast application, it is considered that the user will continue to use the weather forecast application when the armrest 98 is slid. As shown in Fig. 12, the application window slides together with the sliding of the armrest 98, and the entire presented information of the weather forecast application is projected as the projection image 141, thereby maintaining usability.

[0153] Next, when the user is not operating the weather forecast app but is viewing the projected image 141, which is the presentation information of the weather forecast app, and slides (shifts) the armrest 98 backward as shown in state B of Figure 9, the app type is the weather forecast app, the app state is viewing, and the trigger is the user's action of shifting the armrest 98 or a change in the position of the armrest 98.

[0154] In this case, the situation corresponds to record R12 in FIG. 11, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R12 in FIG.

[0155] When the action of record R12 is executed, the application window of the weather forecast application slides backward together with the armrest 98, as in the case of record R11, and the projected image 141 slides backward together with the armrest 98, as in FIG.

[0156] When the user is viewing a weather forecast application, it is considered that the user will continue to use the weather forecast application when the armrest 98 is slid. As shown in Fig. 12, the application window slides together with the sliding of the armrest 98, and the entire presented information of the weather forecast application is projected as the projection image 141, thereby maintaining usability.

[0157] Next, when the user is not operating the weather forecast app and is not viewing the projected image 141, which is the presentation information of the weather forecast app, and slides (shifts) the armrest 98 backward as shown in state B of Figure 9, the app type is the weather forecast app, the app state is normal, and the trigger is the user's action of shifting the armrest 98 or a change in the position of the armrest 98.

[0158] In this case, the situation corresponds to record R13 in FIG. 11, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R13 in FIG.

[0159] FIG. 13 is a diagram illustrating the state of the armrest 98 and the projected image 141 when the action of record R13 in FIG. 11 is executed.

[0160] In the action of record R13, the detection processing unit 32 does nothing in response to the sliding of the armrest 98. As a result, the application window that projects the presentation information of the weather forecast application is not changed, and the projection image 141 does not slide together with the armrest 98 as shown in Fig. 13. Therefore, part of the presentation information of the weather forecast application is missing.

[0161] However, since the user is neither operating nor viewing the screen, no particular inconvenience occurs even if part of the projected image 141 of the presented information of the weather forecast application is temporarily lost. The user can view the entire presented information of the weather forecast application by returning the armrest 98 to the standard position of state A in FIG.

[0162] In addition, the actions in the “Action” fields of records R11 to R13 of the action definition table in FIG. 11 can be applied as actions for any type of application and any presentation part (including but not limited to automobiles) whose “Application Status” and “Trigger” fields can be classified into the same combinations as those of records R11 to R13.

[0163] For example, when the presentation information of an image viewing application is projected onto the armrest 98, or when the presentation information of the image viewing application is projected onto a presentation part other than the armrest 98 where a state change similar to that of the armrest 98 (not limited to a state change due to sliding) occurs, the actions in the "Action" fields of records R11 to R13 can be applied.

[0164] (When music app information is projected) FIG. 14 is a diagram illustrating a state in which presentation information of a music application is projected onto an armrest 98. As shown in FIG.

[0165] 9, when the user performs an operation to project presentation information of a music application onto the armrest 98 as a presentation part, a projection image 161 of the presentation information of the music application is projected onto the armrest 98 by, for example, the projector 82 of Fig. 6. The projection image 161 is projected onto, for example, the front end side of the upper surface of the armrest 98.

[0166] FIG. 15 is a diagram illustrating an example of a record in which the "application type" field is a music application in the action definition table of FIG. 2 stored in action definition storage unit 52 of FIG.

[0167] When a music app is playing a specific piece of music or the like at a certain volume or above and the user places their hand on the armrest 98 to slide (move) the armrest 98 backwards, the app type is a music app, the app state is playing at a certain volume or above, and the trigger is the action of the user placing their hand on the armrest 98 (the gesture of trying to move the armrest 98).

[0168] In this case, the situation corresponds to record R31 in FIG. 15, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 to execute the action in the "action" field of record R31 in FIG.

[0169] FIG. 16 is a diagram illustrating the state of the armrest 98 and the projected image 161 when the action of record R31 in FIG. 15 is executed.

[0170] When the action of record R31 is executed, as shown in FIG. 16, an image is projected in which warning information 162 in the form of a text image such as "If you slide it, you will not be able to operate it, it is dangerous!" is superimposed on the projected image 161 of the presentation information of the music app.

[0171] The warning information 162 prevents an unexpected situation in which the armrest 98 is slid and the volume cannot be adjusted. That is, if the volume adjustment operation becomes impossible when a music application is being played at a certain volume or more, the volume may suddenly become very loud depending on the music. If the user slides the armrest 98, some of the presented information of the music application may be lost, and the volume adjustment operation may become impossible, so the warning information 162 warns the user not to slide the armrest 98.

[0172] A trigger occurs when the user slides (shifts) the armrest 98 backward as in state B in Fig. 9 despite the warning information 162 being presented as in Fig. 16. In this case, the type of application is a music application, the application state is playing at a certain volume or above, and the trigger is the user's action of shifting the armrest 98 or a change in the position of the armrest 98.

[0173] In this case, the situation corresponds to record R32 in FIG. 15, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 to execute the action in the "action" field of record R32 in FIG.

[0174] When the action of record R32 is executed, playback by the music app is changed to a paused state. Note that, although not shown in the figure, the app window of the music app is not changed, and some of the information presented by the music app is lost.

[0175] Next, when a music app is playing a specific piece of music or the like at less than a certain volume and the user slides (shifts) the armrest 98 backward as shown in state B of Figure 9, the app type is a music app, the app state is playing at less than a certain volume, and the trigger is the user's action of shifting the armrest 98 or a change in the position of the armrest 98.

[0176] In this case, the situation corresponds to record R33 in FIG. 15, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 to execute the action in the "action" field of record R33 in FIG.

[0177] FIG. 17 is a diagram showing an example of the state of the armrest 98 and the projected image when the action of record R33 in FIG. 15 is executed.

[0178] When the action of record R33 is executed, the application window that projects the presentation information of the music application is not changed, and the icon 163 is projected at a position that can be projected onto the armrest 98 within the range of the application window when the armrest 98 is slid to a position. Also, for example, an operation using an air gesture on the icon 163 becomes valid, and the user can stop the playback of music using a gesture.

[0179] Changing the projected image 161 of the presented information of the music application to the icon 163 is particularly effective when the armrest 98 is slid and there is no longer an area on the armrest 98 for projecting the entire presented information of the music application.

[0180] Next, when the music app is not playing music or the like and the user slides (shifts) the armrest 98 backward as shown in state B of Figure 9, the app state is the music app, the app state is normal, and the trigger is the user's action of shifting the armrest 98 or a change in the position of the armrest 98.

[0181] In this case, the situation corresponds to record R34 in FIG. 15, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R34 in FIG.

[0182] FIG. 18 is a diagram illustrating the state of the armrest 98 and the projected image 161 when the action of record R34 in FIG. 15 is executed.

[0183] In the action of record R34, the detection processing unit 32 does nothing in response to the sliding of the armrest 98. As a result, the application window that projects the presentation information of the music application is not changed, and the projection image 161 does not slide together with the armrest 98 as shown in Fig. 18. Therefore, part of the presentation information of the music application is missing.

[0184] In addition, the actions in the “Action” fields of records R31 to R34 in the action definition table in FIG. 15 can be applied as actions for any type of application and any presentation part (including but not limited to automobiles) whose “Application Status” and “Trigger” fields can be classified into the same combinations as those of records R31 to R34.

[0185] <When the dashboard 102 that can be opened and closed at the passenger seat is used as a movable presentation part> (When information displayed by a car navigation app is projected) FIG. 19 is a diagram illustrating a state in which presented information of a car navigation application is projected onto dashboard 102 in the passenger seat.

[0186] 19, for example, information presented by a car navigation application is projected as a projection image 181 by the projector 84 in Fig. 6 and Fig. 7 onto the dashboard 102 of the passenger seat. When the user presses an opening / closing button 191 for opening or closing the dashboard 102 in a state in which the dashboard 102 is closed as in Fig. 19, a trigger is generated.

[0187] FIG. 20 is a diagram illustrating an example of a record in which the "application type" field is a car navigation application and a gourmet application in the action definition table of FIG. 2 stored in the action definition storage unit 52 of FIG.

[0188] In a state in which a projection image 181 of presentation information of a car navigation application is projected onto the dashboard 102 as shown in FIG. 19, when the user presses the open / close button 191 on the dashboard 102 while the car 71 is traveling, the application type is a car navigation application, the application state is traveling, and the trigger is the action of the user pressing the open / close button 191 on the dashboard 102.

[0189] In this case, the situation corresponds to record R51 in FIG. 20, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R51 in FIG.

[0190] FIG. 21 is a diagram illustrating an example of the state of the dashboard 102 and the projected image 181 when the action of record R51 in FIG. 20 is executed.

[0191] As shown in FIG. 21, when a user's hand 192 presses an open / close button 191, a trigger is generated and the action of record R51 is executed.

[0192] When the action of record R51 is executed, an image is projected on which warning information 182 in the form of a text image such as "If you open it, you will not be able to see the car navigation system, which will interfere with your driving and is dangerous!" or "Do not open until the vehicle has stopped" is superimposed on the projection image 181 of the information presented by the car navigation app.

[0193] The warning information 182 warns the driver not to open the dashboard 102 because if the dashboard 102 is opened, the information presented by the car navigation application will become invisible.

[0194] Moreover, when the automobile 71 is traveling, the dashboard 102 is locked in a closed state and is controlled so as not to be opened even if the open / close button 191 is pressed. The lock mechanism of the dashboard 102 is controlled, for example, by the detection processing unit 32 transmitting a control signal to the lock mechanism.

[0195] Note that, instead of the user pressing the open / close button 191, the action of the user's hand (finger) approaching the open / close button 191 may be used as the trigger.

[0196] Also, when the open / close button 191 is pressed by a user in the passenger seat, rather than the driver, the action of record R51 may not be executed, and the dashboard 102 may be opened as a no-action action.

[0197] Next, in a state where a projection image 181 of presentation information of a car navigation application is projected onto the dashboard 102 and the car 71 is stopped, when the user presses the open / close button 191 on the dashboard 102, the application type is a car navigation application, the application state is stopped, and the trigger is the action of the user pressing the open / close button 191 on the dashboard 102.

[0198] In this case, the situation corresponds to record R52 in FIG. 20, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R52 in FIG.

[0199] FIG. 22 is a diagram illustrating an example of the state of the dashboard 102 and the projected image 181 when the action of record R52 in FIG. 20 is executed.

[0200] In the action of record R52, the detection processing unit 32 does nothing in response to the pressing of the open / close button 191. As a result, the dashboard 102 opens upward.

[0201] When dashboard 102 is opened, storage section 193 that was closed by dashboard 102 is opened. In addition, projected image 181 that was projected onto dashboard 102 is projected into storage section 193. However, if objects are stored in storage section 193, the image is projected onto the surfaces of those objects, and is not necessarily projected onto a flat projection surface.

[0202] In addition, the actions in the “Action” fields of records R51 and R52 in the action definition table in FIG. 20 can be applied as actions for any type of application and any presentation part (including but not limited to automobiles) whose “Application Status” and “Trigger” fields can be classified into the same combinations as records R51 and R52.

[0203] (When information presented by a gourmet app is projected) FIG. 23 is a diagram illustrating a state in which presented information of a gourmet app is projected onto dashboard 102 in the passenger seat.

[0204] 23 , it is assumed that a user presses open / close button 191 to open or close dashboard 102 while projection image 201 of presentation information of a gourmet app is projected onto dashboard 102. In this case, the type of the app is a gourmet app, the app state is normal, and the trigger is the action of the user pressing open / close button 191 on dashboard 102.

[0205] In this case, the situation corresponds to record R53 in FIG. 20, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R53 in FIG.

[0206] FIG. 24 is a diagram illustrating an example of the state of the dashboard 102 and the projected image 201 when the action of record R53 in FIG. 20 is executed.

[0207] In the action of record R52, the detection processing unit 32 does nothing in response to the pressing of the open / close button 191. As a result, the dashboard 102 opens upward.

[0208] When the dashboard 102 is opened, the storage section 193 that was closed by the dashboard 102 is opened. In addition, the projection image 201 that was projected onto the dashboard 102 is projected into the storage section 193.

[0209] When the presented information of the gourmet app is projected on the dashboard 102, the dashboard 102 opens according to a user's operation so that driving is not hindered even if the presented information of the gourmet app becomes unable to be viewed due to the dashboard 102 being open.

[0210] <When the movable headrest 99 is used as the movable presentation part> (When information displayed by a car navigation app is projected) FIG. 25 is a diagram illustrating the operation of the headrest 99 of the driver's seat 221.

[0211] In state A and state B of FIG. 25, the inclination of the headrest 99 of the driver's seat 221 in the front-rear direction is changed by adjusting the inclination of the backrest of the driver's seat 221 or by adjusting the inclination of the headrest 99 itself.

[0212] State A indicates a state in which the headrest 99 is tilted further forward than state B, and state B indicates a state in which the headrest 99 is tilted further rearward than state A.

[0213] In state A, when the user performs an operation to project presentation information of a video application using the rear side of the headrest 99 (hereinafter simply referred to as the headrest 99) as a movable presentation part, the presentation information of the video application is projected onto the headrest 99, for example, by the projector 83 in Figures 6 and 7.

[0214] FIG. 26 is a diagram illustrating a state in which presentation information of a video application is projected onto a headrest 99 in state A of FIG.

[0215] 26, presentation information of a video application is projected onto headrest 99 as projection image 241. Projection image 241 has a playback unit 242 where a video is presented, and an operation unit 243 for operating playback and stop of an action. In a state where headrest 99 is tilted forward as in state A of FIG. 25 (a state where the headrest is tilted forward more than a predetermined angle of tilt backward), playback unit 242 is projected below operation unit 243.

[0216] When a user (such as a driver) reclines the headrest 99 or the driver's seat 221 backward (reclines the seat) in a state in which the projection image 241 is projected onto the headrest 99 as shown in FIG. 26, a trigger is generated.

[0217] FIG. 27 is a diagram illustrating an example of a record in which the "application type" field is a video application or a weather forecast application in the action definition table of FIG. 2 stored in the action definition storage unit 52 of FIG.

[0218] When a video application is playing a specific video and the user reclines the seat back from state A to state B in FIG. 25, the application type is a video application, the application state is playing, and the trigger is the user's action of reclining the seat back or a change in the tilt of the driver's seat 221 or headrest 99.

[0219] In this case, the situation corresponds to record R71 in FIG. 27, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R71 in FIG.

[0220] FIG. 28 is a diagram illustrating the state of the headrest 99 and the projected image 241 when the action of record R71 in FIG. 27 is executed.

[0221] When the action of record R71 is executed, the positions of the playback section 242 and the operation section 243 in the projected image 241 are swapped vertically according to the inclination of the headrest 99.

[0222] When the headrest 99 is tilted backward as in state B in Fig. 25 (when tilted backward at a predetermined angle), the playback unit 242 in the projected image 241 is projected above the operation unit 243 as in Fig. 28. This prevents the line of sight of the user viewing the playback unit 242 from changing significantly from before the reclining position.

[0223] On the other hand, if the video app is not playing and the user reclines from state A to state B in Figure 25, the app type in this case is a video app, the app state is normal, and the trigger state is the user's action of reclining the seat or a change in the tilt of the driver's seat or headrest 99.

[0224] In this case, the situation corresponds to record R72 in FIG. 27, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R72 in FIG.

[0225] FIG. 29 is a diagram illustrating the state of the headrest 99 and the projected image 241 when the action of record R72 in FIG. 27 is executed.

[0226] In the action of record R72, the detection processing unit 32 does nothing in response to the reclining. Therefore, the arrangement of the playback unit 242 and the operation unit 243 in the projection image 241 is the same as in the case of FIG. 26, with the playback unit 242 at the bottom and the operation unit 243 at the top.

[0227] Even when the video application is not playing, if the seat is reclined, the playback unit 242 may be projected above the operation unit 243, just as when the video application is playing a video.

[0228] FIG. 30 is a diagram illustrating an example of a line of sight 254 of a user 253 viewing a projected image 241 on a headrest 99 onto which presentation information of a video application is projected.

[0229] A in Fig. 30 shows a line of sight 254 of a user 253 when a projected image 241 on the headrest 99 is laid out such that a playback unit 242 is disposed on the lower side and an operation unit 243 is disposed on the upper side in a state in which the headrest 99 is tilted forward as in state A in Fig. 25. In A in Fig. 30, the line of sight 254 of the user 253 is, for example, substantially horizontal.

[0230] 26, 28, and 29 show the boundary between a windshield 251 and an opaque portion 252 (instrument panel or dashboard), and the field of view of a user 253 in a line of sight 254 of A in FIG. 30 is as shown in FIG. 26.

[0231] In FIG. 26, the user's peripheral vision includes much of the scenery outside the car seen through the windshield 251, so the user can be aware of the speed and swaying of the car 71 while watching the video on the playback unit 242, and is less likely to get car sick.

[0232] B in Figure 30 shows the line of sight 254 of the user 253 when the headrest 99 is tilted backward as in state B in Figure 25, and the projected image 241 on the headrest 99 is laid out in the same manner as A in Figure 30, with the playback unit 242 at the bottom and the operation unit 243 at the top.

[0233] At B in Fig. 30, the line of sight 254 of the user 253 is directed downward, and the field of view of the user 253 is as shown in Fig. 29. In Fig. 29, the scenery outside the vehicle through the windshield 251 is reduced in the user's peripheral vision, making the user more susceptible to car sickness.

[0234] C in Figure 30 shows the line of sight 254 of the user 253 when the headrest 99 is tilted backward as in state B in Figure 25, and the projected image 241 on the headrest 99 is laid out in such a way that the playback unit 242 is on the upper side and the operation unit 243 is on the lower side.

[0235] In C of Fig. 30, the line of sight 254 of the user 253 is approximately horizontal as in A of Fig. 30, and the field of view of the user 253 is as shown in Fig. 28. Fig. 28 is a diagram for explaining the layout of the projected image 241 in which the playback unit 242 is arranged on the upper side and the operation unit 243 is arranged on the lower side, and therefore does not accurately depict the state of the user's peripheral field of view. However, if the vicinity of the center of the playback unit 242 is set as the center of the field of view, the user's peripheral field of view includes much of the scenery outside the vehicle through the windshield 251 as in the case of A of Fig. 30, and therefore the user is less likely to get car sick.

[0236] Therefore, when the seat is reclined while a video is being played back, a projection image 241 is projected with a playback unit 242 on the upper side and an operation unit 243 on the lower side, as shown in FIG. 28 and FIG. 30C.

[0237] On the other hand, when the presentation information of a weather forecast application is projected onto the headrest 99, if the seat is reclined, the action of record R73 in Fig. 27 is executed. In this case, the detection processing unit 32 in Fig. 1 does nothing in response to the reclining. Unlike the case of the presentation information of a video application, the case of the presentation information of a weather forecast application does not take into consideration car sickness because the user does not keep his or her eyes on the projected image.

[0238] In the action definition table of FIG. 27, the trigger is the user reclining the seat back (or a change in the inclination of the driver's seat or headrest 99), but the trigger may also be the user moving the seat back or a change in the inclination of the seat back, including the user reclining the seat back.

[0239] In this case, when the reclining angle becomes more forward than a predetermined angle, or when the amount of forward change in the reclining angle is greater than a predetermined amount, the playback section 242 of the projection image 241 is projected below the operation section 243.

[0240] When the reclining angle becomes more rearward than a predetermined angle, or when the amount of rearward change in the reclining angle is greater than a predetermined amount, the playback unit 242 of the projected image 241 may be positioned above the operation unit 243.

[0241] Also, when the headrest of another seat such as the passenger seat is used as the movable presentation part, the same processing as that when the headrest 99 of the driver's seat 221 is used as the movable presentation part can be applied.

[0242] In addition, the actions in the “Action” fields of records R71 and R72 in the action definition table in FIG. 27 can be applied as actions for any type of application and any presentation part (including but not limited to automobiles) whose “Application Status” and “Trigger” fields can be classified into the same combinations as those of records R71 and R72.

[0243] <When the rear seat right door glass 96 is used as the movable presentation part> (When video app display information is projected) FIG. 31 is a diagram illustrating a state in which presented information of a video application is projected onto a right door glass 96 of the rear seat.

[0244] When a user performs an operation to project presentation information of a video application onto the right door glass 96 (see FIG. 8) in the rear seat as a movable presentation part, the presentation information of the video application is projected as a projection image 271 onto the right door glass 96 by, for example, projector 83 in FIG. 6, as shown in FIG. 31.

[0245] In a state in which the projection image 271 is projected onto the right door glass 96 as shown in FIG. 31, when a user 281 or the like opens the window (right door glass 96) or the door (right door), a trigger is generated.

[0246] FIG. 32 is a diagram illustrating an example of a record in which the "application type" field is a video application in the action definition table of FIG. 2 stored in action definition storage unit 52 of FIG.

[0247] When a video application is playing a specific video and a user 281 or the like opens a window, the application type is a video application, the application state is playing, and the trigger is the user's action of opening the window or the displacement of the window (right door glass 96) due to the lowering.

[0248] In this case, the situation corresponds to record R91 in FIG. 32, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R91 in FIG.

[0249] FIG. 33 is a diagram showing an example of the state of the right door glass 96 and the projected image 271 when the action of record R91 in FIG. 32 is executed.

[0250] For example, when the user 281 operates the power window switch in the opening direction, the action of record R91 is executed, and the playback of the video in the video application is paused.

[0251] This prevents the video playback from progressing while the right door glass 96 is lowered and the projection image 271 cannot be projected.

[0252] On the other hand, when the video application is not playing a video and the user 281 etc. opens a window, the application type is a video application, the application state is normal, and the trigger is the user's action of opening the window or the displacement due to the window lowering.

[0253] In this case, the situation corresponds to record R92 in FIG. 32, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R92 in FIG.

[0254] In the action of record R92, the detection processing unit 32 does nothing in response to the window being opened.

[0255] Next, when the video application is playing a video or in normal mode and user 281 etc. opens the right door of the rear seat, the application type in this case is a video application, the application state is playing or normal, and the trigger is the user's action of opening the door or the displacement of the door.

[0256] In this case, the situation corresponds to record R93 in Fig. 32, so the detection processing unit 32 in Fig. 1 causes the graphics display processing unit 31 in Fig. 1 to execute the action in the "action" field of record R93 in Fig. 32. Note that the normal application state and the playing application state are essentially separate records, but for the sake of brevity, they are treated as the same record.

[0257] When the action of record R93 is executed, the video being played by the video application is paused, and the projector (projector 83) projecting the projection image 271 is turned off. Note that all projectors may be turned off. This prevents the projector light from being irradiated onto the eyes of the user who is ascending or descending.

[0258] In addition, when other door glass, such as the rear seat left door glass 94 or the passenger seat door glass 101, is used as the movable presentation part, the same processing as when the rear seat right door glass 96 is used as the movable presentation part can be applied.

[0259] In addition, the actions in the "Action" fields of records R91 and R92 in the action definition table of FIG. 32 can be applied as actions for any type of application and any presentation part (including but not limited to automobiles) whose "application state" and "trigger" fields can be classified into the same combinations as those of records R91 and R92.

[0260] Furthermore, the action in the "Action" field of record R93 in the action definition table of Figure 32 can be applied as an action for any type of application and any presentation part (not limited to automobiles) whose "application state" field and "trigger" field can be classified into the same combinations as record R93 (application state is normal and playing, and trigger is a change in the state of the presentation part).

[0261] <When a roof with a sunroof is used as a movable presentation part> (When information provided by a map app is projected) FIG. 34 is a diagram illustrating a state in which presented information of a map application is projected onto a ceiling 92. As shown in FIG.

[0262] When the user 311 or the like performs an operation to project presentation information of a map application onto a portion of the ceiling 92 including the sunroof as a movable presentation part, the presentation information of the map application is projected onto the ceiling 92 as a projection image 291, for example, by the projector 81 in Figures 6 and 7.

[0263] In Fig. 34, the sunroof is in a light-blocking state, for example, by a sliding panel. When the sliding panel slides and retreats from the sunroof, the sunroof becomes in a light-transmitting state. In the following, putting the sunroof in a light-transmitting state is referred to as opening the sunroof.

[0264] FIG. 35 is a diagram illustrating an example of a record in which the "application type" field is a map application in the action definition table of FIG. 2 stored in action definition storage unit 52 of FIG.

[0265] In a state in which a projection image 291 of the presentation information of a map application is projected onto the ceiling 92 and the user 311 is operating the map application, when the user 311 or a user other than the user 311 opens the sunroof, the application type is the map application, the application state is operating, and the trigger is the user's action of opening the sunroof or a change in the sunroof to an open position.

[0266] In this case, the situation corresponds to record R111 in FIG. 35, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R111 in FIG.

[0267] In addition, when user 311 is viewing projected image 291, which is presentation information of a map application, and user 311 or another user opens the sunroof, the application type is map application, the application state is viewing, and the trigger is the user's action of opening the sunroof or a change in the sunroof (sliding panel) from a closed position.

[0268] In this case, the situation corresponds to record R112 in FIG. 35, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R112 in FIG.

[0269] FIG. 36 is a diagram illustrating the state of the vehicle interior and the projected image when record R111 or record R112 in FIG. 35 is executed.

[0270] The actions of records R111 and R112 are the same, and when the action of record R111 or record R112 is executed, the presentation area on which the map application's presentation information was projected moves from the ceiling 92 (sunroof 301) to, for example, the headrest 100, and the map application's presentation information is projected onto the headrest 100 as a projection image 292.

[0271] As a result, even if the sunroof is opened while the user 311 is operating the map application projected on the ceiling 92 or viewing the projected image 291, the user 311 can continue to use the map application.

[0272] Here, the destination presentation part where the presentation information is projected is a presentation part at a position where the user (target person) who has been operating or viewing can see. In Fig. 36, since the user 311 in the back seat is the user who has been operating or viewing, the headrest 99 of the passenger seat where the user 311 can be seen is the destination presentation part.

[0273] For example, the presentation site at a position where the subject can see it includes a normal monitor attached to the dashboard or the like and the front seat door glass for a subject in the front seat, and includes the front seat headrests 99, 100, the rear seat door glass, etc. for a subject in the rear seat. When the presentation information of the app is displayed on a normal monitor, the normal monitor is included as a component of the output unit 24 in FIG. 1.

[0274] In the state of FIG. 34, when the sunroof is opened while all users are not operating a map application or viewing the projection image 291, the application type is the map application, the application state is normal, and the trigger is the user's action of opening the sunroof or a change in the sunroof (sliding panel) from a closed position.

[0275] In this case, the situation corresponds to record R113 in FIG. 35, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R113 in FIG.

[0276] In the action of record R113, the detection processing unit 32 does nothing in response to the sunroof being opened.

[0277] Since no one is using the map app, there is no problem if the map app is temporarily unavailable.

[0278] The actions in the "Action" fields of records R111 to R113 of the action definition table in FIG. 35 can be applied as actions for any type of application and any presentation part (including but not limited to automobiles) whose "Application Status" and "Trigger" fields can be classified into the same combinations as those of records R111 to R113.

[0279] <<Application Example 2 of Information Processing Device 11>> As a second application example of the information processing device 11 in FIG. 1, a case will be described in which the information processing device 11 is applied as a projection device that projects presentation information of various applications (discussion applications) onto a movable presentation part in a room such as a conference room.

[0280] FIG. 37 is a diagram illustrating the arrangement of a table in a room such as a conference room when the table top is used as a movable presentation part.

[0281] Fig. 37A illustrates a pair of table 401 and chair 402 placed in a room. Fig. 37B illustrates a state in which four table tops 403 of table 401 of Fig. 37A are connected and arranged in a 2×2 horizontal×vertical arrangement during a meeting or the like, and each table 401 and chair 402 are represented as tables 401a to 401d and chairs 402a to 402d. In addition, a projector 404 is installed on the ceiling of the room.

[0282] <When the table (top) is used as a movable presentation part> FIG. 38 is a diagram illustrating a state in which presentation information of a discussion application is projected onto the connected top plates of two tables 401a and 401b.

[0283] 38, a top plate 403a represents the top plate of a table 401a, and a top plate 403b represents the top plate of a table 401b. The top plates 403a and 403b face each other with almost no gap between them and are connected vertically in the figure, and the top plates 403a and 403b (their upper surfaces) are used as presentation sites onto which presentation information of a discussion application is projected.

[0284] The projected image 411a and the projected image 411b represent the projected images projected onto the top board 403a and the top board 403b, respectively. When the top board 403a and the top board 403b are connected with almost no gap between them, the projected image 411a and the projected image 411b are projected onto the top board 403a and the top board 403b as one continuous projected image.

[0285] The projection image 411a and the projection image 411b include images of sticky notes 412a to 412f arranged in a background image on one side. The sticky notes 412a to 412f are images on which notes created by the user are written using the functions of a discussion application.

[0286] Furthermore, when the user performs an action of sliding a finger close to any of the sticky notes 412a to 412f, the projection position of the sticky note moves in accordance with the action.

[0287] FIG. 39 is a diagram illustrating an example of a record in which the "application type" field is a discussion application in the action definition table of FIG. 2 stored in action definition storage unit 52 of FIG.

[0288] When a user is in a discussion and places his / her hands on either side of table top 403a of table 401a or table top 403b of table 401b, the application type is a discussion application, the application state is in discussion, and the trigger is the action of the user placing his / her hands on the table.

[0289] In this case, the situation corresponds to record R121 in FIG. 39, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R121 in FIG.

[0290] Regarding the determination of whether the application state is in discussion or after discussion, the graphics display processing unit 31 determines that the application state is after discussion if a user operation to end the discussion has been performed on the discussion application, and determines that the application state is in discussion if a user operation to end the discussion has not been performed.

[0291] Alternatively, when the areas of table tops 403a and 403b are grouped into group A and group B as shown in FIG. 38, if there is a sticky note such as sticky note 412C or sticky note 412D that extends over the areas of both groups, the graphics display processing unit 31 determines that a discussion is in progress because it has not been determined which group the sticky note that extends over the areas of both groups belongs to, and determines that a discussion is in progress if there is no sticky note that extends over the areas of both groups.

[0292] FIG. 40 is a diagram illustrating the state of the table tops 403a and 403b, and the projected images 411a and 411b when the action of record R121 in FIG. 39 is executed.

[0293] When the action of record R121 is executed by the user placing hands 414 on both sides of the tabletop 403b, an image is projected on the projection image 411b with warning information 413 in the form of a text image such as "Discussion in progress. Do not leave the table" or "Do not leave the desk!" superimposed thereon.

[0294] This reminds users not to move the table when a discussion is in progress.

[0295] If the user moves table 401b away from table 401a as shown in FIG. 40 despite being presented with warning information 413, the application type is a discussion application, the application state is in discussion, and the trigger is a change in the position of table 401b (a state change in which the table is moved away).

[0296] In this case, the situation corresponds to record R122 in FIG. 39, so detection processing unit 32 in FIG. 1 causes graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R122 in FIG.

[0297] FIG. 41 is a diagram illustrating the state of table tops 403a and 403b, and projected images 411a and 411b when the action of record R122 in FIG. 39 is executed.

[0298] When the action of record R122 is executed due to tabletop 403b being separated from tabletop 403a, copies of sticky notes 412c, 412d, 412e, and 412f projected onto tabletop 403b of table 401b are projected onto tabletop 403a of table 401a.

[0299] On top plate 403b of table 401b, the entire sticky notes 412e and 412f are projected as they are onto top plate 403b.

[0300] If the sticky notes 412c and 412d, only a portion of which was projected onto the top board 403b, i.e. the sticky notes in the area of ​​the top board 403a are grouped as group A and the sticky notes in the area of ​​the top board 403b are grouped as group B, then the sticky notes 412c and 412d, which cover both the areas of the top board 403a and the top board 403b, are projected only onto the top board 403a as sticky notes of group A, and are not projected onto the top board 403b. However, the sticky notes 412c and 412d may also be projected onto the top board 403b.

[0301] In addition, as an action of record R122 in Figure 39 when table 401b is moved away from table 401a during a discussion, it is possible to project all of the sticky notes onto each of top board 403a of table 401a and top board 403b of table 401b (another example 1 of the action of record R122), or to iconize (reduc) the sticky notes projected onto the top board of the other table and project them (another example 2 of the action of record R122), or to project only the sticky notes projected onto each table (another example 3 of the action of record R122).

[0302] FIG. 42 is a diagram for explaining another example 1 of the action of record R122 in FIG.

[0303] FIG. 42A illustrates a projected image 411a of the table top 403a and a projected image 411b of the table top 403b when the table 401a and the table 401b are connected to each other.

[0304] In A of FIG. 42, tags 412a to 412f are projected onto tabletops 403a and 403b.

[0305] B of Figure 42 shows a projected image 411a of the tabletop 403a and a projected image 411b of the tabletop 403b when another example 1 of the action of record R122 is executed when tables 401a and 401b are separated from the state of A of Figure 42.

[0306] In B of FIG. 42, all of the sticky notes 412a to 412f that were included in the projected image 411a of the top board 403a and the projected image 411b of the top board 403b in A of FIG. 42 are projected onto the top board 403a and the top board 403b, respectively, while maintaining their relative positions to each other.

[0307] FIG. 43 is a diagram for explaining another example 2 of the action of record R122 in FIG.

[0308] A of Fig. 43 illustrates a projected image 411a of the table top 403a and a projected image 411b of the table top 403b when the table 401a and the table 401b are connected to each other. A of Fig. 43 is the same as A of Fig. 42, so a description thereof will be omitted.

[0309] B of Figure 43 shows a projected image 411a of the tabletop 403a and a projected image 411b of the tabletop 403b when another example 2 of the action of record R122 is executed when tables 401a and 401b are separated from the state of A of Figure 42.

[0310] In B of Fig. 43, sticky notes 412a to 412c that were projected onto top board 403a in state A of Fig. 43 are projected onto top board 403a. Note that in the case of a sticky note like sticky note 412a that covers both top boards 403a and 403b in state A of Fig. 43, for example, that sticky note is the one that was projected onto the top board that covers the larger area of ​​each of top boards 403a and 403b.

[0311] Furthermore, onto top plate 403a of table 401a, sticky notes 412ds, 412es, and 412fs, which are iconized (reduced) versions of sticky notes 412d through 412f projected onto top plate 403b in FIG. 42A, are projected.

[0312] Onto top board 403b of table 401b, tags 412d through 412f that were projected onto top board 403b in the state of A in FIG. 43 are projected.

[0313] Furthermore, onto top plate 403b of table 401b, sticky notes 412as, 412bs, and 412cs, which are iconized (reduced) versions of sticky notes 412a to 412c projected onto top plate 403a of table 401a in FIG. 43A, are projected.

[0314] FIG. 44 is a diagram for explaining another example 3 of the action of the record R122 in FIG.

[0315] A of Fig. 44 illustrates a projected image 411a of the table top 403a and a projected image 411b of the table top 403b when the table 401a and the table 401b are connected to each other. A of Fig. 44 is the same as A of Fig. 42, so a description thereof will be omitted.

[0316] B of Figure 44 shows a projected image 411a of the tabletop 403a and a projected image 411b of the tabletop 403b when another example 3 of the action of record R122 is executed when tables 401a and 401b are separated from the state of A of Figure 44.

[0317] In B of FIG. 44, the tags 412a to 412c that were projected onto the top board 403a in the state of A of FIG. 44 are projected onto the top board 403a.

[0318] In addition, in the case of a sticky note such as sticky note 412a that covers the area of ​​both top boards 403a and 403b in state A of FIG. 44, for example, that sticky note is considered to be projected onto the top board 403a or 403b that has the larger enclosed area in each of its areas.

[0319] Furthermore, the tags 412d through 412f that were projected onto the top plate 403b in the state of A in FIG. 44 are projected onto the top plate 403b of the table 401b.

[0320] In addition, when the discussion ends and the user releases table 401a and table 401b, the application type is the discussion application, the application state is after discussion, and the trigger is a change in the position of table 401b (a state change in which the table is released).

[0321] In this case, the situation corresponds to record R123 in FIG. 39, so the detection processing unit 32 in FIG. 1 causes the graphics display processing unit 31 in FIG. 1 to execute the action in the "action" field of record R123 in FIG.

[0322] In the action of record R123, the detection processing unit 32 does nothing in response to the table being released.

[0323] Although not explicitly stated in the action definition table of FIG. 39, when executing any of the actions of records R121 to R123, the detection processing unit 32 of FIG. 1 changes the position of the application window onto which the presentation information is projected in accordance with the movement of the table top, and projects the projection image so that the table top is always the presentation area for the presentation information.

[0324] In addition, the actions in the “Action” fields of records R121 to R123 of the action definition table in FIG. 39 can be applied as actions for any type of application and any presentation part (not limited to tables) whose “Application Status” field and “Trigger” field can be classified into the same combination as records R121 to R123.

[0325] <Other forms> 1 projects presentation information generated by processing of an application or the like onto a movable or immovable presentation part by a projector of the output unit 24. On the other hand, as a component of the output unit 24, a flat panel or a flexible image display (liquid crystal display, organic EL (Organic ElectroLuminescence), etc.) may be provided as the output unit 24 on the surface of a movable or immovable member that can be the movable or immovable presentation part, and the presentation information from the arithmetic processing unit 22 may be displayed on the image display of the presentation part.

[0326] 1, at least the arithmetic processing unit 22 out of the arithmetic processing unit 22 and the storage unit 23 may be a server device communicatively connected to the input unit 21 and the output unit 24 via a communication line such as the Internet. The communication line may be wireless or wired, or may be a mixture of wireless and wired.

[0327] <Program> 1 may be executed by hardware or software. When a series of processes is executed by software, a program constituting the software is installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, capable of executing various functions by installing various programs.

[0328] FIG. 45 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0329] In the computer, a CPU (Central Processing Unit) 601, a ROM (Read Only Memory) 602, and a RAM (Random Access Memory) 603 are interconnected via a bus 604.

[0330] An input / output interface 605 is further connected to the bus 604. An input unit 606, an output unit 607, a storage unit 608, a communication unit 609, and a drive 610 are connected to the input / output interface 605.

[0331] The input unit 606 includes a keyboard, a mouse, a microphone, etc. The output unit 607 includes a display, a speaker, etc. The storage unit 608 includes a hard disk, a non-volatile memory, etc. The communication unit 609 includes a network interface, etc. The drive 610 drives removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0332] In the computer configured as above, the CPU 601 loads, for example, a program stored in the storage unit 608 into the RAM 603 via the input / output interface 605 and the bus 604, and executes the program, thereby performing the series of processes described above.

[0333] The program executed by the computer (CPU 601) can be provided by being recorded on a removable medium 611 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0334] In the computer, the program can be installed in the storage unit 608 via the input / output interface 605 by mounting the removable medium 611 in the drive 610. The program can also be received by the communication unit 609 via a wired or wireless transmission medium and installed in the storage unit 608. Alternatively, the program can be installed in the ROM 602 or the storage unit 608 in advance.

[0335] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0336] The present technology can also be configured as follows. <1> a control unit that controls at least one of a presentation state of the presentation information and a state of the processing based on at least one of presentation part information on a change in a state of the presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part. An information processing device having the above configuration. <2> The presentation part information is information indicating a state change of the presentation part, or information indicating a user's action related to the state change of the presentation part. <1> The information processing device described in <3> The user's action is a user's action of operating an operation unit that moves a member including the presentation portion, or a user's action of manually moving a member including the presentation portion. <2> The information processing device described in <4> The processing information is information indicating the type of the processing and information indicating the status of the processing. <1> ~ <3> 13. An information processing device according to claim 12, <5> Information indicating whether or not the user is viewing the presentation information presented in the presentation area, or information indicating whether or not the user has performed an operation on the process. <1> ~ <4> The information processing device described in <6> The control unit controls the presentation state of the presentation information by controlling the presentation part or the presentation form of the presentation information. <1> ~ <5> 13. An information processing device according to claim 12, <7> The presentation portion is a portion onto which the presentation information is projected. <1> ~ <6> 13. An information processing device according to claim 12, <8> The presentation site is a surface of a movable member. <1> ~ <7> 13. An information processing device according to claim 12, <9> The movable member is a member of an automobile. <8> The information processing device described in <10> The movable member is a table. <8> The information processing device described in <11> When the processing information indicates that the state of the processing is being played back and the presentation part information indicates that a change in the state of the presentation part has occurred, the control unit stops the processing. <1> ~ <9> 13. An information processing device according to claim 12, <12> When the processing information indicates that the state of the processing is being played back at a certain volume or more, the control unit stops the processing. <11> The information processing device described in <13> When the processing information indicates that the state of the processing is being played back and the presentation part information indicates that a user action that causes a change in the state of the presentation part has been performed, the control unit causes the presentation part to present warning information. <1> ~ <9> The information processing device according to claim 1. <14> When the environmental information indicates that the vehicle is moving and the presentation part information indicates that a user action that causes a change in the state of the presentation part has been performed, the control unit causes the presentation part to present warning information. <9> The information processing device described in <15> When the environmental information indicates that the vehicle is moving and the presentation part information indicates that a user action that causes a change in the state of the presentation part has been performed, the control unit locks a member of the presentation part. <9> The information processing device described in <16> The presentation information includes a video, When the processing information indicates that the state of the processing is that the video is being played back and the presentation part information indicates that a change in state of the presentation part has occurred, the control unit changes the presentation part on which the video is presented. <1> ~ <9> 13. An information processing device according to claim 12, <17> When the user information indicates that a user is operating the process, or when the user is viewing the presentation information presented on the presentation part, and when the presentation part information indicates that a state change has occurred on the presentation part, the control unit changes the presentation part on which the presentation information is presented. <1> ~ <9> 13. An information processing device according to claim 12, <18> The presentation information includes a sticky note to be presented in a discussion; When the processing information indicates that the state of the processing is under discussion and the presentation part information indicates that a change in the state of the presentation part has occurred, the control unit changes the position within the range of the presentation part where the sticky note is presented. <10> The information processing device described in <19> Control Unit An information processing device having The control unit: At least one of the presentation state of the presentation information and the state of the processing is controlled based on at least one of presentation part information on a state change of the presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part. Information processing methods. <20> Computer, a control unit that controls at least one of a presentation state of the presentation information and a state of the processing based on at least one of presentation part information on a change in a state of the presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part. A program to function as a [Explanation of symbols]

[0337] 11 information processing device, 21 input section, 23 storage section, 24 output section, 31 graphics display processing section, 32 detection processing section, 41 input information analysis section, 42 application execution section, 43 display control section, 51 action execution section, 52 action definition storage section, 53 urgency definition storage section, 54 application state definition storage section

Claims

1. a control unit that controls at least one of a presentation state of the presentation information and a state of the processing based on at least one of presentation part information on a change in a state of the presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part. having When the presentation part information indicates that a state change has occurred in the presentation part while the presentation information is being presented at least on the presentation part, the control unit changes the presentation part on which the presentation information is presented. Information processing device.

2. The presentation part information is information indicating a state change of the presentation part, or information indicating a user's action related to the state change of the presentation part. The information processing device according to claim 1 .

3. The user's action is a user's action of operating an operation unit that moves a member including the presentation portion, or a user's action of manually moving a member including the presentation portion. The information processing device according to claim 2 .

4. The processing information is information indicating the type of the processing and information indicating the status of the processing. The information processing device according to claim 1 .

5. The user information is information indicating whether or not a user is viewing the presentation information presented in the presentation portion, or information indicating whether or not a user has performed an operation on the process. The information processing device according to claim 1 .

6. The control unit controls the presentation state of the presentation information by controlling the presentation part or the presentation form of the presentation information. The information processing device according to claim 1 .

7. The presentation portion is a portion onto which the presentation information is projected. The information processing device according to claim 1 .

8. The presentation site is a surface of a movable member. The information processing device according to claim 1 .

9. The movable member is a member of an automobile. The information processing device according to claim 8.

10. The movable member is a table. The information processing device according to claim 8.

11. When the processing information indicates that the state of the processing is music or video being played and the presentation part information indicates that a change in the state of the presentation part has occurred, the control unit stops the processing. The information processing device according to claim 1 .

12. When the processing information indicates that the state of the processing is that music is being played at a certain volume or more, the control unit stops the processing. The information processing device according to claim 11.

13. When the processing information indicates that the state of the processing is music being played and the presentation part information indicates that a user action that causes a change in the state of the presentation part has been performed, the control unit causes the presentation part to present warning information. The information processing device according to claim 1 .

14. When the environmental information indicates that the vehicle is moving and the presentation part information indicates that a user action that causes a change in the state of the presentation part has been performed, the control unit causes the presentation part to present warning information. The information processing device according to claim 9.

15. When the environmental information indicates that the vehicle is moving and the presentation part information indicates that a user action that causes a change in the state of the presentation part has been performed, the control unit locks a member of the presentation part. The information processing device according to claim 9.

16. The presentation information includes a video, When the processing information indicates that the state of the processing is that the video is being played back and the presentation part information indicates that a change in state of the presentation part has occurred, the control unit changes the presentation part on which the video is presented. The information processing device according to claim 1 .

17. When the user information indicates that a user is operating the process, or when the user is viewing the presentation information presented on the presentation part, and when the presentation part information indicates that a state change has occurred on the presentation part, the control unit changes the presentation part on which the presentation information is presented. The information processing device according to claim 1 .

18. The presentation information includes a sticky note to be presented in a discussion; When the processing information indicates that the state of the processing is under discussion and the presentation part information indicates that a change in the state of the presentation part has occurred, the control unit changes the position within the range of the presentation part where the sticky note is presented. The information processing device according to claim 10.

19. Control Unit An information processing device having The control unit: controlling at least one of a presentation state of the presentation information and a state of the processing based on at least one of presentation part information on a change in a state of a presentation part where the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part; When the presentation part information indicates that a state change has occurred in the presentation part while the presentation information is being presented at least on the presentation part, the presentation part on which the presentation information is presented is changed. Information processing methods.

20. Computer, a control unit that controls at least one of a presentation state of the presentation information and a state of the processing based on at least one of presentation part information on a change in a state of the presentation part on which the presentation information is presented, processing information on a process for generating the presentation information, user information on a user's response to the processing, and environmental information on an environmental state of the presentation part, and changes the presentation part on which the presentation information is presented when the presentation part information indicates that a change in a state of the presentation part has occurred while the presentation information is presented on at least the presentation part. A program to function as a

Images