Information processing device and method

Abstract

The present disclosure relates to an information processing device and a method which enable a greater variety of communications to be implemented. In a user space, an input of information by the user is received and tag information is generated by using the received information, the tag information allowing control information for controlling the occurrence of an event for a communication counterpart of the user to correspond to the space of the communication counterpart. In addition, it is determined whether an occurrence condition of the event is met, the occurrence condition being designated by the tag information corresponding to the user space and allowing the control information for controlling the occurrence of the event for the communication counterpart to correspond to the space of the communication counterpart, and when the occurrence condition is determined to be met, the event designated by the tag information is made to occur. The present disclosure can be applied to, for example, an information processing device, an image processing device, an electronic apparatus, an information processing method, or a program.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that enable more diverse communication to be realized. [Background technology]

[0002] Conventionally, a telepresence system has been considered that allows users in remote locations to communicate with each other (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-300602 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional telepresence systems only allow users to exchange images and voices with each other, and only allow real-time communication. Therefore, more diverse communication methods are required.

[0005] The present disclosure has been made in light of the above circumstances, and makes it possible to realize more diverse communication. [Means for solving the problem]

[0006] According to one aspect of the present technology, there is provided an information processing device comprising: a receiving unit that receives a first captured image of a space of the communication partner from an information processing device of the communication partner; a display unit that displays the received first captured image in a user's space; an input of control information that controls occurrence of an event for the communication partner; and a designation of an object in the space of the communication partner that is shown in the displayed first captured image and that associates the control information. An input unit for accepting The above input Control information , the specified target a generation unit for generating tag information to be associated with the a tag information transmitting unit that transmits the generated tag information; The information processing device includes:

[0007] An information processing method according to one aspect of the present technology includes: receiving an image of a space of a communication partner from an information processing device of the communication partner, displaying the received image in a user's space, inputting control information for controlling occurrence of an event for the communication partner, associating the control information, and designating an object in the space of the communication partner that is shown in the displayed image; Accept, The input control information is Generate tag information to correspond to and transmitting the generated tag information. It is an information processing method.

[0008] An information processing device according to another aspect of the present technology includes: An imaging unit that images a space of a user and generates a first captured image, a transmission unit that transmits the generated first captured image to an information processing device of a communication partner, a tag information receiving unit that receives tag information that associates control information for controlling the occurrence of an event for the user with an object in the first captured image, and a determination unit that determines whether a condition for the event occurrence is satisfied; The above and an event generating unit that generates the event specified by the tag information when it is determined that the generation condition is satisfied.

[0009] An information processing method according to another aspect of the present technology includes: A first captured image is generated by capturing an image of a user's space, the generated first captured image is transmitted to an information processing device of a communication partner, tag information is received that associates control information for controlling the occurrence of an event for the user with an object in the first captured image, and the control information is specified by the received tag information. The information processing method judges whether a generating condition for the event is satisfied, and when it is judged that the generating condition is satisfied, generates the event specified by the tag information.

[0010] In one aspect of the present technology, an information processing device and method include: An image captured of the communication partner's space is received from the communication partner's information processing device, the received image is displayed in the user's space, input of control information for controlling the occurrence of an event for the communication partner and designation of an object in the communication partner's space shown in the displayed image to which the control information is associated are accepted, tag information is generated that associates the input control information with the designated object, and the generated tag information is transmitted.

[0011] In another aspect of the present technology, an information processing device and a method thereof include: A first captured image is generated by capturing an image of the user's space, the generated first captured image is transmitted to an information processing device of the communication partner, tag information is received that corresponds to an object in the first captured image as control information for controlling the occurrence of an event for the user, it is determined whether or not the occurrence condition of the event specified by the received tag information is satisfied, and if it is determined that the occurrence condition is satisfied, the event specified by the tag information is caused to occur. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an overview of a telepresence system. [Diagram 2] FIG. 1 is a diagram illustrating an overview of a telepresence system. [Diagram 3] FIG. 2 is a diagram for explaining an overview of a tag. [Figure 4] FIG. 11 is a diagram illustrating an example of tag generation. [Diagram 5] FIG. 11 is a diagram illustrating an example of an event occurrence. [Figure 6] 1 is a block diagram showing an example of a main configuration of a telepresence system. [Figure 7] 1 is a block diagram showing an example of the main configuration of a telepresence device; [Figure 8] FIG. 2 is a functional block diagram showing an example of main functions realized by the telepresence device. [Figure 9] FIG. 2 is a diagram showing an example of a main configuration of a tag. [Figure 10] FIG. 2 is a block diagram showing an example of a main configuration of a server. [Figure 11] FIG. 2 is a functional block diagram showing an example of main functions realized by the server. [Figure 12] 13 is a flowchart illustrating an example of the flow of a tag registration process. [Figure 13] 13 is a flowchart illustrating an example of the flow of an event occurrence process. [Figure 14] 13 is a flowchart illustrating an example of the flow of a tag deletion process. [Figure 15] FIG. 13 is a diagram illustrating an example of tag display. [Figure 16] FIG. 13 is a diagram illustrating an example of tag display. [Figure 17] FIG. 11 is a diagram illustrating another example of a telepresence system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. First embodiment (Telepresence system) 2. Second embodiment (other examples) 3. Notes

[0014] <1. First embodiment> <Telepresence system> FIG. 1 is a diagram illustrating an overview of a telepresence system that is an embodiment of a communication system (information processing system) to which the present technology is applied.

[0015] The telepresence system 100 is a communication system for users in remote locations to communicate with each other. The telepresence system 100 has a plurality of telepresence devices as terminal devices. For example, in the case of FIG. 1, the telepresence system 100 has a telepresence device 110A and a telepresence device 110B. The telepresence device 110A and the telepresence device 110B are installed in remote locations.

[0016] In this specification, a remote location refers to a location that is at a different position relative to a certain reference location. The spaces of the multiple locations that are remote from each other are assumed to be independent of each other. For example, the space of each location is assumed to be partially or entirely physically blocked from the spaces of other locations by obstacles such as ceilings, floors, walls, and some objects (including the telepresence device itself), or is assumed to be sufficiently separated from the spaces of other locations. Users in remote locations communicate with each other via the telepresence system 100, and do not communicate directly with voices, gestures, etc.

[0017] The telepresence device 110A includes a touch panel display 111A and a camera 112A. The telepresence device 110A also includes a microphone, a speaker, etc., but these are not shown in the figure. Similarly, the telepresence device 110B includes a touch panel display 111B and a camera 112B. The telepresence device 110B also includes a microphone, a speaker, etc., but these are not shown in the figure.

[0018] The camera 112A captures an image of the space in front of the telepresence device 110A, such as the user 121A of the telepresence device 110A and its surroundings. Similarly, the camera 112B captures an image of the space in front of the telepresence device 110B, such as the user 121B of the telepresence device 110B and its surroundings. The telepresence device 110A and the telepresence device 110B communicate with each other, and exchange data of captured images generated by the camera 112A and data of captured images generated by the camera 112B.

[0019] Then, the touch panel display 111A displays the captured image generated by the camera 112B, i.e., the captured image of the space in front of the telepresence device 110B. The touch panel display 111B displays the captured image generated by the camera 112A, i.e., the captured image of the space in front of the telepresence device 110A.

[0020] Through this communication, the telepresence device 110A and the telepresence device 110B also exchange audio data collected by their respective microphones, and output the audio from their respective speakers.

[0021] In this way, telepresence device 110A and telepresence device 110B exchange images and audio (e.g., images and audio of each user) of the space in which each device is located, allowing user 121A and user 121B, who are located far away from each other, to communicate via this telepresence system.

[0022] In addition, in the touch panel display 111A, a touch panel is superimposed on a display that displays an image. This touch panel detects a position on the display surface of the touch panel display 111A that is touched by the user 121A or the like. In other words, the user 121A can perform an input operation of instructions, information, and the like via the touch panel by touching the display surface of the touch panel display 111A.

[0023] Similarly, the touch panel display 111B has a touch panel superimposed on a display that displays an image. This touch panel detects a position on the display surface of the touch panel display 111B that is touched by the user 121B or the like. In other words, the user 121B can perform an input operation of instructions, information, and the like via the touch panel by touching the display surface of the touch panel display 111B.

[0024] In the following, when it is not necessary to distinguish between the telepresence devices 110A and 110B, they will be referred to as the telepresence devices 110. Similarly, when it is not necessary to distinguish between the touch panel display 111A and the touch panel display 111B, they will be referred to as the touch panel display 111. Furthermore, when it is not necessary to distinguish between the cameras 112A and 112B, they will be referred to as the cameras 112. Furthermore, when it is not necessary to distinguish between the users 121A and 121B, they will be referred to as the users 121.

[0025] The telepresence system 100 enables natural communication between remote users "as if they were in the same space," for example, high-presence communication that is constant and has no particular purpose or time frame, with the aim of improving the quality of relationships between teams or communities made up of users in both spaces.

[0026] While the telepresence devices 110 are connected to each other, an image of the communication partner's space (also referred to as the partner's space) captured by the communication partner's camera 112 is constantly displayed on the touch panel display 111.

[0027] 2, a captured image of space 140 on the user 121B side captured by camera 112B is constantly displayed on touch panel display 111A installed in space 130 on the user 121A side. That is, images of user 121B, object 141, object 142, etc. in space 140 are displayed on touch panel display 111A.

[0028] This makes the user 121A in the space 130 feel as if a space 140' corresponding to the space 140 exists behind the touch-panel display 111A. For example, the user 121A feels as if a user 121B' corresponding to the user 121B, an object 141' corresponding to the object 141, an object 142' corresponding to the object 142, and the like exist behind the touch-panel display 111A.

[0029] Similarly, a captured image of the space 130 on the user 121A side captured by the camera 112A is constantly displayed on the touch panel display 111B installed in the space 140 on the user 121B side. That is, images of the user 121A, object 131, object 132, object 133, etc. in the space 130 are displayed on the touch panel display 111B.

[0030] This gives the user 121B in the space 140 the feeling that a space 130' corresponding to the space 130 exists behind the touch-panel display 111B. For example, the user 121B feels as if a user 121A' corresponding to the user 121A, an object 131' corresponding to the object 131, an object 132' corresponding to the object 132, an object 133' corresponding to the object 133, and the like exist behind the touch-panel display 111B.

[0031] In other words, user 121A and user 121B can feel as if they (i.e., the other communicating user 121) are in the same space (a single space consisting of space 130 and space 140), allowing for more natural communication.

[0032] This can improve the quality of trust and relationships between users 121 in remote locations, and strengthen teams within communities and organizations.

[0033] <Communication through captured images> However, while real-time (instantaneous) communication is easily possible with such mutual exchange of captured images and voices, users at each location must meet face-to-face at the same time to communicate. Therefore, it is difficult to realize non-real-time (non-instantaneous) communication. For example, it is difficult to realize information exchange such as one user leaving a message for another user at a certain time and the other user receiving the message at a later time.

[0034] <Digital sticky notes> Previously, there were "digital sticky notes" that could be used to add arbitrary notes to a 2D desktop area (x, y) by using a mouse, etc., for the purpose of organizing individual tasks and to-do lists. There were also services that allowed multiple users to edit such digital sticky notes. Furthermore, there were also services that allowed users to place digital sticky notes on a virtual whiteboard area and record or recall them.

[0035] However, such digital sticky notes require a common working area for multiple users, and no consideration has been given to a method for implementing such a system in which captured images of each other's spaces are displayed to each other, such as the telepresence system 100.

[0036] There is also a method of placing tags in three-dimensional space using special goggles or gloves, but this not only increases costs, but also creates an unnatural feeling or discomfort for the user due to the need to wear the equipment. There is also a risk that the communication partner will feel visually uncomfortable.

[0037] <Communication using tag information> For this reason, it is possible to communicate using tag information. For example, information input by a user is accepted in the user's space, and tag information is generated by using the accepted information to associate control information for controlling the occurrence of an event for the user's communication partner with the space of the communication partner.

[0038] For example, an information processing device may include an input unit that accepts information input by a user in the user's space, and a generation unit that uses the information accepted by the input unit to generate tag information that associates control information that controls the occurrence of an event for the user's communication partner with the space of the communication partner.

[0039] Also, for example, control information that controls the occurrence of an event for a communication partner is tag information that corresponds to the space of the communication partner, and a determination is made as to whether or not the occurrence condition of the event specified by the tag information associated with the user's space is satisfied, and if it is determined that the occurrence condition is satisfied, the event specified by the tag information is caused to occur.

[0040] For example, in an information processing device, control information for controlling the occurrence of an event for a communication partner is tag information that corresponds to the space of the communication partner, and the information processing device is provided with a judgment unit that judges whether or not the occurrence condition of the event specified by the tag information associated with the user's space is satisfied, and an event generation unit that generates the event specified by the tag information when the judgment unit judges that the occurrence condition is satisfied.

[0041] By doing this, users can enjoy a variety of interpersonal combinations and spontaneous communication through constant connectivity, while easily placing tag information (additional information) on the display boundary surface to efficiently and effectively communicate what they want to say to people they want to meet or talk to.

[0042] For example, in a scene where users in remote locations are constantly connecting their living / office spaces with no specific purpose or time constraints, tag information (additional information) can be set in a natural and simple manner, such as by touching the display or by voice input, for the location where the display is installed in the space, any point (x,y) on the display surface, or any point (x,y,z) or area in both spaces with 3D depth and object placement. In addition, users who receive this information can be guided to take new next actions, which can activate mutual interactions more effectively.

[0043] <Enter tag information> As described above, tag information (also called a tag) is a function that associates control information that controls the occurrence of an event for a communication partner with the space of the communication partner. For example, in the case of A in Fig. 3, the user 121A causes the telepresence device 110 to generate a tag that is associated with the partner's space displayed on the touch panel display 111A, and includes in the tag the control information that controls the occurrence of an event for the communication partner.

[0044] <Specify the target> The user 121 designates an object to which the tag is to be associated (an object to which the tag is to be assigned) from within the other space displayed on the touch-panel display 111. For example, the user 121 designates an object to which the tag is to be associated by operating the touch-panel display 111 (the touch panel of the touch-panel display 111), that is, by touching the touch-panel display 111, or the like.

[0045] The telepresence device 110 receives the designation through a touch panel and generates a tag to be assigned to the designated target. For example, a sensor (such as a touch panel) of the telepresence device 110 recognizes the context (state, situation) information of the user who suddenly wants to assign a message tag, as well as the touch input operation, and selects the desired target from the tagging target candidates in the other party's space.

[0046] This target may be any object within the other space. For example, the target to which the tag is associated may be a predetermined object within the other space. For example, in the case of A in FIG. 3, the user 121A generates tags 151-1 to 151-4 and associates them with the other space. As shown in B in FIG. 3, the tag 151-1 is attached to the touch panel display 111B. The tag 151-2 is attached to the object 141. The tag 151-3 is attached to the user 121B. The tag 151-4 is attached to the object 142. Note that when there is no need to distinguish between the tags 151-1 to 151-4 and explain them, they are referred to as tags 151.

[0047] The tag 151 input and generated by the user 121 is registered in a tag database of the server, which will be described later, as multidimensional information such as spatial coordinates, the creating user, the destination user, and the event occurrence conditions.

[0048] In this way, tag 151 can be attached to any object in the other space. In this case, no matter where the object moves to, tag 151 remains attached to the object. In other words, tag 151 moves with the movement of the object. In addition, tag 151 can also be attached to a moving object, such as user 121B.

[0049] In addition, in the case of an object that can go outside the other space, like this user 121B, the tag 151 may be attached only while the object (user 121B) is located in the other space. For example, when user 121B to which tag 151 is attached goes outside the other space, the tag 151 is removed from user 121B, and an event due to the tag 151 does not occur regardless of the occurrence condition. Then, when user 121B re-enters the other space, the tag 151 is again attached to user 121B, and an event due to the tag 151 can occur. Such control may be performed.

[0050] The object to which the tag 151 is associated may be a predetermined position in the other space. In other words, the tag may be attached to any position in the other space. In this case, even if the object is moved, the tag 151 does not move. In this case, the tag 151 may be attached to a place where no object exists, such as the air.

[0051] As described above, the target to which the tag 151 is to be associated is specified by the user 121 touching the touch panel display 111. In other words, when the user 121 touches a specific position (x, y) on the touch panel display 111, the telepresence device 110 detects the position (x, y) by the touch panel, and generates tag information to be assigned to a position or object in the other person's space that corresponds to the position.

[0052] 4, for example, tag 151-11 is assigned to touch panel display 111B at position (x, y) specified by user 121A. Tag 151-12 is assigned to object 161 displayed on touch panel display 111A at position (x, y) specified by user 121A. Tag 151-13 is assigned to object 162 displayed on touch panel display 111A at position (x, y) specified by user 121A. These tags 151-11 to 151-13 will also be referred to as tag 151 when there is no need to distinguish between them in the description.

[0053] For example, by recognizing the depth of space and object placement using a 3D depth sensor or in-image SLAM (Simultaneous Localization and Mapping) technology, it is possible to select a target position or object in the depth direction. In other words, it is possible to associate and set not only any point (x, y) on the display surface, but also the space behind it (x, y, z) and dynamically changing objects or people.

[0054] The position or depth direction (z) of the object to which the tag 151 is to be attached may be specified according to the length of time that the user 121 keeps a hand or the like in contact with the touch panel display 111. In other words, the telepresence device 110 may measure the time that the user 121 touches the touch panel display 111 when specifying the object to which the tag 151 is to be attached, and may identify the object specified by the user 121 in the depth direction (z) according to the length of that time.

[0055] For example, when an object is specified as a target for tagging with tag 151 as shown in FIG. 4, tag 151-14 is assigned to one of object 162 and object 163 corresponding to the position (x, y), depending on the length of time that user 121A keeps his / her hand or the like in contact with touch panel display 111A.

[0056] For example, if the contact time is equal to or less than a predetermined threshold, object 162 may be selected as the object to which tag 151-14 is to be assigned, and if the contact time is longer than the threshold, object 163 may be selected as the object to which tag 151-14 is to be assigned. Conversely, if the contact time is equal to or less than a predetermined threshold, object 163 may be selected as the object to which tag 151-14 is to be assigned, and if the contact time is longer than the threshold, object 162 may be selected as the object to which tag 151-14 is to be assigned.

[0057] Furthermore, while the user 121A is in contact with the touch panel display 111A, the object to which the tag 151-14 is to be assigned may be switched at predetermined time intervals. Then, the object (either the object 162 or the object 163) selected at the time when the user 121A stops contacting the touch panel display 111 (at the time when the hand or the like that had been in contact with the touch panel display 111A is removed from the touch panel display 111A) may be specified as the object to which the tag 151-14 is to be assigned.

[0058] In addition, when a position is designated as a target to which the tag 151 is to be assigned, the tag 151 may be assigned at a position in the depth direction according to the length of time that the user 121 touches the touch panel display 111 with a hand or the like. For example, the shorter the contact time, the closer the tag 151 is to be assigned, and the longer the contact time, the deeper the tag 151 is to be assigned. Conversely, the longer the contact time, the closer the tag 151 is to be assigned, and the shorter the contact time, the deeper the tag 151 is to be assigned. In addition, while the user 121 is touching the touch panel display 111, the position designated as a target to which the tag 151 is to be assigned may be repeatedly changed, or may be moved back and forth from the front to the back and back to the front.

[0059] <Event occurrence condition specification> This tag information causes an event to occur in the partner space when a predetermined condition is satisfied. The user 121 can specify the event occurrence condition (the trigger for initiating the event). For example, the user 121 specifies the event occurrence condition by touching the touch panel display 111 (the touch panel of the touch panel). The telepresence device 110 accepts the specification via the touch panel and generates a tag including information related to the specified occurrence condition.

[0060] This occurrence condition (trigger) is arbitrary. For example, the occurrence condition may be when the user of the communication partner touches the tag 151 (or the object to which the tag 151 is attached) or when the user of the communication partner is located in the vicinity of the tag 151 (or the object to which the tag 151 is attached). Also, for example, the occurrence condition may be when a specific user appears in the partner's space.

[0061] Also, for example, the occurrence condition may be a condition related to the environment in the communication partner's space, such as a change in environmental information in the partner's space (such as darkness, noise, or a drop in temperature).

[0062] Furthermore, the occurrence condition may be, for example, a condition related to time, for example, a specific time period or time.

[0063] <Event specification> Furthermore, the user 121 can specify an event to be generated by using the tag 151. For example, the user 121 specifies an event to be generated by touching the touch panel display 111 (the touch panel of the touch panel). The telepresence device 110 accepts the specification via the touch panel and generates a tag 151 including information related to the specified event.

[0064] The content of the event is arbitrary. For example, it may be the display of a message (which may include text and images) or the reading out of a message (audio output). It may also be lighting control in the other space (for example, on / off, changing brightness, changing color, etc.) or background music (BGM) control in the other space (for example, starting or stopping BGM output, selecting a song, adjusting the volume, etc.). It may also be air conditioning control in the other space (on / off, changing temperature settings, switching the type of operation (heating, cooling, ventilation, drying, etc.), changing airflow settings, changing airflow settings, etc.). It may also be vibration control of an object (for example, vibration in response to a message, etc.). Of course, multiple events may be combined.

[0065] <Multiple tags> In addition, to select multiple objects (display (x, y), space (x, y, z), object, area, person, etc.) in the other space, for example, a tag 151 may be created by pressing and holding the touched coordinates for a certain period of time, and then the finger may be moved up, down, left, or right to recommend candidate objects, and the finger (touch) may be released when the desired selection is set.

[0066] <How to enter information> Note that any method of input operation for generating such tag 151 may be used. As described above, the user 121 may input information or instructions by operating the touch panel of the touch panel display 111. Also, the user may input information or instructions by operating other input devices (e.g., a keyboard, a mouse, other touch panels, an image sensor, a motion sensor, a button, a switch, a lever, a microphone, etc.) provided in the telepresence device 110.

[0067] <Event occurs> 3A, the tag 151 generated as described above and assigned to the partner space is displayed superimposed on the captured image of the partner space on the touch panel display 111 of the telepresence device 110 that generated the tag 151. In other words, the tag 151 assigned to the partner space can be seen in the user's space (also referred to as the user's own space).

[0068] However, the tag 151 is not generally displayed on the touch panel display 111 of the telepresence device 110 in the other space to which the tag 151 is attached. In other words, the tag 151 is generally not visible in the other space.

[0069] However, as described above, when the occurrence condition set in the tag 151 is satisfied, the event set in the tag 151 occurs. For example, in the case of Fig. 5, when the user 121B touches the object 162 to which the tag 151-13 (not visible to the user 121B) is attached, the event occurrence condition of the tag 151-13 is satisfied, and the event set in the tag 151-13, in this case, a voice reading out the message "Hello!! Tom!", is output from the speaker 171 of the telepresence device 110B.

[0070] The occurrence of an event may be performed by something other than the telepresence device 110. For example, an event may be generated (e.g., reading out a message or information) by a speaker, indicator, actuator, or the like provided on an object to which a tag is to be assigned. Also, for example, the driving control of lighting or air conditioning (on / off, changing settings, etc.) may be an event.

[0071] In this way, by assigning tag 151, an event can be generated based on the generation condition, and non-real-time communication can be realized. For example, when a communication partner user is not present in the partner space, a user in one's own space can assign tag 151 like the example in Fig. 5 to the partner space, and then when the communication partner user enters the partner space and the event generation condition is satisfied, a message can be output. Therefore, more diverse communication can be realized.

[0072] <Tag Visibility> In the example of Fig. 5, as described above, the user 121B does not see the tag 151-13, and generates an event without understanding not only the condition for generating the event, but even the existence of the tag 151-13. In other words, the message is output as voice without the user 121B's intention. In this way, by using the tag 151, it is possible to realize communication with entertainment such as a surprise.

[0073] Note that the user 121 may be able to visually recognize the tag 151 assigned to the user's own space by the communication partner. For example, a projector or the like may be used to display the tag 151 on the object to which the tag 151 is to be assigned. Also, only the tag 151 assigned to the touch panel display 111 may be displayed on the touch panel display 111. For example, in the case of A in Fig. 3, only the tag 151-1 may be displayed on the touch panel display 111B.

[0074] <Characteristics of communication using tag information> As described above, communication using tag 151 can be used in situations where users in remote locations (including users in the same space) are constantly connecting each other's living spaces or office spaces without any specific purpose or time constraints.

[0075] Furthermore, the tag 151 can be set by natural and simple methods such as touch operations on the display and voice input to the location where the display is installed in the space, or to any point (x, y) on the display surface, or to any point (x, y, z) or area within both spaces having 3D depth and object placement.

[0076] Furthermore, the user who receives the information can be guided to take a new next action, thereby activating more effective mutual interaction.

[0077] The telepresence device 110 of the above-mentioned telepresence system 100 can be installed so as to blend into the space (for example, as if it were a window), and operation does not require the specification or installation of special equipment, but can be achieved using only natural methods such as touch operation and voice input.

[0078] <Telepresence system configuration> 6 is a diagram showing an example of a main configuration of the telepresence system 100. As shown in FIG. 6, the telepresence system 100 includes a plurality of telepresence devices 110 and a server 201.

[0079] 6 shows telepresence device 110A and telepresence device 110B, the number of telepresence devices 110 constituting telepresence system 100 is arbitrary as long as it is two or more. In other words, telepresence system 100 may have three or more telepresence devices 110.

[0080] Each telepresence device 110 is installed at each point (remote from each other) where communication is performed. Each telepresence device 110 and a server 201 are connected to each other via a network 202 so as to be able to communicate with each other.

[0081] The server 201 provides a service related to tag information in communication between the telepresence devices 110. More specifically, the server 201 manages the tag information generated in each telepresence device 110.

[0082] Network 202 is a communication network composed of any communication medium. Communication performed via network 202 may be wired communication, wireless communication, or both. In other words, network 202 may be a communication network for wired communication, a communication network for wireless communication, or a communication network composed of both. Furthermore, network 202 may be composed of a single communication network, or may be composed of multiple communication networks.

[0083] For example, the Internet may be included in the network 202. A public telephone line network may be included in the network 202. A wide area communication network for wireless mobile devices, such as a so-called 3G line or 4G line, may be included in the network 202. A wide area network (WAN) or a local area network (LAN) may be included in the network 202. A wireless communication network that performs communication in accordance with the Bluetooth (registered trademark) standard may be included in the network 202. A communication path for short-distance wireless communication, such as NFC (Near Field Communication), may be included in the network 202. A communication path for infrared communication may be included in the network 202. A communication network for wired communication in accordance with standards such as HDMI (registered trademark) (High-Definition Multimedia Interface) or USB (Universal Serial Bus) (registered trademark) may be included in the network 202. In this way, the network 202 may include a communication network or communication path of any communication standard.

[0084] Each telepresence device 110 and server 201 are each connected to a network 202 by wired and / or wireless communication.

[0085] In the following, a description will be given using the output of a message (display, audio output, etc.) as an example of an event.

[0086] <Configuration of telepresence device> Fig. 7 is a block diagram showing an example of the main configuration of the telepresence device 110. In the telepresence device 110 shown in Fig. 7, a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, and a RAM (Random Access Memory) 303 are connected to each other via a bus 304.

[0087] An input / output interface 310 is also connected to the bus 304. To the input / output interface 310, an input unit 311, an output unit 312, a storage unit 313, a communication unit 314, and a drive 315 are connected.

[0088] The input unit 311 may include any input device, such as a keyboard, a mouse, a microphone, a touch panel, an image sensor (e.g., the camera 112), a motion sensor, or any other type of sensor. The input unit 311 may also include an input terminal. The output unit 312 may include any output device, such as a display (e.g., the display 111 with a touch panel), a projector, a speaker (e.g., the speaker 171), or the like. The output unit 312 may also include an output terminal.

[0089] The storage unit 313 includes an arbitrary storage medium, such as a hard disk, a RAM disk, or a non-volatile memory, and a storage control unit that writes and reads information to and from the storage medium. The communication unit 314 includes, for example, a network interface. The drive 315 drives an arbitrary removable medium 321, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and writes and reads information to and from the removable medium.

[0090] In the telepresence device 110 configured as above, the CPU 301 loads a program stored in the storage unit 313, for example, into the RAM 303 via the input / output interface 310 and the bus 304, and executes the program, thereby realizing various functions shown in the functional blocks described below. The RAM 303 also appropriately stores data necessary for the CPU 301 to execute various processes of the program.

[0091] The program executed by the computer can be applied by recording it on, for example, removable medium 321 such as a package medium. In this case, the program can be installed in storage unit 313 via input / output interface 310 by mounting removable medium 321 on drive 315.

[0092] The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 314 and installed in the storage unit 313.

[0093] Alternatively, this program can be pre-installed in the ROM 302 or the storage unit 313 .

[0094] <Functional blocks of telepresence equipment> In the telepresence device 110 described above, the CPU 301 executes a program to realize functions shown in the functional blocks, for example, as shown in Fig. 8. That is, the telepresence device 110 includes a tag input processing unit 351 and a tag output processing unit 352 as functional blocks.

[0095] The tag input processing unit 351 performs processing related to the reception of input of information related to the generation of the tag 151. The tag output processing unit 352 performs processing related to the output of the tag 151, that is, the occurrence of an event indicated by the tag 151.

[0096] The tag input processing unit 351 has a tag generating unit 361 , a tag registering unit 362 , a tag icon output unit 363 , and a tag deleting unit 364 .

[0097] The tag generation unit 361 performs processing related to the generation of the tag 151. The tag registration unit 362 performs processing related to the registration of the tag 151 in the server 201 (tag database 431 described later). The tag icon output unit 363 performs processing related to the output of an icon (a display image indicating the tag 151) of the tag 151. The tag deletion unit 364 performs processing related to the deletion of the registered tag 151 from the server 201 (tag database 431 described later).

[0098] The tag output processing unit 352 includes a condition determination unit 371 , a tag message output unit 372 , and a tag deletion unit 373 .

[0099] The condition determination unit 371 performs processing related to determining a condition for outputting a tag message (i.e., occurrence of an event). The tag message output unit 372 performs processing related to outputting a tag message that is an event specified by the tag 151 (i.e., occurrence of an event). The tag deletion unit 364 performs processing related to deleting the registered tag 151 from the server 201 (tag database 431 described later).

[0100] <Tag data configuration> Fig. 9 is a diagram for explaining an example of a main configuration of data of tag 151 (also referred to as tag data) generated by tag generation unit 361 in Fig. 8. As shown in Fig. 9, tag data 390 is data of tag 151, and includes, for example, ID 391, tag assignment target 392, tag message output condition 393, tag message 394, and tag icon 395.

[0101] The ID 391 is identification information for identifying the tag data 390. For example, the ID 391 may include information on the user who created the tag. The tag assignment target 392 is information for specifying a target to which the tag 151 is assigned. For example, the tag assignment target 392 may include information on a space associated with the tag (or a telepresence device 110 located in that space), identification information of a target object, target coordinates, and the like. The tag message output condition 393 is information for specifying an output condition of a message (also called a tag message) output by the tag 151 as an event. For example, the tag message output condition 393 may include information on a phenomenon, environment, time, and identification information of a user who is a destination of the tag (a target of the event to be generated), which are occurrence conditions. The tag message 394 is information for specifying a tag message output by the tag 151 as an event. For example, the tag message 394 may include text data, audio data, image data, and the like of the tag message. The tag icon 395 is information for specifying a tag icon. For example, the tag icon 395 may include an image of the tag icon. This information is set by the tag generator 361.

[0102] <Server configuration> 10 is a block diagram showing an example of the main configuration of the server 201. In the server 201 shown in FIG.

[0103] An input / output interface 410 is also connected to the bus 404. An input unit 411, an output unit 412, a storage unit 413, a communication unit 414, and a drive 415 are connected to the input / output interface 410. Furthermore, a tag database 431 is connected to the input / output interface 410.

[0104] The input unit 411 may include any input device, such as a keyboard, a mouse, a microphone, a touch panel, a camera (image sensor), a motion sensor, or any other type of sensor. The input unit 411 may also include an input terminal. The output unit 412 may include any output device, such as a display, a projector, a speaker, or the like. The output unit 412 may also include an output terminal.

[0105] The storage unit 413 includes an arbitrary storage medium, such as a hard disk, a RAM disk, or a non-volatile memory, and a storage control unit that writes and reads information to and from the storage medium. The communication unit 414 includes, for example, a network interface. The drive 415 drives an arbitrary removable medium 421, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and writes and reads information to and from the removable medium.

[0106] The tag database 431 stores and manages information related to the tag 151. In the telepresence system 100, the user 121 and the spatial context (situation, state) may be recognized. In this case, the tag database 431 records and manages the information as multidimensional meta information in association with additional information such as the tag 151, and can integrate and play back a plurality of pieces of information as necessary.

[0107] In the server 201 configured as above, the CPU 401 loads, for example, a program stored in the storage unit 413 into the RAM 403 via the input / output interface 410 and the bus 404, and executes the program, thereby realizing various functions shown in the functional blocks described below. The RAM 403 also appropriately stores data necessary for the CPU 401 to execute various processes of the program.

[0108] The program executed by the computer can be applied by recording it on a removable medium 421 such as a package medium, for example. In this case, the program can be installed in the storage unit 413 via the input / output interface 410 by mounting the removable medium 421 on the drive 415.

[0109] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 414 and installed in the storage unit 413.

[0110] Alternatively, this program can be pre-installed in the ROM 402 or the storage unit 413 .

[0111] <Server function block> In the above-described server 201, the CPU 401 executes a program to realize functions shown in the functional blocks, for example, as shown in Fig. 11. That is, the server 201 includes, as functional blocks, a tag management unit 451, a tag acquisition unit 452, a tag supply unit 453, and a tag deletion request acquisition unit 454.

[0112] The tag management unit 451 uses the tag database 431 to manage tags generated in the telepresence device 110. The tag acquisition unit 452 performs processing related to acquisition of the tag 151 supplied from the telepresence device 110. The tag supply unit 453 performs processing related to supply of the tag 151 to the telepresence device 110. The tag deletion request acquisition unit 454 performs processing related to acquisition of a deletion request for the tag 151 supplied from the telepresence device 110.

[0113] <Process flow for tag generation and registration> Next, an example of the processing flow from generating the tag 151 (tag data 390) to be assigned to the partner space in the telepresence device 110 to registering the tag 151 in the tag database 431 of the server 201 will be described with reference to the flowchart of FIG.

[0114] For example, user 121 can display an area in which to write a tag message by touching the display surface of touch panel display 111 (the other space reflected within the angle of view of camera 112), and write the tag message in that area by voice input, etc.

[0115] When the process starts, the tag generation unit 361 of the telepresence device 110 sets a target to which the tag 151 is to be assigned in step S101. For example, the user 121 inputs a designation of a target to which the tag 151 is to be assigned by touching with a finger a position on the touch panel display 111 displaying the other space, the position corresponding to the position or object to which the tag 151 is to be assigned in the other space. The tag generation unit 361 accepts the designation via the input unit 311, and sets the designated position or object as a target to which the tag 151 is to be assigned. As a result, for example, a tag assignment target 392 (FIG. 9) of the tag data 390 is generated.

[0116] In step S102, the tag generation unit 361 sets the output conditions of the tag message. For example, the user 121 operates the touch panel display 111 or inputs voice to specify the output conditions of the tag message. The tag generation unit 361 accepts the specification via the input unit 311, and sets the output conditions of the tag message based on the specification. As a result, for example, the tag message output conditions 393 (FIG. 9) of the tag data 390 are generated.

[0117] In step S103, the tag generation unit 361 sets a tag message. For example, the user 121 operates the touch panel display 111 or inputs a voice to specify a tag message as an event. The tag generation unit 361 accepts the specification via the input unit 311 and sets the tag message based on the specification. As a result, for example, a tag message 394 (FIG. 9) of the tag data 390 is generated.

[0118] In step S104, the tag generating unit 361 sets a tag icon. This tag icon is, for example, an image indicating the tag 151 assigned to the partner space, displayed on the touch panel display 111 on the side (own space) where the tag 151 was assigned. A predetermined image may be set as this tag icon, or the user 121 or the like may be allowed to specify the image of this tag icon. In that case, for example, the user 121 operates the touch panel display 111 or the like to specify an image to be used as the tag icon. The tag generating unit 361 accepts the specification via the input unit 311, and sets a tag message based on the specification. As a result, for example, the tag icon 395 (FIG. 9) of the tag data 390 is generated.

[0119] In step S105, the tag generating unit 361 sets identification information (ID) for identifying the tag data 390 composed of the above various information. As a result, for example, the ID 391 of the tag data 390 is generated.

[0120] In step S106, the tag registration unit 362 transmits the tag data 390 generated by the tag generation unit 361 as described above to the server 201 via the communication unit 314, and causes the data to be registered in the tag database 431.

[0121] The tag acquisition unit 452 of the server 201 receives the tag data 390 transmitted from the telepresence device 110 via the communication unit 414 in step S121.

[0122] In step S122, the tag management unit 451 registers the tag data 390 (that is, the tag 151) received in step S121 in the tag database 431 and manages it.

[0123] Also, in step S107, the tag icon output unit 363 causes the tag icon of the tag 151 registered in the tag database 431 as described above to be displayed on the output unit 312 (the touch panel display 111 or the like).

[0124] By executing each process as described above, the user can easily generate and register the tag 151 by performing general UI (User Interface) operations without needing any special tools or the like.

[0125] <Process flow for tag output> Next, an example of a process flow relating to control of a tag in one's own space assigned by a communication partner (occurrence of an event, etc.) will be described with reference to the flowchart of FIG.

[0126] When the process starts, in step S141, the tag management unit 451 of the server 201 reads out the tag 151 (tag data 390) assigned to the space of the telepresence device 110 to be processed from the tag database 431. Then, the tag supply unit 453 transmits information other than the tag message 394 included in the read tag data 390 to the telepresence device 110 to be processed via the communication unit 414.

[0127] The condition determination unit 371 of the telepresence device 110 receives the tag data 390 (other than the tag message 394) transmitted from the server 201 via the communication unit 314 in step S131.

[0128] In step S132, the condition determination unit 371 refers to the tag message output condition 393 of the received tag data 390, and waits until the output condition is satisfied. If it is determined that the output condition is satisfied, the process proceeds to the next step.

[0129] In step S 133 , the condition determination unit 371 requests the tag message 394 of the tag data 390 from the server 201 via the communication unit 314 .

[0130] In step S142, the tag management unit 451 receives the request via the communication unit 414. Based on the request, the tag management unit 451 reads the tag message 394 of the requested tag data 390 from the tag database 431, and transmits the tag message 394 via the communication unit 414 to the telepresence device 110 that is the request source.

[0131] In step S 134 , the tag message output unit 372 receives the tag message 394 transmitted from the server 201 via the communication unit 314 .

[0132] In step S135, the tag message output unit 372 outputs the tag message via an output device of the output unit 312 (for example, the touch panel display 111 or the speaker 171).

[0133] In step S136, the tag deletion unit 373 requests the server 201 via the communication unit 314 to delete the tag 151 to which the tag message has been output.

[0134] In step S144, the tag deletion request acquisition unit 454 of the server 201 receives the deletion request via the communication unit 414.

[0135] In step S 145 , the tag management unit 451 deletes the tag data 390 of the requested tag 151 from the tag database 431 .

[0136] By executing each of these processes, it is possible to output a tag message based on the generation condition, thereby realizing a wider variety of communication.

[0137] <Process flow for deleting tags> Next, an example of the flow of processing related to deleting the tag 151 will be described with reference to the flowchart of FIG.

[0138] When deleting a tag that has been set, the user 121 inputs an instruction to delete the tag 151, for example, by operating the touch panel display 111. In step S151, the tag deletion unit 364 of the telepresence device 110 accepts, via the input unit 311, a designation of the tag 151 to be deleted (which tag 151 to delete).

[0139] When the designation of the tag 151 to be deleted is accepted, the tag deletion unit 364 requests the server 201 via the communication unit 314 to delete the designated tag 151 in step S152.

[0140] The tag deletion request acquisition unit 454 of the server 201 receives the request via the communication unit 414 in step S161.

[0141] Based on the request, the tag management unit 451 deletes the tag 151 for which deletion has been requested from the tag database 431 .

[0142] By performing each process as described above, the user can easily delete the tag 151 through a general UI operation without needing a special tool or the like.

[0143] <2. Second embodiment> <Visualization of tags in your own space> <Overlay display> The tag associated with the user's own space by the communication partner, etc. may be made visible (visualized). For example, the touch panel display 111 may display an image of the user's own space to which a tag has been added in a semi-transparent state, superimposed on an image of the other user's space.

[0144] Fig. 15 is a diagram for explaining a display example in this case. In Fig. 15, an image of the other space 510 is displayed on the touch panel display 111A. From this image, the user 121A can visually recognize that in the other space 510, the tag 521 is assigned to the object 511, the tag 522 is assigned to the communication partner user 121B, and the tag 523 is assigned to the object 512.

[0145] Further, on the touch panel display 111, an image of the own space 530 is displayed in a semi-transparent state, superimposed on the image of the other space 520. This image allows the user 121A to visually recognize that, in the own space 530, a tag 541 is assigned to the object 531, a tag 542 is assigned to the object 532, a tag 543 is assigned to the user 121A himself, and a tag 544 is assigned to the object 533.

[0146] In this way, by making the image of the own space semi-transparent, the image of the own space can be viewed while viewing the image of the other space. Also, since the image of the own space and the image of the other space can be displayed superimposed on one display, the increase in cost can be suppressed compared to the case where multiple displays are used. Also, the image of the own space can be displayed larger than when the display area is divided and displayed.

[0147] Also, in that case, the image of the user's own space may be the captured image with tags synthesized thereon, or may be the captured image with the tags synthesized thereon and then flipped horizontally. By flipping the captured image horizontally and displaying it, the user 121 can visually recognize the image of the user's own space (the user 121 himself / herself and the area behind him / her) in a similar feeling to looking at a "mirror", and the sense of discomfort can be suppressed.

[0148] <Non-overlapping display> Of course, the image of the user's own space may be displayed without overlapping with the image of the other party's space. For example, the image of the user's own space may be displayed on a display different from the display on which the image of the other party's space is displayed. Also, the image of the user's own space and the image of the other party's space may be displayed in different display areas of the display. Furthermore, the display of the display may be time-divided so that the image of the user's own space and the image of the other party's space are alternately displayed.

[0149] <AR display> For example, the tag associated with the user's own space by the communication partner may be superimposed and displayed on the actual user's own space. For example, using the technology of AR (Augmented Reality) augmented reality, the tag associated with the user's own space by the communication partner may be superimposed and displayed on the object in the actual user's own space to which the tag is assigned.

[0150] FIG. 16 is a diagram for explaining a display example in that case. As shown in FIG. 16, in the user's own space, a tag 541 is assigned to an object 531. Also, a tag 542 is assigned to an object 532. Furthermore, a tag 543 is assigned to the user 121A himself / herself. Also, a tag 544 is assigned to an object 533. In the user's own space, these tags are displayed using AR technology and cannot be visually recognized without using AR technology.

[0151] The user 121A wears AR glasses 561 that support AR, and sees his / her own space, etc., through the AR glasses 561. The AR glasses 561 use AR technology to display tags associated with the user's own space by a communication partner, superimposed on the target of the tag in the actual user's own space. Therefore, the user 121A can visually recognize the tags (tags 541 to 544) that have been assigned to the user's own space through the AR glasses 561.

[0152] In this way, the tag can be displayed only to those who are wearing the AR glasses. Using this technology, for example, it is possible to control the users who can view the tag. In other words, it is possible to display the tag only to specific users, or to limit the tags that can be viewed by each user. For example, the tag may be visualized only to the user who is the destination of the tag.

[0153] <Other examples of telepresence devices> The specifications of the telepresence device are arbitrary and are not limited to the above example (the telepresence device 110). For example, the telepresence device may be a device that shares a desktop area (work area) as shown in FIG.

[0154] 17 includes a display 711A that displays an image of the other person's space, a desktop area 712A that is shared with the communication partner, and a camera 713A that captures the user's own space. Similarly, a telepresence device 710B includes a display 711B that displays an image of the other person's space, a desktop area 712B that is shared with the communication partner, and a camera 713B that captures the user's own space.

[0155] In the following, when it is not necessary to distinguish between the telepresence device 710A and the telepresence device 710B, they will be referred to as the telepresence device 710. When it is not necessary to distinguish between the display 711A and the display 711B, they will be referred to as the display 711. When it is not necessary to distinguish between the desktop area 712A and the desktop area 712B, they will be referred to as the desktop area 712. When it is not necessary to distinguish between the camera 713A and the camera 713B, they will be referred to as the camera 713.

[0156] The telepresence device 710A and the telepresence device 710B have a communication function similar to the telepresence device 110, and can communicate with a communication partner. An image of the other party's space captured by the camera 713B is displayed on the display 711A. An image of the other party's space captured by the camera 713A is displayed on the display 711B.

[0157] Moreover, the desktop area 712 is capable of image display and image projection. Furthermore, the desktop area 712 detects operations by the user 121, etc., using various sensors. In other words, the desktop area 712 is an area in which input and output are possible, and essentially functions as an input device and an output device.

[0158] Moreover, the information of the desktop area 712 is shared with the telepresence device 710 of the communication partner. That is, for example, the result of an input made in the desktop area 712A is not only displayed in the desktop area 712A but is also reflected in the desktop area 712B, and vice versa. This allows the user 121A and the user 121B to easily perform a collaborative work using the desktop area 712.

[0159] In the telepresence system 700 using such a telepresence device 710, communication using tags may be possible, similarly to the telepresence system 100 described above.

[0160] <Tag management by telepresence devices> In the above, it has been described that the tag management is performed in the server 201, but the tag management may be performed in the telepresence device. In other words, the telepresence device may be provided with the tag database 431, and the telepresence device may realize the function of the tag management unit 451.

[0161] <3. Notes> <Target of application of this technology> The present technology may be applied to any configuration.

[0162] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.

[0163] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0164] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0165] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.

[0166] The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.

[0167] <Other> The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0168] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0169] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0170] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0171] Also, for example, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel, or individually at a required timing such as when a call is made. In other words, the processing of each step may be executed in an order different from the order described above, so long as no contradiction occurs. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0172] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.

[0173] The present technology can also be configured as follows. (1) an input unit that accepts information input by a user in a space of the user; a generation unit that generates tag information that associates control information for controlling occurrence of an event for a communication partner of the user with a space of the communication partner, using the information received by the input unit; An information processing device comprising: (2) the input unit accepts a designation of an object to which the tag information is to be associated; The generation unit generates the tag information associated with the object received by the input unit. An information processing device as described in (1). (3) The target is a predetermined location in the space of the communication partner. An information processing device according to (2). (4) The target is a specific object in the communication partner's space. An information processing device according to (2). (5) The input unit a display that displays the communication partner's space within the user's space; a touch panel that is superimposed on the display and detects a position that the user touches to specify the target in the space of the communication partner displayed on the display; Equipped with The generation unit generates the tag information associated with the object in the space of the communication partner corresponding to the position touched by the user detected by the touch panel. An information processing device according to (2). (6) The input unit identifies a position in a depth direction in a space of the target communication partner designated by the user according to a length of time the user touches the touch panel. (5) An information processing device according to the present invention. (7) the input unit accepts a specification of a condition for the event to occur, The generation unit generates the tag information that associates the control information including the generation condition received by the input unit with a space of the communication partner. An information processing device as described in (1). (8) the input unit accepts a designation of the event; The generation unit generates the tag information that associates the control information including the designation of the event accepted by the input unit with a space of the communication partner. An information processing device as described in (1). (9) an imaging unit that captures an image of the user's space and generates a captured image of the user's space; a display unit that displays a captured image of the communication partner's space within the user's space; a communication unit that supplies a captured image of the user's space generated by the imaging unit and acquires a captured image of the space of the communication partner; The information processing device according to (1) further comprises: (10) accepting input of information by a user in the user's space; Using the received information, tag information is generated that associates control information for controlling the occurrence of an event with the communication partner of the user with the space of the communication partner. Information processing methods.

[0174] (11) A determination unit that determines whether or not a condition for generating the event is satisfied, the condition being specified by tag information that corresponds to a space of a user, and that corresponds to control information for controlling generation of an event for a communication partner, the tag information being associated with the space of the communication partner; and an event generating unit that generates the event specified by the tag information when the determining unit determines that the generating condition is satisfied; An information processing device comprising: (12) The occurrence condition is that the communication partner is located near a location associated with the tag, or that the communication partner comes into contact with an object associated with the tag. An information processing device according to (11). (13) The occurrence condition is a condition related to an environment in the space of the communication partner. An information processing device according to (11). (14) The occurrence condition is a time-related condition. An information processing device according to (11). (15) The event is displaying a message to the communication partner or outputting a voice. An information processing device according to (11). (16) An imaging unit that captures an image of the user's space and generates a captured image of the user's space; a display unit that displays a captured image of the communication partner's space within the user's space; a communication unit that supplies a captured image of the user's space generated by the imaging unit and acquires a captured image of the space of the communication partner; The information processing device according to (11) further comprising: (17) The display unit further displays the tag information associated with the space of the user by the communication partner. An information processing device according to (16). (18) The display unit displays the tag information associated with the user's space by superimposing it on a captured image of the communication partner's space. An information processing device according to (17). (19) The present invention further includes an augmented reality display unit that displays the tag information associated with the user's space by the communication partner in a superimposed manner on the user's space. An information processing device according to (16). (20) determining whether or not a condition for generating the event is satisfied, the condition being specified by tag information that associates control information for controlling the generation of an event for a communication partner with a space of the communication partner and that is associated with the user's space; When it is determined that the occurrence condition is satisfied, the event designated by the tag information is caused to occur. Information processing methods. [Explanation of symbols]

[0175] 100 Telepresence system, 110 Telepresence device, 111 Display with touch panel, 112 Camera, 121 User, 130 Space, 131 to 133 Object, 140 Space, 141 and 142 Object, 151 Tag, 161 to 163 Object, 171 Speaker, 201 Server, 202 Network, 351 Tag input processing unit, 352 Tag output processing unit, 361 Tag generation unit, 362 Tag registration unit, 363 Tag icon output unit, 364 Tag deletion unit, 371 Condition determination unit, 372 Tag message output unit, 373 Tag deletion unit, 390 Tag data, 391 ID, 392 Tag assignment target, 393 Tag message output condition, 394 Tag message, 395 Tag icon, 431 Tag database, 451 Tag management unit, 452 Tag acquisition unit, 453 Tag supply unit, 454 Tag deletion request acquisition unit, 510 Partner space, 511 and 512 Object, 521 to 523 Tag, 530 Self space, 531 to 533 Object, 541 to 544 Tag, 561 AR glasses, 700 Telepresence system, 710 Telepresence device, 711 Display, 712 Desktop area, 713 Camera

Claims

1. A receiving unit that receives a first captured image of a space of a communication partner from an information processing device of the communication partner; a display unit that displays the received first captured image in a user's space; an input unit that receives an input of control information for controlling an event occurring with respect to the communication partner, and a designation of an object in a space of the communication partner shown in the displayed first captured image, the object being associated with the control information; a generating unit that generates tag information that associates the input control information with the specified target; a tag information transmitting unit that transmits the generated tag information; An information processing device comprising:

2. The target is a predetermined location within the space of the communication partner. The information processing device according to claim 1 .

3. The target is a predetermined object in the communication partner's space. The information processing device according to claim 1 .

4. The input unit has a touch panel superimposed on the display unit, and receives a position touched by the user detected by the touch panel as the designation of the target; The generation unit generates tag information that associates the control information with the target shown at a position touched by the user in the first captured image. The information processing device according to claim 1 .

5. The input unit specifies a position of the target communication partner in a depth direction in a space according to a length of time that the user touches the touch panel. The information processing device according to claim 4.

6. The input unit receives a specification of a condition for the event to occur, The generating unit generates the tag information including the received occurrence condition. The information processing device according to claim 1 .

7. The input unit accepts a designation of the event; The generating unit generates the tag information including a designation of the accepted event. The information processing device according to claim 1 .

8. An imaging unit that captures an image of the user's space and generates a second captured image; an image transmission unit that transmits the generated second captured image to the information processing device of the communication partner; The information processing device according to claim 1 , further comprising:

9. The display unit further displays the second captured image. The information processing device according to claim 8.

10. Receiving an image of a space of a communication partner from an information processing device of the communication partner; Displaying the received captured image in a user's space; receiving an input of control information for controlling an occurrence of an event for the communication partner, and a designation of an object in the space of the communication partner shown in the displayed captured image to which the control information is associated; generating tag information that associates the input control information with the specified target; Transmitting the generated tag information Information processing methods.

11. An imaging unit that captures an image of a user's space and generates a first captured image; a transmission unit that transmits the generated first captured image to an information processing device of a communication partner; a tag information receiving unit that receives tag information that associates control information for controlling occurrence of an event for the user with an object in the first captured image; a determination unit that determines whether a condition for generating the event specified by the received tag information is satisfied; an event generating unit that generates the event specified by the tag information when it is determined that the generation condition is satisfied; An information processing device comprising:

12. The occurrence condition is that the user is located in the vicinity of the target or in contact with the target. The information processing device according to claim 11.

13. The occurrence condition is a condition related to the environment in the user's space. The information processing device according to claim 11.

14. The occurrence condition is a time-related condition. The information processing device according to claim 11.

15. The event is the display of a message to the user or the output of a voice. The information processing device according to claim 11.

16. An image receiving unit that receives a second captured image of the space of the communication partner from an information processing device of the communication partner; a display unit that displays the received second captured image in the user's space; The information processing device according to claim 11 , further comprising:

17. The display unit further displays the received tag information. The information processing device according to claim 16.

18. The display unit displays the received tag information by superimposing it on the second captured image. The information processing device according to claim 17.

19. The present invention further includes an augmented reality display unit that displays the received tag information by superimposing it on the user's space. The information processing device according to claim 11.

20. Imaging a space of a user to generate a first captured image; Transmitting the generated first captured image to an information processing device of a communication partner; receiving tag information that associates control information for controlling occurrence of an event for the user with an object in the first captured image; determining whether or not a condition for generating the event specified by the received tag information is satisfied; When it is determined that the occurrence condition is satisfied, the event designated by the tag information is caused to occur. Information processing methods.

Images