Information processing device, method, and program

Abstract

The present technology relates to an information processing device, method, and program that make it possible to easily acquire information in a virtual space. The information processing device updates a sensing object space position in accordance with a change in a tracking target object space position so as to maintain a positional relationship between a tracking target object space position, which is a spatial position of a tracking target object in a virtual space, and a sensing object space position, which is a spatial position of a sensing object for sensing a sensing range set for the tracking target object in the virtual space, and generates sensing result information, which is the result of sensing of the sensing range by the sensing object at the sensing object space position. The present technology can be applied to a virtual space sensing system.

Description

Information processing device, method, and program

[0001] The present technology relates to an information processing device, a method, and a program, and more particularly to an information processing device, a method, and a program that enable easy acquisition of information in a virtual space.

[0002] Conventionally, it is known to photograph a virtual space with a virtual camera (see Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2022-184958

[0004] However, the shooting range in the virtual space needs to be set by a user operation each time a photograph is taken.

[0005] The present technology has been developed in light of these circumstances, and makes it possible to easily obtain information within a virtual space.

[0006] An information processing device according to one aspect of the present technology includes: a sensing object spatial position update unit that updates the tracking target object spatial position in response to a change in the tracking target object spatial position so that a positional relationship between the tracking target object spatial position, which is the spatial position of the tracked object in a virtual space, and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set for the tracked object in the virtual space, is maintained; and a sensing result information generation unit that generates sensing result information that is a result of sensing the sensing range by the sensing object at the sensing object spatial position.

[0007] In one aspect of the present technology, a tracking target object spatial position, which is the spatial position of the tracked object in a virtual space, and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set based on the tracked object in the virtual space, are updated in response to a change in the tracking target object spatial position so as to maintain a positional relationship between the tracking target object spatial position and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set based on the tracked object in the virtual space. Then, sensing result information, which is a sensing result of the sensing range by the sensing object at the tracking target object spatial position, is generated.

[0008] 6 is a diagram illustrating an example of virtual space sensing according to an embodiment of the present technology. FIG. 7 is a diagram illustrating another example of virtual space sensing according to an embodiment of the present technology. FIG. 8 is a block diagram illustrating a configuration example of a virtual space sensing system according to a first embodiment of the present technology. FIG. 9 is a diagram illustrating an example of a registration UI. FIG. 10 is a diagram illustrating another example of the registration UI of FIG. 4. FIG. 11 is a diagram illustrating an example display of a sensing object detail setting screen of the registration UI. FIG. 12 is a diagram illustrating another display example of the sensing object detail setting screen of FIG. 6. FIG. 13 is a diagram illustrating another display example of a sensing range setting area. FIG. 14 is a diagram illustrating an example of object information. FIG. 15 is a diagram illustrating an example of sensing object information. FIG. 16 is a diagram illustrating a relative shooting position. FIG. 17 is a diagram illustrating a sensing range. FIG. 18 is a diagram illustrating another example of a sensing object detail setting screen. FIG. 19 is a diagram illustrating a shooting direction. FIG. 19 is a diagram illustrating an example of a sensing range when a sensing object has a camera function. FIG. 19 is a diagram illustrating another example of a sensing range when a sensing object has a camera function. FIG. 19 is a diagram illustrating another example of sensing object information. FIG. 20 is a flowchart illustrating processing of the virtual space sensing system of FIG. 3. FIG. 21 is a diagram illustrating a configuration example of a virtual space sensing system according to a second embodiment of the present technology. FIG. 22 is a diagram illustrating an example of a service provided by the present technology. FIG. 23 is a block diagram illustrating an example of a computer.

[0009] Hereinafter, embodiments of the present technology will be described. The description will be made in the following order: 1. Overview 2. First embodiment (system alone) 3. Second embodiment (collaboration with external system) 4. Other

[0010] 1. Overview FIG. 1 is a diagram illustrating an example of virtual space sensing according to an embodiment of the present technology.

[0011] FIG. 1 shows an example in which a sensing object 11a in a virtual space 1 has a camera function and captures an image of a whole body 13a of a tracking target object 12a from the front.

[0012] The tracking target object 12a is an object that is the target of tracking by the sensing object 11a. In the example of Fig. 1, the tracking target object 12a is an avatar, but it may be a person other than an avatar (NPC: Non-player character), an animal, a plant, a product, a background, etc. in the virtual space 1.

[0013] The sensing object 11a has a sensing function of acquiring representations in the three-dimensional virtual space 1 from the virtual space 1. The representations are, for example, images or sounds of objects. The sensing object 11a is set in correspondence with the tracked object 12a. Note that the sensing object 11a only needs to be functionally realized by software, and may or may not have an actual entity in the virtual space 1 (display an actual entity in the virtual space 1).

[0014] 1 has a camera function as one of its sensing functions. When the sensing object 11a is set to sense the entire body 13a of the tracked object 12a from directly in front of the tracked object 12a, the sensing object 11a moves to an appropriate spatial position according to the spatial position of the tracked object 12a, as shown by the dotted arrow, to maintain a spatial positional relationship with the tracked object 12a, and senses (photographs) the entire body 13a from directly in front of the tracked object 12a. Note that, hereinafter, the three-dimensional spatial position in the virtual space 1 will be simply referred to as a position, and the spatial positional relationship will be simply referred to as a positional relationship.

[0015] 1, since the sensing object 11a has a camera function, the entire body of the tracked object 12a is the sensing range, and processing is performed to acquire an image as a sensing result of the sensing range as virtual space sensing. Note that the sensing range is not limited to the entire body of the tracked object 12a, and it is also possible to set a part of the tracked object, the field of view range as seen from the tracked object, etc.

[0016] Then, services using images obtained as a result of sensing using virtual space sensing, analysis results of the images, or information based on the images or analysis results are provided (feedback) to the virtual space user terminal used by users of virtual space 1.

[0017] FIG. 2 is a diagram illustrating another example of virtual space sensing according to the embodiment of the present technology.

[0018] 2 shows an example in which a sensing object 11b has a microphone function and acquires sound around a tracked object 12b, i.e., within a certain spherical range 13b centered on the vicinity, in a three-dimensional virtual space 1. The vicinity may be any position within the certain range 13b that includes the position of the tracked object 12b, or the tracked object 12b may be set as the center of the sphere.

[0019] 2, the sensing object 11b has a microphone function among its sensing functions. Note that, like the sensing object 11a, the sensing object 11b only needs to be functionally realized by software, and may or may not have an actual entity in the virtual space 1 (i.e., its actual entity is displayed in the virtual space 1).

[0020] When the sensing object 11b is set to sense sound within a certain spherical range 13b centered near the tracked object 12b, it moves to an appropriate position according to the position of the tracked object 12b, as shown by the dotted arrow, maintains its positional relationship with the tracked object 12b, and senses sound within a certain spherical range 13b centered near the tracked object 12b.

[0021] That is, in the case of Figure 2, the sensing object 11b has a microphone function, so the sensing range is a certain spherical range 13b centered near the tracked object 12b, and processing is performed to acquire sound as the sensing result of the sensing range as virtual space sensing.

[0022] Services using the results of sensing by virtual space sensing, such as voice, analysis results of the voice, or information based on the voice or analysis results, are provided (feedback) to the virtual space user terminal. Hereinafter, when there is no need to distinguish between sensing objects 11a and 11b, they will be referred to as sensing objects 11. When there is no need to distinguish between tracked objects 12a and 12b, they will be referred to as tracked objects 12.

[0023] 2. First Embodiment (System Standalone) System Configuration FIG. 3 is a diagram illustrating an example of the configuration of a virtual space sensing system according to a first embodiment of the present technology.

[0024] The virtual space sensing system 51 in Figure 3 is a system that realizes the virtual space sensing described above in Figures 1 and 2, and provides services to virtual space user terminals using virtual space sensing result information (images, audio, etc.), analysis result information obtained by analyzing the virtual space sensing result information, or information based on the virtual space sensing result information and analysis result information.

[0025] The virtual space sensing system 51 is configured to include a virtual space construction unit 60 , a virtual space object analysis server 61 , a virtual space user terminal 62 , and an object information registrant terminal 63 .

[0026] In Figure 3, the virtual space system is composed of a virtual space construction unit 60 and an information storage unit 72 in a virtual space object analysis server 61, and the virtual space 1 of Figures 1 and 2 is provided to a virtual space user terminal 62.

[0027] 3, the virtual space sensing system 51 and the virtual space system are managed by the same provider (administrator), but are different systems. That is, the virtual space sensing system 51 in FIG. 3 can provide sensing results or analysis results of the virtual space 1 to an external party. Note that the virtual space sensing system 51 and the virtual space system may be managed by different providers with which they have a business partnership, for example.

[0028] The virtual space construction unit 60 acquires virtual space construction information (e.g., object three-dimensional (3D) data, spatial positions, etc.), which is information necessary for constructing the virtual space 1, from the information storage unit 72, and constructs the three-dimensional virtual space 1. The information on the constructed virtual space 1 (hereinafter referred to as virtual space information) is output to the virtual space user terminal 62 and provided to the virtual space user terminal 62. The virtual space construction unit 60 also outputs the virtual space information to an object information registration unit 73 of the virtual space object analysis server 61, and outputs the virtual space construction information to a sensing unit 74 of the virtual space object analysis server 61.

[0029] The virtual space construction unit 60 not only constructs the virtual space 1, but also receives user input (user object information in the virtual space 1 or the real space) supplied from the virtual space user terminal 62 and reconstructs the virtual space 1. The virtual space information and user object information are output to the information storage unit 72, and the virtual space construction information is updated.

[0030] The user object information in the virtual space 1 is, for example, information about the user's avatar, etc. If the virtual space user terminal 62 is an HMD (Head Mount Display), the user object information in the real space is, for example, facial orientation information and gaze information measured by the HMD worn by the user, images captured by a camera or the like that captures the user, gesture information acquired from the captured images, communication information (audio information) input from a microphone, and UI (User Interface) operation information.

[0031] For example, the virtual space construction unit 60 receives voice information from the virtual space user terminal 62 and recognizes when the user starts, ends, or is currently communicating with other avatars. The communication recognition information is output to and stored in the information storage unit 72.

[0032] As an example of providing (feedback) to the virtual space user terminal 62, the virtual space construction unit 60 generates an In Play advertisement based on analysis information analyzed by the information analysis unit 75 stored in the information storage unit 72, and displays the generated In Play advertisement in the virtual space 1 to provide (feedback) to the virtual space user terminal 62. In this case, the analysis information is, for example, the detection result of the reaction (smile) of the avatar viewing the In Play advertisement, the degree of excitement, In Play advertisement generation information used to generate the In Play advertisement, etc.

[0033] As an example of providing information to the virtual space user terminal 62, the virtual space construction unit 60 also outputs music corresponding to the analysis information analyzed by the information analysis unit 75 stored in the information storage unit 72 to the virtual space 1 in order to provide the music to the virtual space user terminal 62. The analysis information in this case is, for example, emotion estimation result information of the avatar based on the facial expression and voice of the avatar.

[0034] For example, if an irritated expression or movement is estimated from the expression or movement of the avatar, the virtual space construction unit 60 outputs relaxing music in the virtual space 1. In other words, the virtual space construction unit 60 not only constructs the virtual space 1, but also functions as an information providing unit that provides (feeds back) the analysis results of the sensing results acquired from the virtual space 1 or information based on the analysis results to the virtual space 1 or the virtual space user terminal 62 that uses the virtual space 1.

[0035] The virtual space construction unit 60 provides the sensing results, such as images and sounds, photo albums made up of images, sounds, etc., stored in the information storage unit 72, to the virtual space 1 or the virtual space user terminal 62 that uses the virtual space 1. At this time, the virtual space construction unit 60 also functions as an information providing unit that provides the sensing result information to the virtual space 1 or the virtual space user terminal 62 that uses the virtual space 1.

[0036] The virtual space object analysis server 61 is configured to include an information storage unit 72 , an object information registration unit 73 , a sensing unit 74 , and an information analysis unit 75 .

[0037] The information storage unit 72 is a database that stores virtual space construction information, which is information necessary for the virtual space construction unit 60 to construct the virtual space 1, as well as information about the target object (target object information), information about the tracking target object including the target object (tracking target object information), and information about the sensing object (sensing object information). Note that the target object is an object selected as the tracking target object from among objects in the virtual space 1 in response to an operation by a registrant, and information about the target object is registered in the information storage unit 72 by the object information registration unit 73. The tracking target object is an object that is set by a registrant or is a tracking target of a sensing object set by the virtual space sensing system 51. The tracking target object also includes the target object. The target object information, tracking target object information, and sensing object information are registered by the object information registration unit 73, stored and managed by the information storage unit 72, and updated by the sensing unit 74.

[0038] The information storage unit 72 also stores sensing result information (for example, RGB image information, audio information, etc.) acquired by the sensing unit 74. The information storage unit 72 also stores analysis result information analyzed by the information analysis unit 75.

[0039] The object information registration unit 73 is configured to include a registration UI generation unit 81 and a registration information generation unit 82 .

[0040] The registration UI generation unit 81 generates a registration UI such as a GUI for the registrant to register a target object and a sensing object from among the objects in the virtual space 1. Details of the registration UI will be described later with reference to FIG. 4 and subsequent figures. The registration UI generation unit 81 receives virtual space information from the virtual space construction unit 60, and the registration UI projects (displays) a portion of a three-dimensional area of ​​the virtual space 1 as two-dimensional information based on the virtual space information, for example, in a virtual space display area 113 shown in FIG. 4 (described later). The registration UI information is output to the object information registrant terminal 63. Note that the portion of the virtual space displayed in the registration UI may be three-dimensional information instead of two-dimensional information, and the portion of the virtual space that is three-dimensional information may be displayed as the registration UI on a display capable of three-dimensional display, such as an HMD.

[0041] Based on the UI operation information by the registrant supplied from the object information registrant terminal 63, the registration information generation unit 82 sets the TrackingTargetObjectID described below in the sensing object information as information indicating that the object selected by the registrant among the objects in the virtual space 1 is the target object and that it is the sensing object selected to perform sensing on the target object.

[0042] As a result, object information of an object having an ID corresponding to TrackingTargetObjectID is generated as attention object information (tracking target object information), and is sent to the information storage unit 72 and registered therein.

[0043] Furthermore, the registration information generation unit 82 generates sensing object information including Setting information (described later) for setting the sensing range of the selected sensing object. As a result, sensing object information including a TrackingTargetObjectID and Setting information is generated and sent to the information storage unit 72 for registration.

[0044] Although the sensing object information and tracking target object information are registered based on UI operation information by the registrant, they may also be registered in advance in the virtual space sensing system 51. Also, all objects may be set as tracking target objects.

[0045] The sensing unit 74 is configured to include a sensing necessity determining unit 91 , a sensing object space position updating unit 92 , and a sensing result information generating unit 93 .

[0046] The sensing necessity determination unit 91 periodically acquires the tracking target object information and sensing object information registered in the information storage unit 72. The sensing necessity determination unit 91 determines whether sensing is necessary based on the acquired tracking target object information and sensing object information.

[0047] For each sensing object, a sensing condition is set, for example, to acquire an image from the front of the avatar every frame, every few frames, every 30 seconds, or every minute. Based on this sensing condition, the sensing necessity determination unit 91 determines whether sensing is necessary and whether movement is necessary. Sensing is performed continuously under the sensing condition described above. However, as an additional sensing condition, for example, sensing may be started when the user's avatar starts communicating with another avatar, or may be continued until the user's avatar finishes communicating with the other avatar.

[0048] If sensing is necessary and involves movement of the sensing object, the sensing necessity determining unit 91 outputs the tracked object information and sensing object information to the sensing object space position updating unit 92 .

[0049] Note that information necessary for the determination is held in the sensing necessity determination unit 91 as necessary. For example, if the coordinates (position) of the tracked object indicated by KeyPointCoord (described later) move from the previous coordinates (the coordinates at the time of the last sensing) by the number of pixels that is the threshold for determining that the tracked object has moved, the sensing object moves in accordance with the movement of the tracked object. Note that movement also includes rotation.

[0050] In this case, the "previous coordinate" information is stored in the sensing necessity determination unit 91. If the sensing object does not move, the sensing necessity determination unit 91 outputs the sensing object information and the tracked object information (for example, a sensing object ID and KeyPointCoord, which is spatial position information of the corresponding tracked object) to the sensing result information generation unit 93.

[0051] The sensing object space position update unit 92 moves the sensing object to an appropriate position depending on the tracked object. For example, when a sensing range (photography range) is set to sense the area in front of an avatar, which is the tracked object, the sensing object must always be positioned in front of the avatar. Therefore, the sensing object space position update unit 92 updates sensing object position information (KeyPointCoord) indicating the position of the sensing object depending on the position, facial orientation, gaze direction, etc. of the avatar so that the positional relationship between the tracked object and the sensing object is maintained. Information supplied from the sensing necessity determination unit 91 is referenced to determine the position of the tracked object and the sensing object in the virtual space 1.

[0052] For example, if it is difficult to accurately obtain the avatar's facial direction and gaze information, the information may be used as complementary information to the real-space user object information (such as facial direction information and gaze information) supplied to the information storage unit 72 via the virtual space construction unit 60.

[0053] After updating the position information, the sensing object space position update unit 92 outputs the information supplied from the sensing necessity determination unit 91 and the updated information to the sensing result information generation unit 93. At that time, the sensing object space position update unit 92 sends sensing object position information indicating the position of the sensing object included in the sensing object information to the information storage unit 72, updates the sensing object position information stored in the information storage unit 72 with the latest position, and outputs it.

[0054] Based on the supplied sensing object information and tracked object information, the sensing result information generation unit 93 senses (photographs) the sensing range (photographing range) within the virtual space 1 based on the virtual space construction information supplied from the virtual space construction unit 60, and generates sensing result information.

[0055] For example, if the sensing range is the entire tracked object and the sensing object is set to "capture the entire tracked object as an RGB image," the sensing result information will be a two-dimensional RGB image (color image) that captures the entire tracked object from the three-dimensional virtual space 1. The sensing result information is output to and saved in the information storage unit 72. Note that although this has been described as sensing, the actual processing is rendering, and in this case, generating (rendering) a two-dimensional RGB image of the entire tracked object based on the virtual space construction information corresponds to sensing.

[0056] The information analysis unit 75 analyzes the sensing result information stored in the information storage unit 72. For example, if the sensing result information is a facial image of an avatar, the information analysis unit 75 analyzes (estimates) emotions from the facial image. For example, if the sensing result information is audio in a space, the information analysis unit 75 analyzes (estimates) the level of excitement from the audio in the space.

[0057] The analysis results are output to and stored in the information storage unit 72. When performing the analysis, the information analysis unit 75 may additionally use real-space user object information (facial direction information, line of sight information, gesture information, image information in the real space) acquired from the virtual space user terminal 62 using the virtual space 1 as the above-mentioned real-space user object information.

[0058] The information stored in the information storage unit 72 (such as analysis result information and sensing result information) may be provided (feedback) by the virtual space construction unit 60 to the virtual space 1 or the virtual space user terminal 62 using the virtual space 1, or may be provided to the object information registrant terminal 63, or may be provided externally, as will be described in detail later with reference to FIG. 19 . For example, when the analysis result indicates that the space is excited, the virtual space construction unit 60 may generate and display an in-play advertisement in the virtual space 1. Alternatively, when there is analysis result information indicating that anger is estimated from a facial image obtained in the real space, the virtual space construction unit 60 may control the rendering of the facial expression of an avatar in the virtual space 1.

[0059] The virtual space user terminal 62 is a terminal used by a user who uses the virtual space 1, and is configured, for example, as an HMD, a smartphone, a tablet terminal, a personal computer, or a game device. The user uses the virtual space 1 by accessing the virtual space 1 of the virtual space system provided by the virtual space construction unit 60 using the virtual space user terminal 62.

[0060] The object information registrant terminal 63 is a terminal used by a registrant to register object information and sensing object information, and may be, for example, an HMD, a smartphone, a tablet terminal, a personal computer, or a game console. The registrant uses the object information registrant terminal 63 to access a registration UI of the virtual space object analysis server 61 to register information such as an object of interest (object to be tracked) and a sensing object. Although not shown in FIG. 3 , the object information registrant terminal 63 may acquire sensing results and analysis results of the registered object of interest (object to be tracked) and sensing object from the information storage unit 72. In this case, the information storage unit 72 also functions as an information provider that provides sensing results, analysis results, and the like to the object information registrant terminal 63.

[0061] 3 shows an example in which the virtual space user terminal 62 and the object information registrant terminal 63 are configured separately, but the virtual space user terminal 62 and the object information registrant terminal 63 may be the same terminal. That is, a user who uses the virtual space 1 may use the virtual space user terminal 62 to register an object of interest (object to be tracked) and a sensing object, etc. The operation authority for the sensing object, such as registering an object of interest (object to be tracked) or a sensing object, may be set. For example, if a user does not want others to sense their avatar, they may be able to set a refusal of sensing.

[0062] 3, virtual space object analysis server 61 is provided with information storage unit 72, and information storage unit 72 stores virtual space construction information used by virtual space construction unit 60 outside virtual space object analysis server 61, but virtual space construction unit 60 may be configured inside virtual space object analysis server 61. Also, a virtual space server separate from virtual space object analysis server 61 may manage the virtual space construction information, and virtual space object analysis server 61 may use the virtual space construction information managed by that virtual space server.

[0063] <Example of Registration UI> FIG. 4 is a diagram showing an example of a registration UI.

[0064] In FIG. 4, the registration UI 101 is configured to include a target object registration area 111 located on the upper left, a sensing object registration area 112 located on the lower left, and a virtual space display area 113 located on the right.

[0065] The object of interest registration area 111 is an area for manually or automatically registering an object of interest as an object to be tracked from among the objects in the virtual space 1. The object of interest registration area 111 displays radio buttons for indicating whether to use an ID or pointing as a selection method for manually selecting an object of interest, icons for indicating whether to prioritize an object located at the forefront or a central object located at the center of the field of view as a selection method for automatically selecting an object of interest, and a registration button for instructing registration of object information.

[0066] 4, the radio button indicating that IDs are to be used in the manual selection is selected, and so a list of selectable object IDs (5yeg5t, 23g7ra, yq234h, 4y82f5) is displayed to the right of the radio button. The ID list may also display the IDs of selectable objects present in the area where images are displayed in the virtual space display area 113. If there are many selectable object IDs, the ID list can be scrolled.

[0067] When an object of interest is selected in the object of interest registration area 111 and the registration button is selected, the object of interest information of the ID of the object of interest is registered in the information storage unit 72 .

[0068] The sensing object registration area 112 is configured to include an ID selection field, a type selection field, a detailed settings button, and a registration button.

[0069] The ID selection field is a selection field for selecting the ID of a tracking target object including an object of interest targeted by the sensing object to be registered. The type selection field is a selection field for selecting which type of sensing object to specify. The advanced settings button is a button for performing advanced settings for the sensing object. The registration button is a button for registering the settings of the sensing object.

[0070] When an ID selection field is selected, the displayed triangle changes from pointing right to pointing downward, and a list of IDs of already registered attention objects or objects in the virtual space 1 stored in the information storage unit 72 is displayed. The ID of a desired object can be selected as the tracking target object from the list displayed in the ID selection field. In Figure 4, "5yeg5t" is selected.

[0071] The type selection field displays a list of sensing object types that can be set for the tracking target object with the ID selected in the ID selection field. A desired sensing object type can be selected from the list displayed in the type selection field. In Figure 4, "2D RGB Camera" is selected.

[0072] The minimum necessary sensing object information is registered by just the tracked object and sensing object type, but by selecting the advanced settings button, the sensing object advanced settings screen shown in Figure 6, which will be described later, is displayed, allowing for more detailed settings of the sensing object.

[0073] When a tracking target object is selected in the sensing object registration area 112, a sensing object type is selected, and the registration button is selected, sensing object information of the selected object type corresponding to the selected tracking target object is registered in the information storage unit 72.

[0074] A portion of the virtual space 1 (actually, two-dimensional information onto which the space is projected) is displayed in the virtual space display area 113. The field of view of the space displayed in the virtual space display area 113 can be moved by operating the keyboard on the object information registrant terminal 63 or by operating the HMD. In the virtual space display area 113 of Fig. 4, a frame 121 indicating that the object is selected is displayed around the avatar (the avatar in the foreground in the figure) of the object of interest (object to be tracked) "5yeg5t" having the ID selected in the ID selection field in the object of interest registration area 111.

[0075] The registration screen for the object of interest and the registration screen for the sensing object may be provided separately. Furthermore, the person who registered the object of interest and the person who registered the sensing object may be different people (such as a user of the virtual space) or may be the same person.

[0076] In this manner, a tracking target object (object of interest) specific to the registrant, i.e., the user of the virtual space sensing system 51, can be set.

[0077] It is possible to set sensing objects for all objects in advance and obtain sensing results based on those settings, but this would impose a heavy load on the system and would also result in unnecessary sensing results.As described above, it is possible to set tracking target objects (objects of interest) that require sensing specific to the user, thereby reducing the load on all objects.

[0078] <Another Example of Registration UI> FIG. 5 is a diagram showing another example of the registration UI of FIG.

[0079] The target object registration area 111 in FIG. 5 differs from the target object registration area 111 in FIG. 4 in that instead of the radio button indicating that ID is used in the manual case, a radio button indicating that pointing is used in the manual case is selected.

[0080] That is, in the target object registration area 111 of FIG. 5, the radio button indicating that pointing is to be used in the manual case is pressed, and therefore, in the virtual space display area 113, a frame 121 indicating that the target object is being selected, an arrow for selecting the target object, and a pointer 131 indicating that the target object has been selected by the arrow are displayed.

[0081] When a target object is selected in the virtual space display area 113 and the registration button is selected, the position of the target object in the virtual space 1 is calculated from the position in the virtual space display area 113, and the target object information is registered in the information storage unit 72.

[0082] <Sensing Object Detail Setting Screen> FIG. 6 is a diagram showing a display example of a sensing object detail setting screen of the registration UI.

[0083] 6, a sensing object detailed setting screen 141 is displayed when the detailed setting button is selected in the sensing object registration area 112 in FIG.

[0084] The sensing object detail setting screen 141 is configured to include a position and direction setting area 151 for setting the sensing position and sensing direction to be performed by the sensing object, a sensing range setting area 152 for setting the sensing range, and a toggle switch 153 for visualizing existing sensing objects.

[0085] The position / orientation setting area 151 displays a portion of the virtual space 1, i.e., the space in which the sensing object to be set exists (projected two-dimensional information). A movement button 161 and a rotation button 162 are provided at the top of the position / orientation setting area 151. By selecting the movement button 161, the position / orientation setting area 151 becomes an area for setting the position of the sensing object, and by selecting the rotation button 162, the position / orientation setting area 151 becomes an area for setting the sensing direction.

[0086] In Figure 6, since the move button 161 is selected, the position / direction setting area 151 becomes the position setting area for the sensing object, and the position / direction setting area 151 displays a camera icon 163 representing the sensing object with a camera function that is currently being set, and a cursor 164 for setting the position.

[0087] The person registering the sensing object can operate cursor 164 using a mouse or the like to move camera icon 163 to the desired position, thereby setting the position of the sensing object corresponding to camera icon 163 on the plane (x, y, z) that makes up its position and direction setting area 151 in virtual space 1.

[0088] In the position / orientation setting area 151 in Figure 6, the toggle switch 153 for visualizing existing sensing objects is ON, so icons representing existing sensing objects (for example, a microphone icon 165 representing a sensing object with a microphone function) are displayed.

[0089] If you want to change the settings of an existing sensing object instead of the sensing object you are currently setting, you can change the target for position setting operations using command operations (control button and click) while the existing sensing object is visualized.

[0090] Furthermore, by operating the keyboard, it is possible to move (including the depth) the space displayed in the position / direction setting area 151. In this case, the space cut out at a specified size in the position and sensing direction is displayed in the sensing range setting area 152.

[0091] The sensing range setting area 152 displays a space cut out to a specified size in the position and sensing direction set for the sensing object in the position and direction setting area 151 .

[0092] A range setting button 171, a range movement button 172, and a range change button 173 are provided at the top of the sensing range setting area 152. Fig. 6 shows the sensing range setting area 152 in a state where none of the buttons are selected.

[0093] By selecting the range setting button 171, the sensing range setting area 152 becomes the sensing range setting area for the sensing object. By selecting the range movement button 172, the sensing range setting area 152 becomes the sensing range movement area for the sensing object. By selecting the range change button 173, the sensing range setting area 152 becomes the sensing range change area for the sensing object.

[0094] Note that what is displayed in the sensing range setting area 152 is a certain range from the position set in the position / direction setting area 151, so for example, if a sensing object is placed right next to the face of an avatar (object to be tracked), a close-up of the face will be displayed in the sensing range setting area 152.

[0095] If a registrant wishes to sense the entire body of the tracked object, the range displayed in the sensing range setting area 152 can be adjusted by adjusting the position in the position / orientation setting area 151 so that the entire body is displayed in the sensing range setting area 152. The value of the specified size itself (for example, the maximum size of the frame displayed in the sensing range setting area 152) is defined as the maximum range to be cropped in the virtual space sensing system 51. Although a registrant cannot sense an image size larger than this, if the registrant wishes to sense a wider range, this can be achieved by adjusting the position.

[0096] FIG. 7 is a diagram showing another display example of the sensing object detail setting screen of FIG.

[0097] FIG. 7 shows the position / direction setting area 151 with the rotation button 162 selected, and the sensing range setting area 152 with the range setting button 171 selected.

[0098] The position and direction setting area 151 in Figure 7 displays a camera icon 163 representing the sensing object to be set, and a sphere 166 consisting of three axes (Yaw / Pitch / Roll) with the camera icon 163 at the center.

[0099] In the sphere 166, the camera icon 163 can be rotated by sliding the mouse over the Yaw axis indicated by a thick line, the Pitch axis indicated by a dashed line, and the Roll axis indicated by a dotted diagonal line. In response to the rotation of the camera icon 163, the sensing direction of the sensing object corresponding to the camera icon 163 is set, and the display in the sensing range setting area 152 is also updated.

[0100] 7 displays a space cut out to a specified size at the position and sensing direction set for the sensing object in the position / orientation setting area 151. In the sensing range setting area 152, the sensing range can be set by setting a rectangle 175 indicated by an upper left point 174-1 and a lower right point 174-2 in the displayed space.

[0101] <Display Example of Sensing Range Setting Area> FIG. 8 is a diagram showing another display example of the sensing range setting area.

[0102] FIG. 8A shows the sensing range setting area 152 with the range movement button 172 selected.

[0103] In the sensing range setting area 152 of A in Figure 8, if a rectangle 175 has already been set in space, a cursor (arrow) 176 is displayed, and by moving the rectangle 175 using the cursor 176, the sensing range indicated by the rectangle 175 can be moved.

[0104] FIG. 8B shows the sensing range setting area 152 with the range movement button 172 selected.

[0105] 8B, when a rectangle 175 has already been set in space, a desired corner 177 on the rectangle 175 can be highlighted by specifying it with a cursor (arrow) 176. Then, as indicated by an arrow P, the sensing range indicated by the rectangle 175 can be transformed by using a cursor 178 to enlarge (reduce) the corner 177 to a desired position.

[0106] <Object Information> FIG. 9 is a diagram showing an example of object information relating to all objects in the virtual space 1. As shown in FIG.

[0107] In other words, the object information in Figure 9 is information about all objects, and is information exchanged between the information storage unit 72 and the object information registration unit 73, for example, when registering a target object and a sensing object, or when using a service provided by the virtual space sensing system 51.

[0108] In FIG. 9, the object information is configured to include ObjectID, UserID, ObjectType, IsSensing, KeyPointCoord1, and KeyPointCoord2.

[0109] The Object ID is an ID unique to each software object registered in the virtual space system.

[0110] The User ID is a user-specific ID when an object belongs to the operating user (virtual space user terminal 62). For example, when operating an avatar to walk around the virtual space 1, the user operates his or her own avatar, but in the virtual space 1, there is no user who is the subject of operation for certain objects, i.e., objects that do not have a User ID assigned. For example, User IDs are not assigned to objects such as people other than the avatar (non-player characters), animals, plants, products, and backgrounds.

[0111] ObjectType indicates the type of object. Note that no distinction is made here between non-sensing objects and sensing objects. Examples of ObjectType include human, dog, tree, flower, house, 2D RGB camera, and microphone.

[0112] IsSensing is information indicating whether or not an object is a sensing object. In this example, microphone and 2d_rgb_camera are sensing objects, while human, dog, and tree are normal objects (non-sensing objects) other than sensing objects.

[0113] KeyPointCoord1 and KeyPointCoord2 are object position information and are the three-dimensional coordinates of feature points defined for each ObjectType (for example, skeleton points for a person). For example, if the ObjectType is human, the object position information is represented by KeyPointCoord1, which indicates the elbow position, and KeyPointCoord2, which indicates the hand position. For example, in the case of a sensing object whose ObjectType is microphone or 2d_rgb_camera and whose IsSensing is 1, the object position information is represented only by KeyPointCoord1, which indicates the center point.

[0114] The entire position range of the object is then determined based on multiple KeyPointCoords corresponding to the ObjectType. These KeyPointCoord values ​​change as each object moves within the virtual space.

[0115] In the case of FIG. 9, the UserID of the ObjectID 5yeg5t is tfq34f, the ObjectType is human, IsSensing is 0, the (x, y, z) of KeyPointCoord1 is 571, 1215, 5213, and the KeyPointCoord2(x, y, z) is 592, 42, 5213.

[0116] The UserID of ObjectID 242tqa is asdf3, the ObjectType is dog, IsSensing is 0, KeyPointCoord1(x,y,z) is 3421,-1231,456, and KeyPointCoord2(x,y,z) is 4756,-4256,57.

[0117] The UserID of ObjectID ar24rfa is none, ObjectType is tree, IsSensing is 0, KeyPointCoord1(x,y,z) is 435,567,8654, and KeyPointCoord2(x,y,z) is 8675,765,534.

[0118] The UserID of ObjectID gfas5s is none, ObjectType is microphone, and IsSensing is 1, indicating that it is a sensing object, and KeyPointCoord1 (x, y, z) is 571, 1215, 5213, and KeyPointCoord2(x, y, z) is - (none).

[0119] The UserID of ObjectID j211234 is none, the ObjectType is 2d_rgb_camera, and IsSensing is 1, which indicates that it is a sensing object, and KeyPointCoord1 (x, y, z) is 3421, -1231, 456, and KeyPointCoord2(x, y, z) is - (none).

[0120] <Sensing Object Information> FIG. 10 is a diagram showing an example of sensing object information set for a sensing object among the object information in FIG.

[0121] 9 is shown separately for ease of explanation, the sensing object information for each Object ID of the sensing object may be a single table of information that combines the information in Figures 9 and 10. Furthermore, the information shown in Figures 9 and 10 may be information that is set in advance in the virtual space sensing system 51, rather than being set by a registrant as shown in Figures 4 to 8.

[0122] In FIG. 10, the sensing object information is configured to include Object ID, Object Type, Tracking Target Object ID, and Setting 1 to Setting 5.

[0123] The Object ID and Object Type are the same as those in Fig. 9. In the case of Fig. 9, information on non-sensing objects and information on sensing objects are included without distinction, but in Fig. 16, only information on sensing objects is shown.

[0124] The object types of the sensing object include, for example, 2d_rgb_camera, 2d_gray_camera, 3d_rgb_camera, 3d_gray_camera, and microphone.

[0125] The TrackingTargetObjectID is the Object ID of the tracking target object (or the target object if selected by the user) registered as the tracking target of each sensing object. That is, the TrackingTargetObjectID is the identifier of the tracking target object (or the target object if selected by the user via the object information registrar terminal 63) that is to be tracked by each sensing object.

[0126] Setting 1 to Setting 5 differ depending on the object type of the sensing object.

[0127] If the object type is a camera (2d_rgb_camera, 2d_gray_camera, 3d_rgb_camera, 3d_gray_camera, etc.), Settings 1 to 5 include, for example, image_size, relative_position, angle_of_fov, yaw, roll, and pitch.

[0128] image_size is the image size. relative_position is the relative sensing position, i.e., the offset of the sensing object's position relative to the tracked object's position. angle_of_fov is the field of view. yaw, roll, and pitch are the sensing angles yaw, roll, and pitch of the sensing target.

[0129] The definitions of these setting values, including the yaw, roll, and pitch axes, are held by the virtual space sensing system 51.

[0130] When the object type is a microphone, Setting1 to Setting3 include, for example, radius and relative_position.

[0131] "radius" is the radius of the sound collection. "relative_position" is the relative sound collection (sensing) position.

[0132] That is, when the object type is a microphone, the sensing range is the spherical range around the position of relative_position, that is, the spherical range having the radius of radius.

[0133] In the case of Figure 10, the Object ID of j211234 is the Object Type of 3d_rgb_camera, and the key and value of Setting 1 are image_size and (256,256). The key and value of Setting 2 are relative_position and (-5,40,10), and the key and value of Setting 3 are angle_of_fov and 60. The key and value of Setting 4 are yaw and 15, the key and value of Setting 5 are roll and 49, and the key and value of Setting 6 are pitch and 330.

[0134] The ObjectType of ObjectID gfas5s is microphone, the key and value of Setting1 are radius and 1000, and the key and value of Setting2 are relative_position and (0,15,15).

[0135] The ObjectID of tw3tgf is 2d_gray_camera, and the key and value of Setting1 are image_size and (256,256). The key and value of Setting2 are relative_position and (0,0,0), and the key and value of Setting3 are angle_of_fov and 40. The key and value of Setting4 are yaw and 0, the key and value of Setting5 are roll and 90, and the key and value of Setting6 are pitch and 0.

[0136] The ObjectType of ObjectID 234j8s is 2d_gray_camera, and the key and value of Setting1 are image_size and (256,256). The key and value of Setting2 are relative_position and (0,0,57), and the key and value of Setting3 are angle_of_fov and 40. The key and value of Setting4 are yaw and 0, the key and value of Setting5 are roll and 90, and the key and value of Setting6 are pitch and 0.

[0137] <Relative Shooting Position> FIG. 11 is a diagram illustrating the relative shooting position.

[0138] In FIG. 11, the sensing object 11 is shown in a coordinate system with the tracked object 12 at the center, the x-axis being the two-dimensional horizontal direction, the y-axis being the depth direction, and the z-axis being the two-dimensional vertical direction.

[0139] Although the sensing object 11 is originally a point coordinate and has no width, it is shown as a square shape and a round shape in FIG. 11 for the sake of convenience.

[0140] 11, a center position CP is indicated for the tracked object 12. The location of the center position CP for the tracked object 12 is defined by the ObjectType. For example, when the ObjectType is Human, the center position CP is defined as the center of gravity of all KeyPointCoords.

[0141] FIG. 11A shows the sensing object 11 placed at the relative image capturing position relative_position=(0,0,57) in coordinates with the center position CP of the tracked object 12 as the origin.

[0142] FIG. 11B shows the sensing object 11 placed at the relative image capturing position relative_position=(0,0,0) in coordinates with the center position CP of the tracked object 12 as the origin.

[0143] That is, when the relative shooting position is (0,0,0), the position of the sensing object 11 overlaps with the position (origin) of the tracked object 12 .

[0144] <Sensing Range> FIG. 12 is a diagram showing an example of a sensing range.

[0145] FIG. 12 shows the sensing object 11 placed at a relative image capturing position relative_position=(x, y, z) in coordinates where the tracked object 12 is the origin (0, 0, 0).

[0146] The space of the image size (w, h) in the direction (photographing angles yaw, roll, pitch) seen from the sensing object 11 is the sensing range.

[0147] Of the yaw, roll, and pitch that represent the shooting angles, the yaw angle is a parameter that indicates whether the shooting direction of the sensing object 11 is the direction of the tracked object, or a direction other than the direction of the tracked object (the line of sight as seen from the tracked object). For example, when the yaw angle is 0 to 90 degrees and 270 to 360 degrees, the shooting direction is the direction of the tracked object, and when the yaw angle is 91 to 269 degrees, the shooting direction is the line of sight as seen from the tracked object.

[0148] Specifically, if yaw=0 is the direction toward the center of the tracked object, for example, for the ObjectType with ObjectID j211234 shown in FIG. 10, yaw=15 means the shooting direction is toward the tracked object. For the ObjectType with ObjectID tw3tgf, yaw=0 means the shooting direction is toward the tracked object. In these cases, the sensing range is the tracked object itself. On the other hand, for the ObjectType with ObjectID 234j8s, yaw=180 means the shooting direction is toward the line of sight of the tracked object. In this case, the sensing range is the field of view of the tracked object.

[0149] It should be noted that the screen for setting the details of a sensing object (i.e., the setting method) is not limited to the configuration of the sensing object detail setting screen 141 in Fig. 6. Furthermore, the screen for setting the details of a sensing object (setting method) may be configured, for example, like a sensing object detail setting screen 181 in Fig. 13, which will be described next.

[0150] <Sensing Object Detail Setting Screen> FIG. 13 is a diagram showing another display example of the sensing object detail setting screen of FIG.

[0151] The sensing object detail setting screen 181 is configured to include a display field for the sensing object ID, a field for selecting the ID of the object to be tracked, a field for selecting the type of sensing object, a setting button for visualizing the sensing object in the registration UI, an input field for the image size, an input field for the field of view angle, a selection field for the shooting direction, a selection field for the shooting area, and an input field for the relative shooting position.

[0152] The sensing object ID is an identifier of the sensing object. In Fig. 13, "j211234" is displayed in the sensing object ID display field. The sensing object ID is set appropriately by the system.

[0153] The ID selection field for the tracking target object and the type selection field for the sensing object type are the same as those in FIG. 4, and therefore a description thereof will be omitted.

[0154] The setting button for visualizing the sensing object in the registration UI is a YES button or a NO button that sets whether or not to display the camera icon 163 or microphone icon 165 described above in Figure 6 in the space displayed in the virtual space display area 113.

[0155] By selecting the YES button, in the case of a sensing object with a camera function, a camera icon 163 of the sensing object set on the corresponding tracked object (the avatar in the foreground in the figure) on the two-dimensional information in the virtual space display area 113 is displayed.

[0156] By selecting the YES button, in the case of a sensing object with a microphone function, a microphone icon 165 of the sensing object set for the corresponding tracking target object (the avatar at the back in the drawing) on ​​the two-dimensional information in the virtual space display area 113 is displayed. At this time, the position and direction of the displayed icon may be set to correspond to the position and direction of the corresponding sensing object.

[0157] The image size (pixels) is the size of the image sensed by the sensing object. In the case of Fig. 13, in the image size input field, (256) is input as the vertical width and (256) is input as the horizontal width.

[0158] The viewing angle (degrees) is the viewing angle sensed by the sensing object. In the viewing angle input field, 45 degrees is input for the top and bottom, or 45 degrees for the left and right.

[0159] The shooting direction is the direction in which the sensing object shoots. In the shooting direction selection field, you can select either the object itself (inside) or the outside direction (outside). In the case of Figure 6, the object itself (inside) is selected in the shooting direction selection field.

[0160] The photographing portion is the portion photographed by the sensing object. When the self is selected in the photographing direction selection field, the whole body (all) or face (face) can be selected in the photographing portion selection field. In the example of FIG. 13, face (face) is selected in the photographing portion selection field. Note that when the outward direction is selected in the photographing direction selection field, no selection is possible in the photographing portion selection field.

[0161] The relative shooting position is the relative position from the center of the sensing object to the center of the tracked object. In the input field for the relative shooting position, the x-coordinate, y-coordinate, and z-coordinate of the relative shooting position can be entered to specify the relative shooting position. In Figure 13, (-5, 40, 40) is entered in the (x, y, z) input field for the relative shooting position.

[0162] Here, the parameter setting area 182 includes an input field for image size, an input field for field of view angle, a field for selecting the shooting direction, a field for selecting the shooting portion, and an input field for the relative shooting position. The configuration of the parameter setting area 182 differs depending on the sensing object type. In the example of Fig. 13, the configuration of the parameter setting area 182 when the sensing object type is "2D RGB camera" is shown. That is, a tracking target object is set in the sensing object registration area 112, and parameters for setting the sensing range are set in this parameter setting area 182, thereby setting the sensing range targeted by the sensing object.

[0163] In addition, in Figure 13, an example is shown in which the x-coordinate, y-coordinate, and z-coordinate of the relative shooting position are input numerically in the input field for the relative shooting position, but the relative shooting position may also be input intuitively by operating the camera icon 153 of the sensing object displayed in the virtual space display area 113.

[0164] That is, in the virtual space display area 113, by selecting the camera icon 153 of the sensing object displayed behind the tracked object and moving the selected camera icon 153 to the desired position (in front of the tracked object), the relative positions of the tracked object and the sensing object during and after the movement are calculated and input in real time.

[0165] The two-dimensional coordinates are calculated and input in accordance with the movement of the camera icon 153, and the depth coordinates can be input by controlling the display of the virtual space in the virtual space display area 113, for example by scrolling the mouse.

[0166] At this time, the x, y, and z coordinates of the relative position calculated in response to the movement of the camera icon 153 and the control of the display of the virtual space are displayed in real time in the input field for the relative shooting position.

[0167] <Regarding the Shooting Direction> FIG. 14 is a diagram illustrating the shooting direction in FIG.

[0168] In FIG. 14, the sensing object 11 is shown as a square shape and a round shape for convenience, similarly to FIG.

[0169] In FIG. 14A, the sensing object 11 is shown in a coordinate system with the tracked object 12 at the center, the x-axis being the two-dimensional horizontal direction, the y-axis being the depth direction, and the z-axis being the two-dimensional vertical direction.

[0170] When taking the relative coordinates of the sensing object 11 and the tracked object 12, the forward direction of the tracked object 12 (for example, the face direction or line of sight direction in the case of a person) indicated by the arrow P is taken as the positive direction of the x-axis, as shown in A of Fig. 14. This way of taking the coordinates is defined by the settings.

[0171] For example, when the relative coordinates of the sensing object 11 are x>0, as indicated by relative_position = (20, 0 10), the sensing object 11 is placed in front of the tracked object 12. The relative coordinates are input as the relative shooting position on the sensing object detail setting screen described above in FIG.

[0172] The shooting direction (for example, "inside" indicating that the sensing object faces the tracked object, or "outside" indicating that the sensing object faces outward) is determined by the selection in the shooting direction selection field on the sensing object detail setting screen. When the shooting direction is "inside," shooting is performed from the position of the sensing object 11 toward the sensing object itself (the tracked object). At that time, the specific part to be shot is set in more detail (whole body or face) depending on the shooting part (sensing part), and then shot. In this case, the sensing range is a range that includes at least a part of the tracked object itself. On the other hand, when the shooting direction is "outside," shooting is performed from the position of the sensing object 11 toward the outside. In this case, the sensing range is the field of view as seen from the tracked object.

[0173] 14B and 14C, the coordinates of the sensing object 11 are shown with the tracked object 12 at the center, the x-axis being the horizontal direction, and the y-axis being the vertical direction.

[0174] 14B shows a case where the x coordinate of the relative shooting position is a positive value, in which case the sensing object 11 is placed in front of the tracked object 12, so when the shooting direction is towards the user himself, the front side of the tracked object 12 is shot. When the shooting direction is outward, the line of sight of the tracked object 12 is shot. On the other hand, FIG. 14C shows a case where the x coordinate of the relative shooting position is a negative value, in which case the sensing object 11 is placed behind the tracked object 12, so when the shooting direction is towards the user himself, the back of the tracked object 12 is shot. When the shooting direction is outward, the line of sight of the tracked object 12 when its eyes are at the back of its head is shot.

[0175] Although the UI example in FIG. 13 does not show an example of an instruction to capture an image behind the tracked object, a UI may be added that allows selection of angles such as 90 degrees left or right, or behind (180 degrees) relative to the face direction or gaze direction other than the viewpoint direction. For example, when the capture direction is outward, the system automatically sets the position of the sensing object to a (relative) position on the x-axis at the origin (or a slight positive offset from the eye position). In such cases, if sensing of the rear, etc. is also permitted, further settings may be made so that when 180 degrees is selected, the x value becomes negative, and when 90 degrees or 270 degrees is selected, the x value is positioned on the y-axis. Furthermore, more detailed settings may be possible. Furthermore, when the capture direction is oneself, the sensing object may be automatically set to a (relative) position where x is greater than or equal to a threshold value, taking into account the need to move away from the user to a position where oneself can be sensed.

[0176] <Example of Sensing Range> FIG. 15 is a diagram showing an example of a sensing range when a sensing object has a camera function.

[0177] Figure 15 shows an example in which the parameter setting area 182 of Figure 13 has the image size for the tracked object (avatar) with ID "5yeg5t" set in Figure 4 set to (height: 1024, width: 256), the field of view set to 45 degrees up and down and left and right, the shooting direction set to the user himself, and the shooting area set to the whole body.

[0178] That is, in the case of FIG. 15, the sensing range (photographing range) targeted by the sensing object is "the entire object to be tracked."

[0179] When using the sensing object detail setting screen 141 in FIG. 6, the registrant can set the shooting directions yaw, roll, and pitch by selecting the range in the position and direction setting area 151 as described above in FIG. 7 for the sensing range in FIG. 15.

[0180] <Another Example of Sensing Range> FIG. 16 is a diagram showing another example of the sensing range when the sensing object has a camera function.

[0181] Figure 16 shows an example in which the parameter setting area 182 of Figure 13 has set the image size for the tracked object (avatar) with ID "5yeg5t" set in Figure 4 to (height: 256, width: 256), the field of view to 45 degrees up and down and left and right, the shooting direction to the user himself, and the shooting area to the face.

[0182] That is, in the case of FIG. 16, the sensing range targeted by the sensing object is "a part of the tracking target object (face part)."

[0183] In addition, when using the sensing object detail setting screen 141 of FIG. 6, the registrant can set the shooting directions yaw, roll, and pitch for the sensing range of FIG. 16 by selecting the range in the position / direction setting area 151, as described above in FIG. 7.

[0184] <Sensing Object Information> FIG. 17 is a diagram showing an example of sensing object information in the case of the sensing object detail setting screen 181 of FIG.

[0185] Note that, similarly to Fig. 10, the sensing object information in Fig. 17 is information set for a sensing object among the object information in Fig. 6, and for convenience of explanation, it is described separately from the object information in Fig. 9, but the sensing object information for each Object ID of a sensing object may be a single table of information that combines the information in Fig. 9 and Fig. 17. Furthermore, both the information shown in Fig. 9 and Fig. 17 may be information that is set in advance in the system, rather than being set by a user as shown in Fig. 4 and Fig. 13.

[0186] In FIG. 17, the sensing object information is configured to include Object ID, Object Type, Tracking Target Object ID, and Setting 1 to Setting 5.

[0187] The Object ID and Object Type are the same as those in Fig. 9. In the case of Fig. 9, information on non-sensing objects and information on sensing objects are included without distinction, but in Fig. 17, only information on sensing objects is shown.

[0188] The object types of the sensing object include, for example, 2d_rgb_camera, 2d_gray_camera, 3d_rgb_camera, 3d_gray_camera, and microphone.

[0189] The TrackingTargetObjectID is the Object ID of the tracking target object (or the target object if selected by the user) registered as the tracking target of each sensing object. That is, the TrackingTargetObjectID is the identifier of the tracking target object (or the target object if selected by the registrant using the object information registrant terminal 63) that is to be tracked by each sensing object.

[0190] Setting 1 to Setting 5 differ depending on the object type of the sensing object.

[0191] When the object type is a camera (2d_rgb_camera, 2d_gray_camera, 3d_rgb_camera, 3d_gray_camera, etc.), Setting1 to Setting5 include, for example, image_size, relative_position, angle_of_fov, sensing_direction, sensing_part, etc.

[0192] image_size is the image size. relative_position is the relative shooting (sensing) position. angle_of_fov is the field of view angle. sensing_direction is the sensing direction (shooting direction) by the sensing object, and can be selected from inside or outside.

[0193] "Inside" indicates that when the position of the tracked object is the origin, sensing will occur in the direction inside (the origin) as seen from the sensing object. "Outside" indicates that when the position of the tracked object is the origin, sensing will occur in the direction outside (opposite the origin) as seen from the sensing object. Note that when the relative shooting position is (0,0,0), that is, when the tracked object and the sensing object are in the same position, this means that sensing will occur for the tracked object itself, and "outside" cannot be specified in sensing_direction.

[0194] "sensing_part" is a setting for selecting the entire or partial sensing (photography) range of the object to be tracked when "sensing_direction" is "inside." "sensing_direction" is "outside." "sensing_part" cannot be specified. The system maintains the definitions of these setting values.

[0195] When the object type is a microphone, Setting1 to Setting3 include, for example, radius and relative_position.

[0196] "radius" is the radius of the sound collection. "relative_position" is the relative sound collection (sensing) position.

[0197] In the case of Figure 17, the ObjectType of ObjectID j211234 is 3d_rgb_camera, and the key and value of Setting1 are image_size and (256,256). The key and value of Setting2 are relative_position and (-5,40,10), and the key and value of Setting3 are angle_of_fov and 60. The key and value of Setting4 are sensing direction and inside, and the key and value of Setting5 are sensing part and all.

[0198] The ObjectType of ObjectID gfas5s is microphone, the key and value of Setting1 are radius and 1000, and the key and value of Setting2 are relative_position and (0,15,15). The ObjectType of ObjectID tw3tgf is 2d_gray_camera, the key and value of Setting1 are image_size and (256,256). The key and value of Setting2 are relative_position and (0,0,0), and the key and value of Setting3 are angle_of_fov and 40. The key and value of Setting4 are sensing direction and inside, and the key and value of Setting5 are sensing part and face.

[0199] The ObjectType of ObjectID 234j8s is 2d_gray_camera, and the key and value of Setting1 are image_size and (256,256). The key and value of Setting2 are relative_position and (0,0,57), and the key and value of Setting3 are angle_of_fov and 40. The key and value of Setting4 are sensing direction and outside, and the key and value of Setting5 are none.

[0200] Note that, in both the cases of raw, roll, and pitch in the setting method of Fig. 6 described above and the cases of inside and outside in the setting method of Fig. 13 , the algorithm is based on information indicating the shooting direction with the sensing object as the origin (the direction as seen from the sensing object), but these are merely examples. For example, the origin of the shooting direction may be the tracking target object.

[0201] <Processing of Virtual Space Sensing System> FIG. 18 is a flowchart illustrating processing of the virtual space sensing system 51.

[0202] In order to use the virtual space sensing system 51 , the registrant uses the object information registrant terminal 63 to access the registration UI 101 of the virtual space object analysis server 61 .

[0203] In step S111, the registration UI generation unit 81 generates a registration UI 101 such as a GUI for the registrant to register a target object and a sensing object from among the objects in the virtual space 1, and outputs information of the registration UI 101 to the object information registrant terminal 63.

[0204] The registrant operates the object information registrant terminal 63 to input information required for registering the target object and the sensing object to the registration UI 101 displayed on the monitor (not shown) of the object information registrant terminal 63 .

[0205] In step S112, the object information registration unit 73 generates attention object information (tracking target object information) and sensing object information based on the UI operation information by the registrant, and registers them in the information storage unit 72. At this time, the relevant object information in the virtual space construction information stored in the information storage unit 72 is referenced.

[0206] The sensing necessity determination unit 91 periodically acquires tracking target object information including the target object and sensing object information (ObjectType, ObjectType, KeyPointCoord, etc. in FIG. 9 for ObjectID corresponding to TrackingTargetObjectID in FIG. 13 or FIG. 16).

[0207] In step S113, the sensing necessity determination unit 91 waits until it determines that sensing is necessary based on KeyPointCoord, which is position information of the sensing target object (object of interest) included in the acquired tracked object information. The sensing object information may include, for example, sensing conditions such as a sensing start time, a sensing end time, the number of sensing operations, and a predetermined sensing interval (e.g., every 10 seconds). If it is determined in step S113 that sensing is necessary, the process proceeds to step S114. Note that if sensing is performed for each frame and there are no additional conditions described above, step S113 is unnecessary.

[0208] In step S114, the sensing necessity determination unit 91 determines whether the tracked object (object of interest) has moved. If it is determined in step S114 that the tracked object (object of interest) has moved, it is assumed that this involves movement of the sensing object, and the processing proceeds to step S115. At this time, the sensing necessity determination unit 91 outputs tracked object information (object of interest information) and sensing object information to the sensing object space position update unit 92.

[0209] In step S115, the sensing object space position update unit 92 tracks the sensing target object (target object) and moves the sensing object to an appropriate position relative to the sensing target object (target object) so that the positional relationship with the sensing target object (target object) is maintained.

[0210] The sensing object space position update unit 92 updates the sensing object position information included in the sensing object information among the information supplied from the sensing necessity determination unit 91, and outputs the information supplied from the sensing necessity determination unit 91 to the sensing result information generation unit 93. Furthermore, the sensing object space position update unit 92 updates the sensing object position information with the latest position for the sensing object information stored in the information storage unit 72 and outputs it.

[0211] In step S116, the sensing result information generation unit 93 sets a sensing range in the virtual space based on the supplied tracking target object information (target object information) and sensing object information, and senses the set sensing range. That is, the sensing result information generation unit 93 generates the sensing results of the sensing range by rendering based on the virtual space construction information. The sensing results are output to and saved in the information storage unit 72.

[0212] In step S117, the sensing necessity determination unit 91 determines whether or not to end sensing based on information such as the end of use of the virtual space sensing system 51. If it is determined in step S117 that sensing should not be ended, the process returns to step S114, and the subsequent processes are repeated.

[0213] If it is determined in step S117 that sensing is to be ended, the processing of the virtual space sensing system 51 in FIG. 18 ends.

[0214] Note that the processing in Fig. 18 is one example. For example, if the registration of the target object or sensing object has been performed in advance, or if information previously registered in the virtual space sensing system 51 is used without any settings by a registrant, processing is performed without the registration-related processing in steps S111 and S112 when an actual service using the virtual space sensing system 51 is used. Furthermore, although the processing in Fig. 18 is processing performed in real time on the virtual space, it may also be performed on stored virtual space content, for example.

[0215] In the virtual space sensing system 51, the sensing result information (images and audio) obtained as described above is stored in the information storage unit 72, and the images can then be provided to users of the virtual space 1 or registered users of the object of interest as individual images or as part of a photo album, or the audio can be provided for personal enjoyment.

[0216] Additionally, the information analysis unit 75 can provide analysis result information (such as the degree of excitement and estimated result information on the user's emotions) obtained by analyzing the sensing results to the virtual space user terminal 62. Furthermore, the analysis result information can be used to provide in-play advertisements in the virtual space 1 that are more favorable to users and advertisers.

[0217] As described above, the above processing can be performed using information registered in advance in the virtual space sensing system 51 without any settings by a registrant or the like, but a registrant or user may not necessarily need sensing results based on all objects. Therefore, by being able to set tracking target objects (objects of interest) specific to a user such as a registrant or user, it is possible to provide the user with the sensing results they need and reduce the load on the system.

[0218] 3. Second Embodiment (Cooperation with External System) <System Configuration> FIG. 19 is a diagram illustrating an example of the configuration of a virtual space sensing system according to a second embodiment of the present technology.

[0219] The virtual space sensing system 201 of FIG. 19 differs from the virtual space sensing system 51 of FIG. 3 in that an external system 211, an external website server 212, and an external device (including an information analysis unit 213) are added.

[0220] In other words, the virtual space sensing system 201 in Figure 19, like the virtual space sensing system 51 in Figure 3, is a system different from a virtual space system, and therefore can provide to the outside the sensing results or analysis results of the virtual space 1 that could not be provided by conventional virtual space systems alone.

[0221] The external system 211 is, for example, a system that generates and provides In-Play advertisements. For example, like the virtual space construction unit 60 in FIG. 3 , the external system 211 acquires analysis result information stored in the information storage unit 72 that analyzes the level of excitement in the virtual space 1, and generates In-Play advertisements to be displayed in the virtual space 1 based on the acquired analysis result information. The external system 211 outputs the generated In-Play advertisements to the virtual space construction unit 60. The external system 211 also generates a photo album containing images that are sensing result information stored in the information storage unit 72 and outputs the photo album to the virtual space construction unit 60. That is, in FIG. 19 , the information storage unit 72 also functions as an information providing unit that provides, to an external party, sensing result information acquired from the virtual space 1, analysis result information of the sensing result information, or information based on the analysis result information.

[0222] The external website server 212 is, for example, a server for a website that introduces a virtual space. The external website server 212 acquires analysis result information that analyzes the degree of excitement in the virtual space 1, which is stored in the information storage unit 72, and reflects fixed-point observations of the excitement in the virtual space 1 based on the analysis result information on the website that introduces the virtual space 1 of the virtual space sensing system 51. Note that the external website server 212 may also reflect fixed-point observations using images that are sensing result information stored in the information storage unit 72 on the website that introduces the virtual space 1 of the virtual space sensing system 51.

[0223] The information analysis unit 213 configured in the external device acquires the sensing result information or the analysis result information by the information analysis unit 75 from the information storage unit 72 via the information analysis unit 75 .

[0224] For example, the information analysis unit 213 analyzes the sensing result information in the same manner as the information analysis unit 75. If the sensing result information is a facial image of an avatar, the information analysis unit 75 analyzes (estimates), for example, joy, anger, sadness, or happiness from the facial image. If the sensing result information is spatial audio, the information analysis unit 213 analyzes (estimates), for example, the degree of excitement from the spatial audio. The analysis result by the information analysis unit 213 is fed back to the information storage unit 72 via the information analysis unit 75, for example.

[0225] Furthermore, for example, if the external device of the information analysis unit 213 is a device of a company that specializes in estimating emotions, the information analysis unit 75 extracts feature points from the facial image, and the resulting information is supplied to the information analysis unit 213, which can then share the analysis processing according to its area of ​​expertise, such as estimating emotions based on the extracted feature points.

[0226] In addition, like the information analysis unit 75, when performing analysis, the information analysis unit 213 may additionally use, as external system information 214, the above-mentioned real-space user object information, such as facial orientation information, gaze information, and gesture information in the real space obtained from the virtual space user terminal 62 using the virtual space.

[0227] As described above, the virtual space sensing system 201 of Figure 19 can cooperate with external parties such as the external system 211, the external website server 212, and the information analysis unit 213 configured in the external device, and instead of providing sensing result information or analysis result information, it can obtain feedback that incorporates the technology in which the external company excels, thereby providing even more convenient services to users.

[0228] Although not specifically mentioned in FIG. 19, when providing sensing result information or analysis result information, a right to use the information to be provided may be set.

[0229] <4. Others> <Examples of Services> FIG. 20 is a diagram showing examples of services provided by the present technology.

[0230] FIG. 20 shows an example in which a user wears an HMD 231 as a virtual space user terminal 62 and uses the virtual space 1 .

[0231] For example, the user uses the registration UI to register in advance in the information storage unit 72 that the avatars 251 and 252 of the user's friends are to be set as attention objects and that sensing will be performed using the respective sensing objects having camera functions. At this time, the faces of the avatars 251 and 252 themselves are set as the sensing ranges.

[0232] Based on the above-mentioned registration, the sensing result information generation unit 93 generates RGB images of the facial parts of the avatars 251 and 252 and stores them in the information storage unit 72. The information analysis unit 75 then detects feature points of the facial parts from the RGB images stored in the information storage unit 72 and estimates emotions.

[0233] The emotion estimation result information is stored in the information storage unit 72, and the emotion estimation result information (happy degree) is acquired by the virtual space construction unit 60, and, for example, speech bubbles 261 and 262 indicating the happy degrees of avatars 251 and 252 are displayed in the virtual space 1 of the HMD 231 that the user is viewing. The speech bubble 261 indicates that the happy degree is 60%, as indicated by "Happy: 60". The speech bubble 262 indicates that the happy degree is 90%, as indicated by "Happy: 90".

[0234] In this way, by estimating emotions based on the sensing results and displaying the emotion estimation result information on the HMD 231, the user using the virtual space 1 can recognize the emotions of their friend's avatars 251 and 252 during the conversation.

[0235] Similarly, it is registered in advance in the information storage unit 72 that the user's avatar will be used as the target object and that sensing will be performed by each sensing object having a camera function.

[0236] In this case, by setting the user's avatar's own face as the sensing range, for example, when communicating with friends' avatars 251 and 252, an image 263, which is sensing result information of the user's avatar, can be displayed on the child screen W.

[0237] This allows communication with friends' avatars 251 and 252 in virtual space 1 while checking the facial expression of one's own avatar.

[0238] Furthermore, in the example of Figure 20, RGB images of the facial portions of avatars 251 and 252 are generated and stored in information storage unit 72, so even if avatars 251 and 252 are facing backwards, it is possible to display the RGB images of the facial portions of avatars 251 and 252 on a sub-screen W, for example.

[0239] <Another Example of Service> FIG. 21 is a diagram showing another example of a service provided by the present technology.

[0240] FIG. 21 shows an example in which a user uses a virtual space user terminal 62 to use VR shopping 271 as one of the virtual spaces 1 .

[0241] In VR shopping 271, all products including product 281 are set in advance as objects of interest. At this time, the imaging direction is, for example, in the case of the sensing object detail setting screen 141 in Fig. 6, the line of sight direction (field of view range) of the object of interest is set by the imaging angles yaw, roll, and pitch, and in the case of the sensing object detail setting screen 181 in Fig. 13, it is set to outside.

[0242] At this time, the sensing object with a camera function registered for the object of interest is set to always perform sensing, and the fact that the product 281, which is the object of interest, has been picked up by the avatar 283 is detected by storing an image, which is sensing result information, in the information storage unit 72 and analyzing the stored image by the information analysis unit 75.

[0243] For example, the information analysis unit 75 analyzes the image to determine the range 291 in which to focus the avatar 283, and provides feedback (provides) this information to the information storage unit 72 to correct the sensing range of the avatar 283 by the sensing object to the range 291 in which to focus the avatar 283.

[0244] The sensing object information in the information storage unit 72 is updated by feedback of the focus range of the avatar 283, so that the sensing unit 74 can detect the position of the sensing range with high quality using the updated sensing object. This allows sensing of an image and more accurate analysis of the facial expression of the avatar 283 from the sensed image.

[0245] For example, by linking the facial expression of an avatar when picking up a product to purchasing behavior, it is possible to analyze the facial expression of the purchaser when they first saw the product.

[0246] The examples of services in FIGS. 20 and 21 are examples of services provided by a sensing object having a camera function.

[0247] Although not shown in the drawings, examples of services provided by sensing objects with microphone functionality include the following: For example, suppose that in virtual space 1, there are multiple target objects (e.g., avatars) to which sensing objects with microphone functionality are set, and each target object is engaged in a conversation within a group of avatars that includes the target object itself.

[0248] In this case, for example, by analyzing information about sounds in the virtual space 1 acquired from a sensing object with a microphone function, it is possible to understand the state of the area around the group (whether they are talking, debating, or calm). This allows a service to provide music in each group in the virtual space 1 according to the group's state, such as lowering the volume for groups that are talking and raising the volume for groups that are not talking.

[0249] <Effects of the Present Technology> In the present technology, the tracking target object spatial position, which is the spatial position of the tracked object in virtual space, and the sensing object spatial position, which is the spatial position of the sensing object for sensing the sensing range set for the tracked object in virtual space, are updated in accordance with changes in the tracking target object spatial position so that the positional relationship is maintained, and sensing result information, which is the sensing result of the sensing range by the sensing object at the sensing object spatial position, is generated.

[0250] This makes it possible to easily obtain information about the virtual space, and as a result, it is possible to provide a wide variety of services to both users who use the virtual space and users who do not.

[0251] <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into a general-purpose personal computer, etc.

[0252] FIG. 22 is a block diagram showing an example of the hardware configuration of a computer 900 that executes the above-described series of processes according to a program.

[0253] A CPU (Central Processing Unit) 901 , a ROM (Read Only Memory) 902 , and a RAM (Random Access Memory) 903 are interconnected by a bus 904 .

[0254] An input / output interface 910 is also connected to the bus 904. An input unit 911 including a keyboard, a mouse, etc., and an output unit 912 including a display, a speaker, etc. are connected to the input / output interface 910. Also connected to the input / output interface 910 are a storage unit 913 including a hard disk, a nonvolatile memory, etc., a communication unit 914 including a network interface, etc., and a drive 915 that drives a removable recording medium 921.

[0255] In the computer 900 configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the memory unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing it.

[0256] The program executed by the CPU 901 is recorded on a removable recording medium 921, or is provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and installed in the storage unit 913.

[0257] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0258] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0259] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0260] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0261] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0262] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0263] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0264] <Examples of Combinations of Configurations> The present technology may also be configured as follows. (1) An information processing device comprising: a sensing object spatial position update unit that updates a tracking target object spatial position in response to a change in the tracking target object spatial position so as to maintain a positional relationship between the tracking target object spatial position, which is the spatial position of the tracking target object in a virtual space, and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set for the tracking target object in the virtual space; and a sensing result information generation unit that generates sensing result information that is a sensing result of the sensing range by the sensing object at the sensing object spatial position. (2) The information processing device according to (1), further comprising an information analysis unit that analyzes the sensing result information. (3) The information processing device according to (2), further comprising an information provision unit that provides an analysis result by the information analysis unit or information based on the analysis result. (4) The information processing device according to (3), in which the information provision unit provides the analysis result or information based on the analysis result to the virtual space or a virtual space user terminal that uses the virtual space. (5) The information processing device according to (2), wherein the information analysis unit uses sensing information in real space to analyze the sensing result information. (6) The information processing device according to any of (1) to (5), wherein the sensing result information generation unit generates the sensing result information by performing rendering processing using virtual space information for constructing the virtual space. (7) The information processing device according to any of (1) to (6), wherein the sensing object has a camera function. (8) The information processing device according to (7), wherein the sensing range includes at least a part of the tracked object itself. (9) The information processing device according to (7), wherein the sensing range is a field of view as seen from the tracked object. (10) The information processing device according to any of (1) to (6), wherein the sensing object has a microphone function.(11) The information processing device according to (10), wherein the sensing range is a periphery of the tracked object. (12) The information processing device according to any one of (1) to (11), further comprising a registration unit that registers at least one of an object of interest and the sensing object as the tracked object. (13) The information processing device according to (12), wherein the registration unit generates the UI that registers at least one of the object of interest and the sensing object. (14) The information processing device according to (13), wherein the registration unit registers the object of interest in accordance with a selection result of identification information of an object in the virtual space via the UI. (15) The information processing device according to (13), wherein the registration unit generates the UI that includes a display of two-dimensional information or three-dimensional information of a partial area of ​​the virtual space. (16) The information processing device according to (15), wherein the registration unit generates the UI that registers the object of interest in accordance with a selection result of an object displayed on the display. (17) The information processing device according to (15), wherein the registration unit generates the UI for registering the sensing range of the sensing object according to a movement position of the sensing object displayed on the display. (18) The information processing device according to (1), further comprising an information providing unit that provides the sensing result or information based on the sensing result. (19) The information processing device according to any of (1) to (18), configured as a system separate from a virtual space system that provides the virtual space.(20) An information processing method in which an information processing device updates a tracking target object spatial position in accordance with a change in the tracking target object spatial position so that a positional relationship between the tracking target object spatial position, which is the spatial position of the tracking target object in a virtual space, and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set for the tracking target object in the virtual space, is maintained, and generates sensing result information, which is a result of sensing the sensing range by the sensing object at the sensing object spatial position. (21) A program that causes a computer to function as a sensing object spatial position update unit that updates the tracking target object spatial position in accordance with a change in the tracking target object spatial position so that a positional relationship between the tracking target object spatial position, which is the spatial position of the tracking target object in a virtual space, and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set for the tracking target object in the virtual space, is maintained, and a sensing result information generation unit that generates sensing result information, which is a result of sensing the sensing range by the sensing object at the sensing object spatial position.

[0265] 1 Virtual space, 11, 11a, 11b Sensing object, 12, 12a, 12b Tracked object, 51 Virtual space sensing system, 61 Virtual space object analysis server, 62 Virtual space user terminal, 63 Object information registrant terminal, 71 Virtual space construction unit, 72 Information storage unit, 73 Object information registration unit, 74 Sensing unit, 75 Information analysis unit, 81 Registration UI generation unit, 82 Registration information generation unit, 91 Sensing necessity determination unit, 92 Sensing object space position update unit, 93 Sensing result information generation unit, 101 Registration UI, 111 Attention object registration area, 112 Sensing object registration area, 113 Virtual space display area, 141 Sensing object detail setting screen, 151 Position and direction setting area, 152 Sensing range setting area 181 Sensing object detailed setting screen, 182 Parameter setting area, 201 Virtual space sensing system, 211 External system, 212 External website server, 213 Information analysis unit

Claims

1. An information processing device comprising: a sensing object spatial position update unit that updates a tracking target object spatial position in response to a change in the previous tracking target object spatial position so as to maintain a positional relationship between the tracking target object spatial position, which is the spatial position of the tracking target object in a virtual space, and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set for the tracking target object in the virtual space; and a sensing result information generation unit that generates sensing result information, which is the sensing result of the sensing range by the sensing object at the sensing object spatial position.

2. The information processing device according to claim 1, further comprising an information analysis unit that analyzes the sensing result information.

3. The information processing device according to claim 2, further comprising an information providing section that provides a result of the analysis by said information analysis section or information based on said analysis result.

4. The information processing device according to claim 3, wherein the information providing unit provides the analysis result or information based on the analysis result to the virtual space or a virtual space user terminal that uses the virtual space.

5. The information processing device according to claim 2, wherein the information analysis unit uses sensing information in real space to analyze the sensing result information.

6. The information processing device according to claim 1, wherein the sensing result information generation unit generates the sensing result information by performing a rendering process using virtual space information for constructing the virtual space.

7. The information processing device according to claim 1, wherein the sensing object has a camera function.

8. The information processing device according to claim 7, wherein the sensing range includes at least a portion of the tracked object itself.

9. The information processing device according to claim 7, wherein the sensing range is a visual range seen from the object to be tracked.

10. The information processing device according to claim 1, wherein the sensing object has a microphone function.

11. The information processing device according to claim 10, wherein the sensing range is a periphery of the tracked object.

12. The information processing device according to claim 1, further comprising a registration unit that registers at least one of an object of interest and the sensing object as the tracked object.

13. The information processing device according to claim 12, wherein the registration unit generates the UI for registering at least one of the target object and the sensing object.

14. The information processing device according to claim 13, wherein the registration unit registers the object of interest in accordance with a selection result of identification information of an object in the virtual space via the UI.

15. The information processing device according to claim 13, wherein the registration unit generates the UI including a display of two-dimensional or three-dimensional information of a partial area of ​​the virtual space.

16. The information processing device according to claim 15, wherein the registration unit generates the UI for registering the object of interest in accordance with a selection result of an object displayed on the display.

17. The information processing device according to claim 1, further comprising an information providing unit that provides the sensing result or information based on the sensing result.

18. The information processing device according to claim 1, configured as a system separate from the virtual space system that provides the virtual space.

19. An information processing method in which an information processing device updates a tracking target object spatial position, which is the spatial position of a tracking target object in a virtual space, in response to a change in the tracking target object spatial position so that a positional relationship between the tracking target object spatial position, which is the spatial position of a sensing object for sensing a sensing range set for the tracking target object in the virtual space, is maintained, and generates sensing result information, which is the sensing result of the sensing range by the sensing object at the sensing object spatial position.

20. A program that causes a computer to function as: a sensing object spatial position update unit that updates a tracking target object spatial position in response to a change in the tracking target object spatial position so that a positional relationship between the tracking target object spatial position, which is the spatial position of the tracking target object in a virtual space, and a sensing object spatial position, which is the spatial position of a sensing object for sensing a sensing range set for the tracking target object in the virtual space, is maintained; and a sensing result information generation unit that generates sensing result information that is the sensing result of the sensing range by the sensing object at the sensing object spatial position.