Information processing system, information processing device, control method for information processing device, and program

Abstract

Provided is an information processing device that is connected to a display device and a sensor for detecting the relative positions of the user and the display device, wherein the information processing device acquires information representing the relative positions of the user and the display device detected by the sensor, selects at least one of the virtual objects as an object to be controlled, and controls the position or posture the virtual object that is being controlled in the video data thereof on the basis of the acquired information representing the relative positions of the user and the display device. The information processing device then displays the video data, which has been generated on the basis of information on the virtual space in which the virtual object is placed, by outputting the data to the display device.

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing device, a control method for an information processing device, and a program. [Background technology]

[0002] In recent years, display devices have become more diverse and their performance has improved, and there are now display devices with an expanded viewing angle so that the displayed content can be viewed not only from the front of the display device but also from directions relatively close to the side, as well as display devices that enable stereoscopic viewing, whereby different displayed content can be viewed depending on the viewing direction. Summary of the Invention [Problem to be solved by the invention]

[0003] However, even when displaying objects such as game characters on such a display device, while a user can enjoy a visual experience, the entertainment value of the user moving relative to the object being viewed has not been taken into consideration.

[0004] The present invention has been made in consideration of the above-mentioned situation, and one of its objects is to provide an information processing system, an information processing device, a control method for an information processing device, and a program that can improve entertainment value by enabling control of objects to be displayed based on the user's relative position with respect to the display device. [Means for solving the problem]

[0005] One aspect of the present invention for solving the problems of the above-described conventional examples includes an information processing device connected to a display device and a sensor that detects a relative position between a user and the display device, the information processing device including means for acquiring information detected by the sensor that indicates the relative position between the user and the display device, and a generation means for generating video data including at least one virtual object, the generation means controlling, with at least one of the virtual objects as a control target, a position or orientation of the virtual object that has become the control target within the video data based on the acquired information that indicates the relative position between the user and the display device, and the generated video data is output to the display device for display. Effect of the Invention

[0006] This aspect of the invention allows control of displayed objects depending on the user's relative position with respect to the display device, enhancing entertainment value. [Brief description of the drawings]

[0007] [Figure 1] 1 is a schematic block diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an explanatory diagram illustrating an example of a coordinate system set by an information processing device according to an embodiment of the present invention. [Diagram 3] 1 is a functional block diagram illustrating an example of a functional configuration of an information processing device according to an embodiment of the present invention. [Figure 4] FIG. 2 is an explanatory diagram illustrating an example of an arrangement of each device in the information processing system according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] An embodiment of the present invention will be described with reference to the drawings. As shown in Fig. 1, an information processing system 1 according to the embodiment of the present invention includes an information processing device 10, a display device 20 connected thereto, and a sensor device 30. The information processing device 10 also includes a control unit 11, a storage unit 12, an operation control unit 13, a display control unit 14, and a communication unit 15.

[0009] In this embodiment, the display device 20 may be a general liquid crystal display or the like, or may be a stereoscopic display such as that provided by Voxon Photonics, which dynamically displays a stereoscopic image in space.

[0010] The sensor device 30 detects the position of the user around the display device 20 and generates information indicating the relative position of the user and the display device 20. As an example, the sensor device 30 detects the user around the display device 20 by a predetermined method. This detection method may be, for example, a method that uses image data captured by a camera with an angle of view covering the range in which the user can move around the display device 20, or a method that uses infrared rays or the like.

[0011] 2, the sensor device 30 generates information (hereinafter referred to as relative position information) indicating the detected user's position relative to the position of the display device 20, using a predetermined global coordinate system whose origin is the placement position of the display device 20, for example, the center C of the display device 20 (the center of a virtual rectangular parallelepiped circumscribing the image space M displayed by the display device 20). Note that, if the display device 20 is a stereoscopic display that displays a stereoscopic image, the image space M is the space in which the stereoscopic display is performed, and if the display device 20 is a display that displays a planar image, it is the surface itself.

[0012] As an example, the global coordinates are expressed by, for example, the relative position information of the user in a cylindrical coordinate system as follows. That is, r in this cylindrical coordinate system (r, θ, z) is the distance from the center C of the display device 20 to the position of the target (such as the center of the user's head, the same applies below) in a horizontal plane (a plane parallel to the floor surface on which the user stands). In addition, a predetermined direction R in this horizontal plane is taken as the reference direction, and the angle (assumed to be a clockwise angle in a planar view) between a line segment extending from the center C of the display device 20 in the reference direction and a line segment extending from the center C of the display device 20 toward the target is taken as θ. In addition, the distance from a plane parallel to the horizontal plane including the center C of the display device 20 to the target position, with the vertical upward direction from the horizontal plane being positive, is taken as z.

[0013] In this case, the sensor device 30 generates and outputs information on the relative position of the detected user as a position (r, θ, z) in the global coordinate system in the cylindrical coordinate system.

[0014] Furthermore, the sensor device 30 may generate and output information representing the user's posture instead of or together with the detected relative position information of the user. The information representing the posture may be, for example, information representing the direction in which the user is facing (front direction) (for example, an angle from a reference direction in the above global coordinate system may be used) or information on the user's pose. The pose information may be expressed using information on the positions of the nodes of a bone model when each part of the user's body, such as the hands and feet, is fitted to the bone model.

[0015] The control unit 11 of the information processing device 10 is a program-controlled device such as a CPU, and operates according to a program stored in the storage unit 12. In one example of the present embodiment, the control unit 11 acquires information indicating a relative position between the user and the display device 20 detected by the sensor device 30. The control unit 11 also manages information on a virtual space including at least one virtual object. The control unit 11 then generates video data indicating the virtual object in the virtual space, and outputs the video data to the display device 20 for display.

[0016] In this embodiment, the control unit 11 generates video data by controlling at least one of the virtual objects as a control target, based on information (relative position information) indicating the relative position between the user and the display device 20, acquired from the sensor device 30, and controlling the position or orientation of the virtual object as a control target in the video data. The operation of the control unit 11 will be described in detail later.

[0017] The storage unit 12 is a memory device or the like, and holds a program executed by the control unit 11. This program may be provided by being stored in a computer-readable and non-transitory recording medium, and may be stored in the storage unit 12. The storage unit 12 also operates as a work memory for the control unit 11.

[0018] The operation control unit 13 accepts a user's operation and outputs it to the control unit 11. In one example of the present embodiment, the operation control unit 13 accepts information indicating the content of the operation performed by the user on a controller (not shown) held by the user, and outputs the information to the control unit 11.

[0019] The display control unit 14 outputs video information to the display device 20 according to instructions input from the control unit 11. The communication unit 15 includes an interface such as a USB, and receives information output from the sensor device 30 and outputs it to the control unit 11. The communication unit 15 also includes a network interface, and sends data to other information processing devices 10, external server devices, etc. via a communication means such as a network according to instructions input from the control unit 11. Furthermore, the communication unit 15 outputs data received via this network interface from other information processing devices 10, external server devices, etc. via a communication means such as a network to the control unit 11.

[0020] Next, a description will be given of an example of the operation of the control unit 11 of this embodiment. The control unit 11 of this embodiment functionally includes a position information acquisition unit 111, an application execution unit 112, and a video data generation unit 113, as shown in FIG.

[0021] The position information acquisition unit 111 acquires the relative position information of the user outputted by the sensor device 30 and the like.

[0022] The application execution unit 112 executes a process of placing a virtual object in a virtual space (e.g., a virtual space such as a game space) in accordance with an application program such as a game application. As an example, this virtual space is a space represented by an x, y, z Cartesian coordinate system with a predetermined point as the origin, and the horizontal plane of the virtual space is defined as the xy plane.

[0023] In one example of this embodiment, application execution unit 112 places virtual objects representing a ground surface having undulations from the horizontal plane and a road formed on the ground surface in a virtual three-dimensional space in accordance with instructions from an application program for a game using a vehicle. Furthermore, application execution unit 112 places a virtual object representing a vehicle on the virtual object on the ground surface (particularly, on the virtual object on the road surface initially) based on information acquired by position information acquisition unit 111, and sets the direction, moving speed, and the like of the vehicle in the virtual space.

[0024] As an example, based on the user's relative position information (r, θ, z) acquired by the position information acquisition unit 111, the application execution unit 112 positions the vehicle virtual object so that the longitudinal direction of the vehicle virtual object is parallel to the line segment connecting the center of the display device 20 and the user's position, and so that the user is located behind the vehicle virtual object (in the backward direction of the vehicle).

[0025] In this example, the application execution unit 112 determines a point Q' in the virtual space corresponding to the position of a predetermined point Q in an image displayed by the display device 20 in the real space. The application execution unit 112 also determines the direction in this horizontal plane relative to the reference direction of the real space, assuming that the horizontal plane in the real space is parallel to the xy plane in the virtual space. The application execution unit 112 then positions the vehicle in the virtual object so that its center is at Q' in the virtual space. At this time, the application execution unit 112 positions the vehicle so that the traveling direction of the vehicle is in the direction of (θ+180 degrees) from the above-mentioned reference direction.

[0026] Furthermore, the application execution unit 112 repeatedly moves the position of the point Q' in the virtual space by a predetermined distance Δd from the reference direction of the virtual space toward the vehicle's traveling direction (θ+180 degrees) at each predetermined timing. Then, the application execution unit 112 repeatedly instructs the video data generation unit 113 to render a virtual object in a partial virtual space (called a rendering target space) of a predetermined range determined so that the point Q' in the virtual space is located at the position of the point Q in the image at each predetermined timing.

[0027] If the display device 20 is a stereoscopic display, then the application execution unit 112 determines the rendering target space so that the horizontal plane in the real space of the image displayed by the display device 20 is parallel to the xy plane in the virtual space.

[0028] The video data generating unit 113 receives a rendering instruction from the application executing unit 112 and generates an image of a virtual object in the instructed rendering target space as video information that can be displayed on the display device 20.

[0029] As an example, if the connected display device 20 is a stereoscopic display, the video data generation unit 113 generates video data in which the voxels to be displayed are arranged based on a virtual object in the rendering target space instructed by the application execution unit 112.

[0030] Furthermore, if the connected display device 20 is a general LCD display or the like, the video data generation unit 113 generates two-dimensional video data of the rendering target space viewed in the direction of point Q' from a predetermined direction (for example, the z-axis direction of the virtual space) based on a virtual object in the rendering target space instructed by the application execution unit 112.

[0031] As described above, with regard to the process of generating image data to be displayed on display device 20 based on data of virtual objects set within a virtual space, various processes corresponding to the type of display device 20 are widely known, and therefore a detailed explanation will be omitted here.

[0032] [Operation] This embodiment basically has the above configuration and operates as in the following example. In the following example, the display device 20 is assumed to be a stereoscopic display that displays a stereoscopic image in voxels, and is placed on a table T or the like, as shown in Fig. 4. It is assumed that a user (U in Fig. 4) can move to any position around the table T on which the display device 20 is placed, and that the display device 20 is capable of presenting an image seen from any position around the table T when the user is standing there.

[0033] Furthermore, in this example, the sensor device 30 has a camera installed on the ceiling of the room in which the table T is placed, for example, and detects the position of the display device 20 and the position of the user, and generates and outputs information on the relative position of the user with respect to the display device 20 using the cylindrical coordinate system illustrated in FIG. 2.

[0034] Furthermore, the information processing device 10 connected to this display device 20 places virtual objects representing an undulating ground surface and a road formed on this ground surface in a virtual three-dimensional space, and places a virtual object representing a vehicle on the virtual object representing the road, in accordance with instructions from the game application. The information processing device 10 in this example then controls the traveling direction of the virtual object of the vehicle, based on relative position information of the user with respect to the display device 20.

[0035] Specifically, when the virtual space in which the virtual objects are placed is represented by an (x, y, z) Cartesian coordinate system, the information processing device 10 places a virtual object of the ground having undulations from the xy plane in this coordinate system. Also, a virtual object representing a road is placed on this virtual object of the ground. In the following example, the virtual object of the ground or road has a size and length sufficient to represent the state in which the vehicle is traveling relative to the size of the vehicle (i.e., is sufficiently larger or longer than the virtual object of the vehicle).

[0036] Furthermore, the information processing device 10 initially determines a point Q' in the virtual space on this road, which corresponds to the position of a predetermined point Q in the image displayed by the display device 20 in the real space. At this time, the information processing device 10 sets the virtual space so that the horizontal plane of the real space and the xy plane in the virtual space are parallel to each other.

[0037] Then, the information processing device 10 places the vehicle in the virtual object so that its center is at Q' in the virtual space. At this time, the information processing device 10 acquires the relative position information (r, θ, z) of the user from the sensor device 30, and places the vehicle so that the traveling direction is (θ+180 degrees) from the above-mentioned reference direction.

[0038] After the game starts, the information processing device 10 repeatedly acquires the user's relative position information (r, θ, z) from the sensor device 30 at predetermined timings. When the information processing device 10 acquires the relative position information (r, θ, z), the information processing device 10 moves the position of the point Q' in the virtual space by a predetermined distance Δd in the direction (θ+180 degrees) that is the traveling direction of the vehicle from the reference direction of the virtual space to a point on a virtual object on the ground in the vertical direction, and performs rendering processing on the virtual object in a predetermined rendering target space that is determined so that the point Q' in the virtual space is the position of the point Q in the image, to generate video data that can be displayed by the display device 20.

[0039] As the information processing device 10 repeatedly performs this process, the user moves in a circumferential direction around the display device 20 (in a direction that changes the value of θ detected by the sensor device 30) to control the traveling direction of the vehicle.

[0040] According to this example of the present embodiment, entertainment is provided by the user being able to move relative to an object that the user is viewing.

[0041] [Example of using distance etc.] In addition, in this example of the present embodiment, information on the viewing angle of the display device 20 is used from among the relative position information of the user output by the sensor device 30, but the present embodiment is not limited to this.

[0042] For example, the information processing device 10 may determine the traveling direction of the virtual vehicle based on the direction of the angle θ represented by the user's relative position information, and may determine its moving speed (corresponding to the magnitude of the value of Δd in the above example) based on the distance r represented by the user's relative position information.

[0043] In this way, for example, it is possible to perform control such that the traveling speed of the vehicle increases when the user approaches the display device 20, and decreases when the user moves away from the display device 20.

[0044] In another example, the information processing device 10 may determine the moving speed of the virtual vehicle based on the head height z represented by the relative position information of the user. In this case, it is possible to perform control such that the traveling speed of the vehicle increases as the user lowers his / her head.

[0045] [Using a controller] In this example of the present embodiment, the user does not necessarily need to use a controller to perform the operation.

[0046] [Control based on the viewing angle of the display device and the detection range of the sensor device] Furthermore, when the range in the horizontal plane in which the image on the display device 20 can be viewed is limited and not 360 degrees, the information processing device 10 may perform the following process so that the user's movement range is within the limited range.

[0047] In one example of this embodiment, the information processing device 10 maps the user's movable range onto the control range of a control target (for example, the virtual object of the vehicle in the above example) based on the user's relative position information.

[0048] Specifically, the object of control based on the user's relative position information is the placement angle of the virtual object in the virtual space, and its range is within a horizontal plane, with the center C of the display device 20 as the central position, and is ±θmax degrees of the reference direction (where 360 ​​degrees > θmax > 0 degrees).

[0049] Furthermore, when the viewing angle of the display device 20 is used as the range in which the user can move, and the viewing angle is ±φ degrees from the reference direction (where 360 ​​degrees > φ > 0 degrees), the information processing device 10 uses the angular direction information θ (here, θ≦φ) from the relative position information of the user detected by the sensor device 30, and if θmax≦φ: determines the placement angle of the virtual object based on the value of θ itself output by the sensor device 30.

[0050] On the other hand, when θmax>φ, the information processing device 10 calculates θ′ by mapping the angle range of ±φ degrees to the range of ±θmax using θ′=θ×(θmax / φ), and determines the placement angle of the virtual object based on this θ′.

[0051] According to this example, if one wishes to set the placement angle of a virtual vehicle to ±150 degrees, even if the user's range of movement (the range within which the user can view the image on the display device 20) is ±90 degrees, if the user moves within this range, it will be possible to control the placement angle over the full range.

[0052] [Control when multiple users exist] In the above description, the user present around the display device 20 is a single user, but the present embodiment is not limited to this example. For example, in addition to a user playing a game application, another user who watches the play of the user may be present around the display device 20.

[0053] In this example, the sensor device 30 can identify users playing the game by, for example, having the users playing the game wear markers such as light-emitting objects, or by making the color of the LEDs on the controller devices carried by the users different for each user.

[0054] In yet another example, the sensor device 30 may acquire relative position information of each of a plurality of users existing around the display device 20. In this example, the sensor device 30 assigns unique identification information to each user in advance. The sensor device 30 detects relative position information for each user, associates the detected relative position information with the identification information unique to the corresponding user, and outputs the information to the information processing device 10.

[0055] As described above, each user can be identified by changing the color of the LED of the controller device carried by the user for each user. However, this is just one example, and the sensor device 30 may identify each user by face recognition processing or the like.

[0056] In this example, when the number of users involved in processing a game application or the like is N, the information processing device 10 may set the movable range of each user by dividing the range into φ / N intervals with respect to the viewing angle φ of the display device 20, and may display a display informing each user of the movable range, for example, at the start of processing of an application program.

[0057] In this way, for example, in the case of a stereoscopic display where φ=360 degrees and N=2 users are present around the display device 20, the information processing device 10 assigns a range of 180 degrees to each user as their movable range, assigns a range of -90 degrees to +90 degrees including 0 degrees with respect to the reference direction to one user, and assigns a range of +90 degrees to 270 degrees (i.e., -90 degrees) with respect to the reference direction to the other user, and displays a guide to that effect.

[0058] Then, after the users follow this guidance and take their positions around the display device 20, the information processing device 10 may start processing an application program and perform control such as generating image data to be displayed on the display device 20 based on the relative position information of each user.

[0059] Furthermore, in this manner, when multiple users are present around the display device 20, the information processing device 10 may control the movement of a virtual object corresponding to each user differently based on the position and movement of each user, depending on the distance r to each user.

[0060] Specifically, the information processing device 10 controls the amount of movement Δd of a virtual object (e.g., a game character to be controlled by a user) corresponding to each of multiple users based on information of a distance r included in the relative position information of the corresponding user, so that the amount of movement Δd increases, for example, in proportion to the distance r, and may also control the direction of the movement based on information of an angle θ included in the relative position information of the corresponding user.

[0061] In this way, the positioning of each user relative to display device 20 is reflected in the control of each user's game character, etc., further improving the entertainment value of having the users themselves move around display device 20.

[0062] Although control of the amount of movement is used as an example here, the present embodiment is not limited to this, and the information processing device 10 may control, for each user, the display range of information for the corresponding user within the display device 20 according to the distance r included in the user's relative position information, so that the display range becomes larger as the distance r becomes smaller.

[0063] In such an example, the provision of information to a user closer to the display device 20 is promoted, so that the user can perform an operation such as moving closer to the display device 20 to obtain information as necessary.

[0064] Furthermore, the information processing device 10 may vary the parameters of the physical simulation for the virtual object controlled by each user in the virtual space according to the distance r included in the relative position information of the corresponding user. For example, the information processing device 10 may control the gravity parameter acting on the virtual object (fall acceleration relative to the ground surface in the virtual space) so that it becomes larger as the distance r becomes closer. Similarly, when a user uses an application program that performs an operation on a virtual object controlled by the user by posing, the information processing device 10 may vary the magnitude of the control according to the distance r included in the relative position information of the corresponding user. In this example, even when performing an operation with the same pose, it is possible to control the change in the arrangement angle or the movement distance of the virtual object to be larger by posing at a distance closer to the display device 20.

[0065] [Control when multiple display devices exist] In yet another example of the present embodiment, the information processing device 10 may transmit and receive information to and from other information processing devices 10, etc., to process an application program. For example, the information processing device 10 acquires relative position information of a user present around the display device 20 connected to the information processing device 10 from a sensor device 30 connected to the information processing device 10, controls the arrangement of a virtual object to be controlled by the user in the virtual space, and transmits the acquired relative position information of the user to the other information processing device 10.

[0066] Furthermore, the information processing device 10 uses relative position information of users (referred to as other users for the sake of distinction) present around the display device 20 connected to the other information processing device 10 received from the other information processing device 10 to control the placement of virtual objects to be controlled by the other users within the virtual space.

[0067] According to this example, one game application or the like can be processed by multiple users using different display devices 20. In this case, multiple display devices 20 are present, and the viewing angles or the like of the multiple display devices 20 may be different from each other. When display devices 20 with different viewing angles are used, in this example, each information processing device 10 may mutually exchange information on the viewing angles of the display devices 20 connected to itself, obtain information φmin on the narrowest viewing angle, and perform control so that the user's movable range is within this narrowest viewing angle.

[0068] For example, the information processing device 10 connected to the display device 20 having a viewing angle φ larger than the narrowest viewing angle φmin obtains corrected angle information θ' by multiplying angle information θ indicated by relative position information of the user around the display device 20, acquired from the sensor device 30, by φmin / φ. Then, the corrected angle information θ' may be used to control the direction of placement of a virtual object to be controlled by the user.

[0069] [Environment in which the display device is installed] Furthermore, in an example of this embodiment, the information processing device 10 may not only control the position and orientation of a virtual object in a virtual space based on the relative position information of the user, but also perform the following processing.

[0070] That is, in one example of this embodiment, the sensor device 30 may acquire information such as not only the relative position of the user but also the position (relative position), illuminance, color, etc. of lighting in the environment in which the display device 20 is placed, and output it to the information processing device 10.

[0071] The information processing device 10 of this example may place a virtual light source at a position in the virtual space corresponding to the position of the light source input from the sensor device 30, and illuminate the virtual object. In this way, lighting that takes into account the position of the light source in the real space is also performed on the virtual object in the virtual space, so that the direction and shape of the shadow take into account the lighting in the real space, resulting in a more natural display. Furthermore, when the illuminance and color of the corresponding light source in the real space are detected by the sensor device 30, the illuminance and color of the virtual light source may be determined based on the information on the illuminance and color.

[0072] Furthermore, when it is determined that a user is present between a light source in the real space detected by the sensor device 30 and the display device 20, the information processing device 10 may place a virtual object for forming a shadow of the user on the virtual object between a virtual light source in the virtual space corresponding to the above light source and each virtual object, thereby creating an effect in which the shadow of the user is reflected on the virtual object.

[0073] In this example, information such as the position of the light source is acquired by the sensor device 30, but instead, the user may set the position of the light source in the real space relative to the display device 20. In this example, the information processing device 10 sets the position of the virtual light source in the virtual space based on the set information. In this way, the sensor device 30 does not need to detect the position of the light source, and the processing load is reduced.

[0074] In another example, the information processing device 10 may control the illuminance and on / off of lighting in the real space according to the situation of the application program being executed. Since such processing can be performed by widely known home automation processing, detailed explanation of the processing method will be omitted.

[0075] [Processing on pseudo 3D display] Some types of display devices 20 require information on the position of the user relative to the display device 20 in order to display a stereoscopic image. In such an example, the sensor device 30 may output information on the detected relative position of the user not only to the information processing device 10 but also to the display device 20.

[0076] The display device 20 refers to information on the user's position detected by the sensor device 30, and generates and displays an image seen from the user's position based on the video data input from the information processing device 10.

[0077] [Another example of posture control] In the examples shown so far, the information processing device 10 sets the attitude (direction, etc.) of a virtual object to be controlled by the user, for example, so that the front of the virtual object is oriented at (θ+180 degrees) from the reference direction, using information θ of the angle of the user's position with respect to a reference direction included in the information on the user's relative position output by the sensor device 30. However, the present embodiment is not limited to this example.

[0078] For example, in response to an instruction from a user, the information processing device 10 determines the direction of the front of a virtual object as follows, using information θ on the angle of the user's position with respect to a reference direction, which is included in the information on the relative position of the user output by the sensor device 30: θ, θ+90 degrees, θ+180 degrees, θ+270 degrees, In this example, the angle to be added is selected from 0, 90, 180, and 270, but it may be possible to set the angle in more detailed increments of 45 degrees, or to set a substantially continuous value such as 1 degree.

[0079] In this example, a model representing the user himself / herself is treated as a virtual object (called a user model object), and the front direction of this user model object is determined using information θ about the angle of the user's position relative to a reference direction by any of the settings described above. Furthermore, a virtual object of clothing being sold is arranged superimposed on the user model object with its front direction aligned with the front direction of the user model object, and is then subjected to rendering processing.

[0080] In this way, the user can see the front, side, and back of the clothes on sale while virtually trying them on. The sensor device 30 detects the user's pose, and the information processing device 10 transforms the user model object into the shape of the detected pose, and also transforms the shape of the virtual object of the clothes in the same way (this processing can be performed by known processing using a bone model, so a detailed description of the processing will be omitted), allowing the user to view the clothes from a desired direction. [Explanation of symbols]

[0081] 1 information processing system, 10 information processing device, 11 control unit, 12 storage unit, 13 operation control unit, 14 display control unit, 15 communication unit, 20 display device, 30 sensor device, 111 position information acquisition unit, 112 application execution unit, 113 video data generation unit.

Claims

1. An information processing system including a display device, an information processing device that generates data of a video to be displayed on the display device, and a sensor that detects a relative position between each of a plurality of users and the display device, the sensor outputs information on the relative position including information on a distance to a position where each user is located, the distance being determined in relation to an arrangement position of the display device; The information processing device, means for acquiring information representing a relative position between each user and the display device detected by the sensor; a means for generating image data including a virtual object corresponding to each user, the means controlling a position or orientation of the virtual object corresponding to each user as a control target in the image data based on the acquired information representing a relative position between the corresponding user and the display device output by the sensor, the distance information being determined in relation to a placement position of the display device; An information processing system comprising:

2. An information processing device connected to a display device and a sensor that detects a relative position between each of a plurality of users and the display device, the sensor outputs information on the relative position including information on a distance to a position where each user is located, the distance being determined in relation to an arrangement position of the display device; means for acquiring information representing a relative position between each user and the display device detected by the sensor; a generation means for generating image data including a virtual object corresponding to each user, the generation means controlling a position or orientation of the virtual object corresponding to each user within the image data as a control target, based on the acquired information representing a relative position between the corresponding user and the display device output by the sensor, the information being determined in relation to a placement position of the display device; Including, An information processing device that outputs the generated video data to the display device for display.

3. 3. The information processing device according to claim 2, the sensor outputs information on a relative position between each user and the display device, the information including angle information indicating a direction in which each user is located with respect to a reference direction, the direction being a predetermined direction in a horizontal plane determined in relation to an arrangement position of the display device; The generation means is an information processing device that controls the position or orientation of the virtual object of each user who is the subject of control within the image data based on information representing the relative position between the corresponding user and the display device detected by the sensor.

4. 4. The information processing device according to claim 3, The generation means is an information processing device that controls the posture of a virtual object corresponding to the user that is the subject of control within the image data, based on the angle information output by the sensor that represents the direction in which the user is located relative to the reference direction, which is determined in relation to the placement position of the display device.

5. A method for controlling an information processing device connected to a display device and a sensor for detecting a relative position between each of a plurality of users and the display device, comprising the steps of: the sensor outputs information on the relative position including information on a distance to a position where each user is located, the distance being determined in relation to an arrangement position of the display device; an acquisition means for acquiring information representing a relative position between each user and the display device detected by the sensor; a generation means for generating video data including a virtual object corresponding to each user, the generation means controls a position or orientation of the virtual object corresponding to each user within the video data as a control target, based on the acquired information representing a relative position between the corresponding user and the display device output by the sensor, the information being determined in relation to a placement position of the display device; A control method for an information processing device that outputs the generated video data to the display device.

6. An information processing device connected to a display device and a sensor for detecting a relative position between each of a plurality of users and the display device, means for acquiring information representing a relative position between each user and the display device detected by the sensor; a generation means for generating image data including a virtual object corresponding to each user, the generation means controlling a position or orientation of the virtual object corresponding to each user as a control target within the image data based on the acquired information representing a relative position between the corresponding user and the display device; means for outputting the generated image data to the display device; Function as a the sensor outputs information on the relative position including information on a distance to a position where each user is located, the distance being determined in relation to an arrangement position of the display device; When functioning as the generation means, the program controls the position or orientation of the virtual object corresponding to each user within the image data based on the acquired information on the distance of the corresponding user, which is determined in relation to the placement position of the display device.

Images