Information processing device, information processing system, and program

Abstract

The present invention relates to an information processing device, an information processing system, and a program capable of appropriately operating a GUI. An information processing apparatus according to the present technology includes a video generation unit that generates a video including a 3D object placed in a virtual space and a GUI having at least three surfaces. When the 3D object and the GUI overlap each other in the virtual space, the video generation unit generates a video containing the 3D object in which a part of the 3D object is cut along a surface of the GUI. This technology can be applied, for example, to an information processing device for controlling the display of a spatial reproduction display.

Description

Technical Field

[0001] The present technology relates to an information processing device, an information processing system, and a program, and more particularly, to an information processing device, an information processing system, and a program capable of appropriately operating a GUI. Background Art

[0002] Videos representing 3D objects having three-dimensional shapes are generally displayed. For example, Patent Document 1 describes a technology for displaying stereoscopic content arranged in a virtual space in order to achieve stereoscopic viewing.

[0003] Reference List

[0004] Patent Literature

[0005] Patent document 1: WO 2019 / 044188 A. Summary of the invention

[0006] Problems to be solved by the present invention

[0007] In the case of operating a 3D object itself, a graphical user interface (GUI) for controlling the display of a 3D object, etc., it is effective to use an input device that can be moved three-dimensionally in real space. However, the user needs to keep holding the input device in the air, which increases fatigue.

[0008] The present technology has been made in view of such circumstances, and can appropriately operate the GUI.

[0009] Solution to the problem

[0010] According to a first aspect of the present technology, an information processing device includes a video generation unit, which is configured to generate a video including a 3D object arranged in a virtual space and a GUI having at least three planes, the video including the 3D object, and when the 3D object and the GUI overlap in the virtual space, a portion of the 3D object is cut along the plane of the GUI.

[0011] A program according to a first aspect of the present technology causes a computer to perform the following processing: generating a video including a 3D object arranged in a virtual space and a GUI having at least three planes, the video including the 3D object, and in a case where the 3D object and the GUI overlap in the virtual space, cutting a portion of the 3D object along a plane of the GUI.

[0012] According to the second aspect of the present technology, an information processing system includes an information processing device, which includes: a video generation unit, which generates a video including a 3D object arranged in a virtual space and a GUI having at least three planes, the video including the 3D object, and when the 3D object and the GUI overlap in the virtual space, a part of the 3D object is cut along the plane of the GUI; and a display device, which is connected to the information processing device and includes a display unit, which displays the video generated by the video generation unit.

[0013] In a first aspect of the present technology, a video including a 3D object arranged in a virtual space and a GUI having at least three planes is generated, the video including the 3D object, and a portion of the 3D object is cut along a plane of the GUI when the 3D object and the GUI overlap in the virtual space.

[0014] In a second aspect of the present technology, a video including a 3D object arranged in a virtual space and a GUI having at least three planes is generated by an information processing device, the video including the 3D object, and in a case where the 3D object and the GUI overlap in the virtual space, a portion of the 3D object is cut along the plane of the GUI. In addition, the video generated by the information processing device is displayed by a display device connected to the information processing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram showing a configuration example of an information processing device according to an embodiment of the present technology.

[0016] Figure 2 is a diagram illustrating an example of a display unit.

[0017] Figure 3 is a diagram showing a first example of a 3D object and a GUI viewed from a user's viewpoint.

[0018] Figure 4 is a diagram for describing a method of cutting a 3D object using a trihedral corner cube.

[0019] Figure 5 is another diagram for describing a method of cutting a 3D object using a trihedral corner cube.

[0020] Figure 6 is a diagram illustrating an example of a cross-sectional portion of a 3D object in a case where highlight display is turned off.

[0021] Figure 7 is a diagram for describing a method of operating a trihedral corner cube.

[0022] Figure 8 is another diagram for describing a method of operating a trihedral corner cube.

[0023] Fig. 9 is a diagram showing a display example of a trihedral corner cube.

[0024] Fig.10 is a diagram illustrating an example of a UI plane.

[0025] Fig.11 is a diagram showing an arrangement example of a cursor.

[0026] Fig.12 is a diagram illustrating a second example of a 3D object and a GUI viewed from a user's viewpoint.

[0027] Fig.13 is a diagram for describing a method of cutting a 3D object using a cube.

[0028] Fig.14 is a diagram for describing a method of operating a cube.

[0029] Fig.15 is a flowchart describing a process performed by the information processing apparatus.

[0030] Fig.16 is a block diagram showing a configuration example of hardware of a computer. DETAILED DESCRIPTION

[0031] Hereinafter, a mode for carrying out the present technology will be described. The description is given in the following order.

[0032] 1. Configuration of information processing equipment

[0033] 2. Details of the method for displaying 3D objects and GUI

[0034] 3. Operation of information processing equipment

[0035] <1. Overview of Information Processing Equipment>

[0036] Figure 1 is a block diagram showing a configuration example of an information processing device 1 according to one embodiment of the present technology.

[0037] like Figure 1 As shown, the information processing device 1 includes a display control unit 11 , a display unit 12 , a sensor unit 13 , an input unit 14 , and a storage unit 15 .

[0038] The display control unit 11 controls the display of the display unit 12. The display control unit 11 also functions as a video generation unit that generates a video including a 3D object arranged in a virtual space and a GUI for controlling the 3D object. The display control unit 11 generates a video based on the tracking of the viewpoint position of the user. The display control unit 11 controls the display unit 12 to display the generated video. For example, the data of the 3D object and the GUI are acquired from the storage unit 15.

[0039] The display unit 12 displays a video under the control of the display control unit 11. The display unit 12 is constituted by, for example, a spatial reproduction display capable of performing a stereoscopic display by binocular parallax by presenting different videos to the left and right eyes of the user. The display unit 12 may include a lenticular lens array or a parallax barrier for presenting different videos to the left and right eyes of the user, or may support a stereoscopic display using a liquid crystal shutter or active delay glasses.

[0040] The sensor unit 13 acquires (senses) various types of information about the user or the surrounding environment. The sensor unit 13 includes at least a sensor for detecting the viewpoint position of the user. In the sensor unit 13, the sensor unit 13 itself can detect the viewpoint position of the user and provide information indicating the viewpoint position of the user to the display control unit 11, or can provide information for detecting the viewpoint position of the user to the display control unit 11.

[0041] For example, in the case where the information processing device 1 is installed in the real space, the sensor for detecting the viewpoint position of the user may be a camera that images the user. In the case where the information processing device 1 is a device that moves by being worn by the user, the sensor for detecting the viewpoint position of the user may be any one of an acceleration sensor, a gyroscope sensor, an orientation sensor, and a camera, or a combination thereof. In addition, the sensor unit 13 may include a camera that is arranged to image an area corresponding to the user's field of view in the real space.

[0042] The input unit 14 accepts input of the user's operation and provides information indicating the content of the user's operation to the display control unit 11. The input unit 14 is composed of, for example, a mouse or a trackball capable of inputting operations in two-dimensional directions. In addition, the input unit 14 may be composed of a touch panel that accepts a touch operation of the user, and in this case, the display unit 12 may have the function of the input unit 14.

[0043] The storage unit 15 stores programs and parameters for the display control unit 11 to perform functions. In addition, the storage unit 15 may store data of a 3D object or a GUI displayed by the display control unit 11.

[0044] Notice, Figure 1The configuration shown is an example, and the present technology is not limited to such an example. For example, the display control unit 11 can control the display of a display unit provided outside the information processing device 1. In addition, the display control unit 11 can acquire data of a 3D object from another device via a communication unit (not shown). The information processing device 1 may not include a sensor unit, and the above-mentioned information may be sent to the information processing device 1 from a sensor installed outside the information processing device 1. In addition, for example, the information processing device 1 may have a configuration in which each of a PC monitor (display unit 12), a camera (sensor unit 13), and a mouse (input unit 14) is connected to a system of a personal computer having a display control unit 11 and a storage unit 15.

[0045] Figure 2 is a diagram showing an example of the display unit 12 .

[0046] like Figure 2 As shown, for example, the display unit 12 is arranged so that the display surface is tilted upward relative to the horizontal plane in the real space. In this case, for example, the display control unit 11 arranges the virtual space VW to intersect with the display surface of the display unit 12, in other words, when viewed from the user's viewpoint E, to have areas on both the near side and the far side of the display unit 12, and causes the display unit 12 to display the virtual space VW.

[0047] For example, a 3D object arranged in the virtual space VW is displayed so that a portion appears to pop out from the display unit 12. For example, the display unit 12 may represent virtual depth by displaying a stereoscopic video in which different videos are presented to the left and right eyes of the user, and provide the user with a feeling as if the 3D object exists in the real space.

[0048] Note that the present technology can be applied not only to devices that display stereoscopic video, but also to devices that display videos in which the same video is presented to the left and right eyes. For example, 3D objects or GUIs can be arranged in a virtual space VW displayed on a normal PC monitor that does not support stereoscopic display.

[0049] <2. Details of method of displaying 3D objects and GUI>

[0050] First display method

[0051] Figure 3 is a diagram showing a first example of a 3D object and a GUI viewed from a user's viewpoint.

[0052] exist Figure 3In the first display method shown, the 3D object Obj, the three-sided corner cube C1 , the cursor Cur, the menu bar B1 , and the slide bar B2 are arranged in a virtual space displayed on the display unit 12 .

[0053] The 3D object Obj is, for example, a 3D model of an object having a three-dimensional shape, and has not only a surface shape but also an internal shape. The 3D object Obj is, for example, a 3D model indicating the structure of a human body acquired by scanning the human body. The trihedral cube C1 is a GUI for setting a cutting position of a part of the 3D object Obj, and is composed of three planes whose two sides are shared with other planes.

[0054] The cursor Cur is a cursor that moves according to the user's operation input unit 14. Icons indicating various functions related to the display of 3D objects are displayed on the menu bar B1. For example, the user can make the information processing device 1 execute various functions by moving the cursor Cur to an icon on the menu bar B1 and selecting the icon.

[0055] The slide bar B2 is used, for example, to set the size of the 3D object Obj, and the user can set the 3D object Obj to a desired size by determining a position that the operator can move along the slide bar B2.

[0056] Figure 4 and Figure 5 is a diagram describing a method of cutting a 3D object Obj using a trihedral corner cube C1.

[0057] like Figure 4 As shown, in the case where the 3D object Obj and the three-sided corner cube C1 overlap, the portion of the 3D object Obj included in the area on the inner side (concave side) of the three-sided corner cube C1 is cut. In this case, the portion of the 3D object Obj included in the area on the outer side (convex side) of the three-sided corner cube C1 is displayed.

[0058] Specifically, Figure 5 As shown by the thick gray lines in , a part of the 3D object Obj is cut along each of the three planes of the trihedral corner cube C1. By determining the position of the trihedral corner cube C1, the user can set the cutting position of the 3D object Obj and observe the internal shape of the 3D object Obj. In the information processing device 1, since the stereoscopic video is displayed to reproduce the binocular parallax, the user can easily grasp the shape behind the cross-sectional part. The user's recognition accuracy is improved by cutting off unnecessary parts. Hereinafter, the technology of displaying a part of the 3D object Obj in a cut state using the trihedral corner cube C1 is referred to as three-sided clipping.

[0059] For example, when the shape behind the cross-sectional part is transparently displayed in the 3D object Obj, the cross-sectional part and the other shapes are assimilated, and it may be difficult for the user to distinguish the cross-sectional part from the other shapes. In this case, in order to make the cross-sectional part clear, such as Figure 4 and Figure 5 As shown, for example, a cross-sectional portion along a plane of the trihedral corner cube C1 is displayed in a highlighted manner, such as by emitting light or coloring only the cross-sectional portion.

[0060] Note that, in the case where the user wants to view the internal shape of the 3D object Obj in detail, Figure 6 As shown, the highlight display of the cross-sectional portion of the 3D object Obj may also be turned off. As described above, the information processing device 1 may have a function of turning on / off the highlight display of the cross-sectional portion of the 3D object Obj.

[0061] Figure 7 and Figure 8 is a diagram for describing a method of operating a trihedral corner cube C1.

[0062] Hereinafter, it will be described assuming that a trihedral corner cube C1 is constituted by planes P1 to P3 in a space represented by three axes of XYZ. Note that plane P1 indicates a plane corresponding to an XY plane parallel to the floor surface on which the display unit 12 is installed (on the XY plane), and plane P2 indicates a plane corresponding to an XZ plane orthogonal to the XY plane (on the XZ plane). Plane P3 indicates a plane corresponding to a YZ plane orthogonal to the XY plane and the XZ plane (on the YZ plane). In the trihedral corner cube C1, a state that is sufficiently small relative to the 3D object Obj is set as an initial state.

[0063] like Figure 7 As shown by the hollow arrow in , when the cursor Cur moves from a position other than the plane of the three-sided corner cube C1 to the plane, the information processing device 1 enters the three-sided clipping operation mode. In the three-sided clipping operation mode, the cursor Cur moves on the plane below the cursor Cur (any one of the XY plane, XZ plane and YZ plane).

[0064] During the three-sided clipping operation mode, the user can move the three-sided corner cube C1 and change the cutting position of the 3D object Obj by moving the mouse (cursor Cur) while performing a first operation (e.g., pressing the left button of the mouse) serving as the input unit 14. The first operation is, for example, a selection operation of selecting one of the planes constituting the three-sided corner cube C1.

[0065] like Figure 8As shown in A, when the cursor Cur is on the plane P1, the Z value of the trihedral cube C1 is fixed, and the trihedral cube C1 moves so as to slide on the XY plane, as shown by the hollow arrow. Similarly, when the cursor Cur is on the plane P2, the Y value of the trihedral cube C1 is fixed, and the trihedral cube C1 moves so as to slide on the XZ plane. Furthermore, when the cursor Cur is on the plane P3, the X value of the trihedral cube C1 is fixed, and the trihedral cube C1 moves so as to slide on the YZ plane.

[0066] During the three-sided clipping operation mode, the user can rotate the three-sided corner cube C1 and change the cutting direction of the 3D object Obj by moving the mouse (cursor Cur) while performing a second operation (e.g., pressing the right button of the mouse). The second operation is, for example, a selection operation of selecting one of the planes constituting the three-sided corner cube C1.

[0067] like Figure 8 As shown in FIG. 1B , when the cursor Cur is on the plane P1, the trihedral cube C1 rotates on the XY plane around the Z axis of the trihedral cube C1 (e.g., the side shared by the planes P2 and P3). When the right button of the mouse is pressed, the cursor Cur moves on the plane (XY plane) perpendicular to the rotation axis to rotate around the rotation axis (Z axis), as shown in FIG. Figure 8 As shown by the arrow in B. When the cursor rotates around the rotation axis in a state where the angle θ formed by the straight line connecting the rotation axis and the cursor Cur and the plane P3 is fixed, the trihedral corner cube C1 rotates similarly to the cursor Cur.

[0068] Similarly, when the cursor Cur is on the plane P2, the trihedral cube C1 rotates on the XZ plane around the Y axis of the trihedral cube C1 (e.g., the edge shared by the plane P1 and the plane P3). In addition, when the cursor Cur is on the plane P3, the trihedral cube C1 rotates on the YZ plane around the X axis of the trihedral cube C1 (e.g., the edge shared by the plane P1 and the plane P2).

[0069] When the horizontal movement amount of the mouse is converted into the rotation angle of the three-sided corner cube C1, it is assumed that the cursor deviates from the plane of the three-sided corner cube C1, and the operation feeling is deteriorated. For example, in the case where the cursor is on the plane P1, when the right button of the mouse is pressed, the arc movement amount of the mouse is made to correspond to the rotation amount of the three-sided corner cube C1 on the XY plane, thereby performing an intuitive rotation operation. In addition, while the right button of the mouse is pressed, the cursor Cur can be prevented from deviating from the plane of the three-sided corner cube C1.

[0070] As described above, the GUI such as the three-sided corner cube C1 can change at least one of the position and direction by the user's operation in the two-dimensional direction. Note that, for example, when the cursor Cur is on the plane P1, when a predetermined operation (pressing the left and right buttons of the mouse, operation of the mouse wheel, etc.) is performed, the shape of the three-sided corner cube C1 can be changed by the operation in the two-dimensional direction, so that at least one of the plane P2 and the plane P3 is deformed.

[0071] Fig. 9 is a diagram showing a display example of a trihedral corner cube C1.

[0072] like Fig. 9 As shown in A of FIG. 1 , when the cursor Cur is not on any of the planes P1 to P3, the planes P1 to P3 are displayed in a transparent state, for example. Fig. 9 As shown in B, when the cursor Cur is on the plane P1, the plane P1 below the cursor Cur is displayed in a color state, for example, and the planes P2 and P3 are displayed in a transparent state, for example.

[0073] like Fig. 9 As shown in C, when the cursor Cur is on the plane P2, the plane P2 below the cursor Cur is displayed in color, for example, and the planes P1 and P3 are displayed in a transparent state, for example. Fig. 9 As shown in D, when the cursor Cur is on the plane P3, the plane P3 below the cursor Cur is displayed in a color state, for example, and the planes P1 and P2 are displayed in a transparent state, for example.

[0074] As described above, the color clearly indicates which plane of the trihedral cube C1 the cursor Cur is on. Note that the color can be used to clearly indicate which plane of the trihedral cube C1 the cursor Cur is on, not only by the color but also by changing the luminance (brightness) or pattern.

[0075] Note that the cursor Cur can be freely moved in the virtual space displayed on the display unit 12. Usually, the cursor is drawn to move on the display surface of the display device. In the present technology, the cursor Cur is drawn to move basically on a two-dimensional UI plane arranged in the virtual space. In the present technology, for example, the cursor Cur is virtually arranged at a position where a straight line connecting the user's viewpoint position and the cursor position usually displayed on the display device intersects with the UI plane arranged in the virtual space.

[0076] Fig.10 is a diagram illustrating an example of a UI plane.

[0077] exist Fig.10In the example of , the UI plane P11 on which the cursor Cur is arranged is the frontmost plane of the virtual space and is arranged as a plane perpendicular to the XY plane. For example, other GUIs such as a menu bar B1 and a slide bar B2 are also drawn on the UI plane P11 together with the cursor Cur.

[0078] Fig.11 : is a diagram showing an arrangement example of the cursor Cur.

[0079] At normal time (when the cursor Cur is not on the plane of the trihedral cube), the cursor Cur is arranged on the UI plane P11, as shown in FIG. Fig.11 The cursor Cur for the right eye is drawn on a straight line connecting the position of the user's right eye and the position of the cursor normally displayed on the display device, and the cursor Cur for the left eye is drawn on a straight line connecting the position of the user's left eye and the position of the cursor normally displayed on the display device, so that the user feels that the cursor Cur is arranged on the UI plane P11 in the air.

[0080] In the case where the cursor Cur enters the area connecting the position of the user's viewpoint E and the trihedral corner cube C1, the cursor Cur is arranged on the plane of the trihedral corner cube C1, as shown in FIG. Fig.11 As shown in #4 to #6 in .

[0081] As described above, in the case where the cursor Cur is not in the area connecting the position of the user's viewpoint E and the object (e.g., 3D object or GUI) arranged in the virtual space, the display control unit 11 controls the cursor Cur to move along a predetermined plane arranged in the virtual space. In addition, in the case where the cursor Cur enters the area connecting the position of the user's viewpoint E and the object arranged in the virtual space, the display control unit 11 moves the cursor Cur from the predetermined plane to the surface of the object, and controls the cursor Cur to move along the surface of the object.

[0082] Second display method

[0083] Fig.12 is a diagram illustrating a second example of a 3D object and a GUI viewed from a user's viewpoint.

[0084] In the first display method ( Figure 3 ), the 3D object Obj, the three-sided corner cube C1, the cursor Cur, the menu bar B1 and the slide bar B2 are arranged in the virtual space displayed by the display unit 12. Fig.12 In the second display method shown, a cube C51 is arranged in the virtual space instead of the three-sided corner cube C1.

[0085] The cube C51 is a GUI for setting a cutting position of a part of the 3D object Obj, and is a GUI having a substantially box-like shape composed of six planes.

[0086] Fig.13 is a diagram describing a method of cutting a 3D object Obj using a cube C51.

[0087] In the case where the 3D object Obj and the cube C51 overlap, as Fig.13 As shown in A, the portion of the 3D object Obj included in the area outside the cube C51 is cut. In this case, the portion of the 3D object Obj included in the area inside the trihedral cube C1 is displayed.

[0088] By determining the position of the cube C51, the user can set the cutting position of the 3D object Obj and observe the internal shape of the 3D object Obj. Hereinafter, the technology of displaying a part of the 3D object Obj in a cut state using the cube C51 is referred to as six-sided clipping. In the case where the 3D object Obj is, for example, a 3D model indicating the structure of a human body, since a part of the outer side of the cube C51 is cut with six-sided clipping, the 3D model of a small tissue on the inner side of the cube C51 can be observed in a greatly magnified manner at a pin point.

[0089] Note that similarly to the three-sided clip, also in the six-sided clip, the cross-sectional portion is displayed in a highlighted manner, for example, by displaying the cross-sectional portion only by emitting light or coloring, so as to make the cross-sectional portion of the 3D object Obj clear.

[0090] Hereinafter, description will be made assuming that the cube C51 is a rectangular parallelepiped surrounded by an upper surface, a lower surface, and four side surfaces. Note that the upper surface and the lower surface of the cube C51 indicate planes corresponding to the XY plane parallel to the floor surface on which the display unit 12 is mounted (on the XY plane), and the two opposite side surfaces indicate planes corresponding to the XZ plane orthogonal to the XY plane (on the XZ plane). The other two side surfaces indicate planes corresponding to the YZ plane orthogonal to the XY plane and the XZ plane (on the YZ plane).

[0091] When the cursor Cur moves from a position other than the position on the plane of the cube C51 to the plane, the information processing device 1 enters the six-sided clipping operation mode. During the six-sided clipping operation mode, the cursor Cur moves on the plane below the cursor Cur (the upper surface, lower surface or side surface of the cube C51).

[0092] During the six-sided clipping operation mode, the user can change the size of the cube C51 by performing a third operation (e.g., operation of the mouse wheel). Fig.13As shown in FIG. 1B , by operating the mouse wheel, the cube C51 becomes smaller, and the user can further observe the interior of the 3D object Obj. In addition, by further operating the mouse wheel, for example, Fig.13 As shown in FIG. 3C , the cube C51 becomes smaller, and the user can further observe the interior of the 3D object Obj.

[0093] Note that during the six-sided clipping operation mode, the user can also zoom in on the cube C51 by operating the scroll wheel of the mouse.

[0094] Fig.14 is a diagram for describing the operating method of the cube C51.

[0095] During the six-sided clipping operation mode, the user can rotate the cube C51 and change the cutting direction of the 3D object Obj by moving the mouse (cursor) while performing the second operation (eg, pressing the right button of the mouse).

[0096] When the cursor is on the upper surface of the cube C51, the cube C51 rotates on the XY plane around the axis A1 perpendicular to the upper surface, as shown in FIG. Fig.14 As shown by arrow A in . Similarly, when the cursor is on the side surface of the cube C51, the cube C51 rotates on the XZ plane or the YZ plane around an axis perpendicular to the side surface.

[0097] When the horizontal movement amount of the mouse is converted into the rotation angle of the cube C51, it is assumed that the cursor will deviate from the plane of the cube C51, and the operation feeling will be deteriorated. For example, in the case where the cursor is on the upper surface of the cube C51, while pressing the right button of the mouse, the arc movement amount of the mouse is made to correspond to the rotation amount of the cube C51 on the XY plane, thereby performing an intuitive rotation operation. In addition, while pressing the right button of the mouse, the cursor can be prevented from deviating from the plane of the cube C51.

[0098] During the six-sided clipping operation mode, the user can move the cube C51 and change the cutting position of the 3D object Obj by moving the mouse (cursor) while performing a first operation (eg, pressing the left button of the mouse), as shown in FIG. Fig.14 Indicated by the hollow arrow in B.

[0099] When the cursor is on the upper surface of the cube C51, the Z value of the cube C51 is fixed, and the cube C51 is moved to slide on the XY plane. Similarly, when the cursor is on the side, the Y value or Z value of the cube C51 is fixed, and the cube C51 is moved to slide on the XZ plane or the YZ plane.

[0100] Similar to the three-sided cube C1, in the cube C51, the plane below the cursor is also displayed in color, for example. In addition, in the six-sided clip, the cursor is drawn to move basically on the UI plane arranged in the virtual space, and when the cursor enters the area connecting the user's viewpoint position and the cube C51, the cursor is drawn to move on the plane of the cube C51.

[0101] Note that, in a three-sided clip or a six-sided clip, in a case where the cursor is not on the three-sided corner cube C1 or the cube C51, the information processing apparatus 1 enters the 3D object operation mode.

[0102] During the 3D object operation mode, the user can move the 3D object by moving the mouse while pressing the left button of the mouse. In addition, during the 3D object operation mode, the user can rotate the 3D object by moving the mouse while pressing the right button of the mouse. In addition, during the 3D object operation mode, the user can change the size of the 3D object by operating the scroll wheel of the mouse.

[0103] As described above, in the information processing device 1 of the present technology, a portion (plane) constituting the GUI (such as the trihedral cube C1 and the cube C51) is used as a mark for switching the plane on which the GUI slides (control plane) and the rotation axis of the GUI. The user can intuitively perform six-axis operations on the GUI by specifying the plane constituting the GUI using an input device (input unit 14) operable in two-dimensional directions, and switching the control plane and the rotation axis.

[0104] Generally, when operating a 3D object itself, a GUI for controlling the display of a 3D object, etc., it is effective to use an input device that can be moved in three dimensions in real space. However, the user needs to keep holding the input device in the air, which increases fatigue.

[0105] In the information processing device 1 of the present technology, the GUI can be operated spatially using a mouse or the like that is usually used on a table. The user can operate the GUI by moving a familiar mouse or the like on the table, which is less tiring. Therefore, the user can appropriately operate the GUI for controlling the display of a 3D object.

[0106] In addition, in the case where a 3D model of an object having an internal shape is stereoscopically displayed and the user wants to see the internal shape of the object, generally, a process of making a part of the 3D model semi-transparent or cutting it at a predetermined plane to display a cross section is performed. In this case, it takes time and effort to specify the part to be semi-transparent, or the range that can be confirmed is limited to the cross-sectional part.

[0107] In the information processing device 1 of the present technology, since the 3D object is cut at at least three planes, it is relatively easy to specify the area to be cut, and the portion to be viewed can be limited more accurately.

[0108] In the information processing device 1 of the present technology, stereoscopic vision enabling motion parallax representation is achieved based on tracking of the user's viewpoint position. The cross-sectional portion of the 3D object and the structure near the cross-sectional portion are separated and recognized in the user's brain through feedback (such as position changes due to movement, shaking, vibration, etc. of the head and video changes relative to the position changes). Although the easily separated area (depth) is narrow, in the present technology, the portion of the 3D object included in the area up to the front of the gaze area is cut, and therefore the spatial structure can be easily grasped. By displaying the cross-sectional portion in a highlighted manner, there is also an effect of identifying the cross-sectional portion and making the structure behind the cross-sectional portion easy to understand.

[0109] The user can understand the structure of the 3D object in more detail by dynamically changing the cross section of the 3D object by moving the GUI using a mouse, etc. In other words, the information processing device 1 according to the present technology can realize real-time feedback that presents a video corresponding to the user's operation (participation).

[0110] <3. Operation of information processing equipment>

[0111] Next, we will refer to Fig.15 The flowchart describes the processing performed by the information processing device 1 having the aforementioned configuration.

[0112] In step S1 , the sensor unit 13 detects the viewpoint position of the user.

[0113] In step S2 , the input unit 14 receives an input of a user's operation.

[0114] In step S3 , the display control unit 11 controls the position of the cursor and the position and direction of the GUI according to the user's operation.

[0115] In step S4, the display control unit 11 draws the 3D object, cursor, GUI, etc., and generates a video. Specifically, the display control unit 11 determines the cutting position and cutting direction of the 3D object based on the position and direction of the GUI, and hides or deletes the polygons, point group data, textures, etc. of the 3D object along the plane of the GUI. The display control unit 11 transmits the cut part and depicts the inside of the 3D object.

[0116] By hiding or deleting a portion of a 3D object, the required performance of the information processing device 1 that performs rendering can be reduced compared to when the relevant portion is made semi-transparent. In particular, in six-sided clipping, a 3D object can be expressed more accurately by densely rendering a small area.

[0117] The video generated by the display control unit 11 is an image in which an object such as a 3D object or a GUI arranged in a virtual space is perspectively projected onto the display surface of the display unit 12 based on the viewpoint position of the user. In the case where the video is a stereoscopic video, the same object appears in different positions and shapes when viewed from the right viewpoint position and the left viewpoint position. Therefore, when the video visually recognized by the left eye and the right eye is synthesized in the brain, the display control unit 11 performs arrangement processing and deformation processing of the object in the video so that the same object is recognized as the same position, the same shape, and the same size.

[0118] In step S5 , the display unit 12 displays the video generated by the display control unit 11 .

[0119] As described above, in the information processing device 1, when a 3D object and a GUI overlap in a virtual space, a video including a 3D object arranged in a virtual space and a GUI having at least three planes is generated and displayed, the video including the 3D object in which a portion is cut along a plane of the GUI.

[0120] The user can appropriately operate the GUI for designating a cutting position of the 3D object, and observe the internal shape of the 3D object.

[0121] Note that the method of expressing a 3D object or a GUI in the present technology is realized, for example, by a software application, a plug-in tool, a library, or the like executed in an information processing device that controls display of a viewpoint-following-type spatial reproduction display.

[0122] <About Computer>

[0123] The above-described series of processing may be executed by hardware or may be executed by software. In the case where a series of processing steps are executed by software, a program included in the software is installed from a program recording medium on a computer incorporated into dedicated hardware, a general-purpose personal computer, or the like.

[0124] Fig.16 : is a block diagram showing a configuration example of hardware of a computer that executes the above-described series of processes by a program.

[0125] A central processing unit (CPU) 501 , a read only memory (ROM) 502 , and a random access memory (RAM) 503 are connected to one another via a bus 504 .

[0126] An input / output interface 505 is further connected to the bus 504. An input unit 506 including a keyboard, a mouse, etc. and an output unit 507 including a display, a speaker, etc. are connected to the input / output interface 505. In addition, a storage unit 508 including a hard disk, a nonvolatile memory, etc., a communication unit 509 including a network interface, etc., and a drive 510 that drives a removable medium 511 are connected to the input / output interface 505.

[0127] In the computer configured as described above, for example, the CPU 501 loads a program stored in the storage unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 , and executes the program to perform the above-described series of processing.

[0128] For example, the program executed by the CPU 501 is recorded in the removable medium 511 , or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and then installed in the storage unit 508 .

[0129] The program executed by the computer may be a program in which processes are executed in time series in the order described in this specification, or may be a program in which processes are executed in parallel or at necessary timing such as when a call is made.

[0130] Note that in this specification, a system refers to an assembly of multiple components (devices, modules (parts), etc.), and it is not important whether all the components are located in the same housing. Therefore, multiple devices housed in separate housings and connected to each other via a network and a device in which multiple modules are housed in one housing are both systems.

[0131] The effects described in this specification are merely examples and are not limiting, and other effects may be provided.

[0132] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made without departing from the scope of the present technology.

[0133] For example, the present technology may be configured as cloud computing in which one function is shared and jointly processed by a plurality of devices via a network.

[0134] Furthermore, each step described in the above flowchart may be performed by one device or may be shared and performed by a plurality of devices.

[0135] Furthermore, in the case where one step includes a plurality of processes, the plurality of processes included in one step may be executed by one device or may be shared and executed by a plurality of devices.

[0136] <Configuration combination example>

[0137] The present technology may also be configured as follows. (1)

[0139] An information processing device includes: a video generation unit configured to generate a video including a 3D object arranged in a virtual space and a GUI having at least three planes, the video including the 3D object, and when the 3D object and the GUI overlap in the virtual space, a part of the 3D object is cut along the plane of the GUI. (2)

[0141] The information processing device according to (1), wherein the video generation unit changes at least one of a position, a direction, and a shape of the GUI based on an input operation in two-dimensional directions. (3)

[0143] The information processing device according to (1) or (2), wherein, in a case where the cursor is on the plane of the GUI, the video generation unit moves the cursor on the plane of the GUI. (4)

[0145] The information processing device according to (3), wherein the video generation unit moves the GUI on a plane corresponding to the one plane of the GUI in response to an operation of moving the cursor in a state in which the one plane of the GUI is selected. (5)

[0147] The information processing device according to (3) or (4), wherein the video generation unit rotates the GUI on a plane corresponding to the one plane of the GUI in response to an operation of moving the cursor in a state in which the one plane of the GUI is selected. (6)

[0149] The information processing device according to any one of (3) to (5), wherein the video generation unit changes the size of the GUI according to a predetermined operation in a state where the cursor is on a plane of the GUI. (7)

[0151] The information processing device according to any one of (3) to (6), wherein the video generation unit deforms the GUI according to a predetermined operation in a state where the cursor is on a plane of the GUI. (8)

[0153] The information processing device according to any one of (3) to (7), wherein, in a case where the cursor is not on a plane of the GUI, the video generation unit moves the cursor on a plane of the UI arranged in the virtual space. (9)

[0155] The information processing device according to (8), wherein, when the cursor enters a region connecting a viewpoint position of a user and a 3D object, the video generation unit moves the cursor from a plane of the UI to a plane of the GUI in a virtual space. (10)

[0157] The information processing device according to (8) or (9), wherein the video generated by the video generating unit includes another GUI arranged on the UI plane in the virtual space. (11)

[0159] The information processing device according to (3), wherein the video generation unit colors a plane of the GUI below the cursor. (12)

[0161] The information processing device according to any one of (1) to (11), wherein the GUI includes at least three planes sharing both sides with other planes. (13)

[0163] The information processing apparatus according to any one of (1) to (11), wherein the GUI has a substantially box-like shape surrounded by six planes. (14)

[0165] The information processing device according to any one of (1) to (13), wherein the video generation unit cuts out a portion of the 3D object included in a region on the inner side of the GUI. (15)

[0167] The information processing device according to any one of (1) to (13), wherein the video generation unit cuts out a portion of the 3D object included in a region outside the GUI. (16)

[0169] The information processing device according to any one of (1) to (15), wherein the video generation unit generates a video in which a cross-sectional portion of the 3D object along a plane of the GUI is highlighted. (17)

[0171] The information processing device according to any one of (1) to (16), wherein the video generating unit generates the stereoscopic video by binocular parallax. (18)

[0173] The information processing device according to any one of (1) to (17), wherein the video generation unit generates the video based on tracking of a viewpoint position of a user. (19)

[0175] An information processing system, comprising: an information processing device, comprising a video generating unit, the video generating unit generating a video including a 3D object arranged in a virtual space and a GUI having at least three planes, the video including the 3D object, and in a case where the 3D object and the GUI overlap in the virtual space, cutting a part of the 3D object along a plane of the GUI; and

[0176] The display device is connected to the information processing apparatus and includes a display unit that displays the video generated by the video generating unit. (20)

[0178] A program that causes a computer to:

[0179] A video including a 3D object and a GUI having at least three planes arranged in a virtual space is generated, the video including the 3D object, and a portion of the 3D object is cut along a plane of the GUI in a case where the 3D object and the GUI overlap in the virtual space. (twenty one)

[0181] The information processing device according to (18), wherein the image generation unit generates the video by perspective-projecting the 3D object and the GUI based on the viewpoint position.

[0182] Reference Mark List

[0183] 1Information processing equipment

[0184] 11Display control unit

[0185] 12 display units

[0186] 13 sensor units

[0187] 14 Input Units

[0188] 15 storage units.

Claims

1. An information processing device, comprising: A video generation unit is configured to generate a video including a 3D object arranged in a virtual space and a GUI having at least three planes, the video including the 3D object, and when the 3D object and the GUI overlap in the virtual space, a portion of the 3D object is cut along the plane of the GUI.

2. The information processing device according to claim 1, wherein: The video generation unit changes at least one of a position, a direction, and a shape of the GUI based on an input operation in two-dimensional directions.

3. The information processing device according to claim 1, wherein: In a case where a cursor is on a plane of the GUI, the video generation unit moves the cursor on the plane of the GUI.

4. The information processing device according to claim 3, wherein: The video generation unit moves the GUI on a plane corresponding to the one plane of the GUI in response to an operation of moving the cursor in a state in which the one plane of the GUI is selected.

5. The information processing device according to claim 3, wherein: The video generation unit rotates the GUI on a plane corresponding to the one plane of the GUI in response to an operation of moving the cursor in a state in which the one plane of the GUI is selected.

6. The information processing device according to claim 3, wherein: The video generation unit changes the size of the GUI according to a predetermined operation in a state where the cursor is on a plane of the GUI.

7. The information processing device according to claim 3, wherein: The video generation unit deforms the GUI according to a predetermined operation in a state where the cursor is on a plane of the GUI.

8. The information processing device according to claim 3, wherein: In a case where the cursor is not on a plane of the GUI, the video generation unit moves the cursor on a UI plane arranged in the virtual space.

9. The information processing device according to claim 8, wherein: In a case where the cursor enters a region connecting a viewpoint position of a user and the 3D object, the video generation unit moves the cursor from the UI plane to a plane of the GUI in the virtual space.

10. The information processing device according to claim 8, wherein: The video generated by the video generation unit includes another GUI arranged on the UI plane in the virtual space.

11. The information processing device according to claim 3, wherein: The video generation unit colors a plane of the GUI below the cursor.

12. The information processing device according to claim 1, wherein: The GUI includes at least three planes that share two sides with other planes.

13. The information processing device according to claim 1, wherein: The GUI has a generally box-like shape surrounded by six planes.

14. The information processing device according to claim 1, wherein: The video generation unit cuts out a portion of the 3D object included in an area inside the GUI.

15. The information processing device according to claim 1, wherein: The video generation unit cuts out a portion of the 3D object included in an area outside the GUI.

16. The information processing device according to claim 1, wherein: The video generation unit generates a video in which a cross-sectional portion of the 3D object along the plane of the GUI is highlighted.

17. The information processing device according to claim 1, wherein: The video generating unit generates a stereoscopic video through binocular parallax.

18. The information processing device according to claim 1, wherein: The video generation unit generates a video based on tracking of a viewpoint position of a user.

19. An information processing system comprising: An information processing device, comprising a video generating unit, the video generating unit generating a video including a 3D object and a GUI having at least three planes arranged in a virtual space, the video including the 3D object, and in a case where the 3D object and the GUI overlap in the virtual space, cutting a part of the 3D object along a plane of the GUI; as well as A display device is connected to the information processing apparatus and includes a display unit that displays the video generated by the video generating unit.

20. A program that causes a computer to execute the following processing: A video including a 3D object and a GUI having at least three planes arranged in a virtual space is generated, the video including the 3D object, and a portion of the 3D object is cut along a plane of the GUI when the 3D object and the GUI overlap in the virtual space.

Images