computer

Abstract

To allow 2D input of a 3D object in a virtual reality space with the same use feeling as that when a drawing operation is carried out using a so-called plate tablet.SOLUTION: A computer according to the present invention is a computer having a control unit for rendering an object in a virtual space. The control unit executes a rendering process for rendering, in a virtual space, a plane image indicating a plane in a real space used when a position instructed by a stylus is input, a rendering process for rendering, in the virtual space, a display image as a display image for displaying the object and different from the plane image, and a display process for displaying, in the display image, an object generated based on the position instructed by the stylus on the plane.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a rendering apparatus and a rendering method for rendering a 3D object in a virtual reality (including VR: Virtual Reality, AR: Augmented Reality, MR: Mixed Reality) space.

Background Art

[0002] In recent years, there has been an increasing need to design various products while stereoscopically viewing in a virtual reality space.

[0003] Patent Document 1 discloses a technique for generating a 3D object based on a 2D object input to a tablet terminal using an electronic pen in an AR space. Hereinafter, an input method performed by moving an electronic pen on a plane is referred to as "2D input".

[0004] Non-Patent Document 1 discloses a technique for inputting a 3D object by moving a 3D input controller in a VR space. Hereinafter, an input method using such a 3D input controller is referred to as "3D input".

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] As described above, 2D input allows for higher accuracy compared to 3D input because the position of the electronic pen is fixed within a known plane. However, 2D input has the problem that it is less intuitive to use than 3D input because it is limited to drawing within a plane.

[0008] In contrast, 3D input allows for intuitive operation. However, the high degree of freedom in controller positioning means that the technology is not precise enough for design purposes.

[0009] Therefore, one of the objectives of the present invention is to provide a rendering apparatus and rendering method that can achieve intuitive and highly accurate rendering in a virtual reality space. [Means for solving the problem]

[0010] The first aspect of the present invention relates to a rendering device for rendering 3D objects in a virtual reality space displayed on a virtual reality display, and the rendering device performs the following steps: a 3D rendering step of rendering the 3D object as a 3D object for 3D display in a virtual reality space coordinate system; a 2D rendering step of rendering the 3D object as a 3D object for 2D display in a planar coordinate system; a display update step having a 3D display mode that updates the display on the virtual reality display based on the rendering result of the 3D rendering step; and a 2D display mode that updates the display on the virtual reality display based on the rendering result of the 2D rendering step.

[0011] A second aspect of the present invention relates to a rendering method for causing a computer to function as a rendering device that renders 3D objects in a virtual reality space displayed on a virtual reality display, the rendering method causing the computer to perform a 3D rendering step of rendering the 3D object as a 3D object for 3D display in a virtual reality space coordinate system, a 2D rendering step of rendering the 3D object as a 3D object for 2D display in a planar coordinate system, a display update step having a 3D display mode that updates the display on the virtual reality display based on the rendering result of the 3D rendering step, and a 2D display mode that updates the display on the virtual reality display based on the rendering result of the 2D rendering step. [Effects of the Invention]

[0012] According to the present invention, the display method of 3D objects in a virtual reality space (3D display or 2D display) can be switched according to the user's selection, making it possible to achieve intuitive and highly accurate rendering in a virtual reality space. [Brief explanation of the drawing]

[0013] [Figure 1] This figure shows the configuration of a 3D object rendering system 1 according to a first embodiment of the present invention. [Figure 2] This diagram illustrates the relationship between the tablet surface coordinate system and the virtual reality space coordinate system. [Figure 3] Figure 1 is a flowchart showing the processes performed by the control unit 2a. [Figure 4] Figure 2 is a flowchart showing the details of the location information acquisition process. [Figure 5] Figure 2 is a flowchart illustrating the details of the tablet device display process. [Figure 6] This figure shows a state in which a 3D object is being input as a 2D object in a virtual reality space according to the first embodiment of the present invention. [Figure 7]This is a diagram showing a state in which a 3D object is being 3D-input in a virtual reality space according to the first embodiment of the present invention. [Figure 8] This is a diagram showing a state in which a 3D object is being 2D-input in a virtual reality space according to the second embodiment of the present invention. [Figure 9] This is a flowchart showing a part of the processing performed by the control unit 2a according to the second embodiment of the present invention. **Embodiments for Carrying Out the Invention**

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0015] FIG. 1 is a diagram showing the configuration of a 3D object rendering system 1 according to the first embodiment of the present invention. As shown in the figure, the 3D object rendering system 1 according to the present embodiment includes a computer 2, a virtual reality display 3, a tablet 4, an electronic pen 5, a glove unit 6, lightning houses 7a, 7b, and position sensors 8a to 8d. The position sensors 8a, 8c, 8d are attached to the tablet 4, the electronic pen 5, and the glove unit 6, respectively, and the position sensor 8b is provided on the virtual reality display 3. By attaching the position sensor 8c, the electronic pen 5 functions as a stylus and also as a controller. Note that the position sensor 8c may be incorporated in the electronic pen 5.

[0016] Each device shown in FIG. 1 is generally arranged in a room. In the 3D object rendering system 1, almost the entire room can be used as a virtual reality space.

[0017] The computer 2 includes a control unit 2a (controller) and a memory 2b that cooperates with the control unit 2a. Each process performed by the computer 2 described later can be realized by the control unit 2a cooperating with the memory 2b (more specifically, by reading and executing a program stored in the memory 2b).

[0018] The computer 2 is connected to each of the virtual reality display 3 and the lighting houses 7a, 7b, either wired or wirelessly. In FIG. 1, an example is shown in which the computer 2 is connected to each of the virtual reality display 3 and the lighting houses 7a, 7b by a wired communication standard such as USB. Also, although details will be described later, when the tablet 4 is a device having a communication function, the computer 2 is also connected to the tablet 4, either wired or wirelessly. In FIG. 1, an example is shown in which the computer 2 and the tablet 4 are connected by a short-range wireless communication standard such as Bluetooth (registered trademark). In addition, when the tablet 4 or the virtual reality display 3 incorporates the function of a computer, it may be configured that the computer 2 is constituted by that computer.

[0019] The computer 2 is configured to have a function of displaying a virtual reality space on the virtual reality display 3. This virtual reality space may be a VR (Virtual Reality) space, an AR (Augmented Reality) space, or a MR (Mixed Reality) space. When displaying a VR space, a user wearing the virtual reality display 3 recognizes virtual reality and is separated from the real world. On the other hand, when displaying an AR space or a MR space, a user wearing the virtual reality display 3 recognizes a space in which virtual reality and the real world are mixed.

[0020] The computer 2 is configured to function as a rendering device that sets a virtual reality space based on the positions of the lighting houses 7a, 7b and renders various 3D objects within the set virtual reality space. The computer 2 updates the display of the virtual reality display 3 based on the result of the rendering. As a result, various 3D objects will appear in the virtual reality space displayed on the virtual reality display 3.

[0021] Rendering by computer 2 is performed based on 3D objects stored in memory 2b. A 3D object is information indicating the shape, position, and orientation of a 3D object in a virtual reality coordinate system that represents the virtual reality space set up by computer 2, and is stored in memory 2b for each 3D object to be rendered.

[0022] The 3D objects rendered by computer 2 include 3D objects representing the tablet 4, electronic pen 5, and glove unit 6 shown in Figure 1. In rendering these 3D objects, computer 2 first detects the position and orientation of position sensors 8a to 8d in the virtual reality coordinate system. Then, based on the position and orientation of the detected position sensor 8b, it acquires viewpoint information indicating the user's viewpoint. Based on the acquired viewpoint information, the stored shapes of each 3D object, and the positions and orientations of the detected position sensors 8a, 8c, and 8d, computer 2 is configured to render 3D objects representing the tablet 4, electronic pen 5, and glove unit 6 in the virtual reality space.

[0023] Computer 2 is further configured to detect user actions within the virtual reality space by detecting the positions of position sensors 8c and 8d, and to create new 3D objects or update existing 3D objects based on the results.

[0024] Virtual reality display 3 is a VR display (head-mounted display) that is worn on the human head. While commercially available virtual reality displays come in various types, such as "transparent" or "opaque," and "glasses-type" or "hat-type," any of these can be used for virtual reality display 3.

[0025] The virtual reality display 3 is connected to the position sensor 8a, the electronic pen 5 (including position sensor 8c), and the glove unit 6 (including position sensor 8d) by wire or wireless connection. The position sensors 8a, 8c, and 8d are configured to notify the virtual reality display 3 of the light reception level information, which will be described later, through this connection. The virtual reality display 3 is configured to notify the computer 2 of the light reception level information notified from each of the position sensors 8a, 8c, and 8d, along with the light reception level information from its own built-in position sensor 8b. Based on the light reception level information thus notified, the computer 2 detects the position and orientation of each of the position sensors 8a to 8d in the virtual reality spatial coordinate system. The electronic pen 5 and the glove unit 6 are also configured to notify the virtual reality display 3 of operation information, which will be described later, through the above connection. The virtual reality display 3 is configured to transfer the operation information thus notified to the computer 2.

[0026] The tablet 4 has a tablet surface 4a. The tablet surface 4a is preferably a flat surface and may be made of a material suitable for sliding the tip of the electronic pen 5. In one example, the tablet 4 is a so-called digitizer and is configured to have a touch sensor that detects the indicated position of the electronic pen 5 within the touch surface and a communication function that notifies the computer 2 of the detected indicated position. In this case, the tablet surface 4a is made of the touch surface of the digitizer. In another example, the tablet 4 is a so-called tablet computer and is configured to have a display, a touch sensor that detects the indicated position of the electronic pen 5 within the display surface of the display, and a communication function that notifies the computer 2 of the detected indicated position. In this case, the tablet surface 4a is made of the display surface of the display. In yet another example, the tablet 4 is a physical object that does not have the function of detecting the indicated position of the electronic pen 5 (including a simple board, a table, a display or computer that does not have the function of detecting the indicated position of the electronic pen 5, etc.). In this case, the tablet surface 4a is made of a plane provided on the surface of the tablet 4.

[0027] The position sensor 8a is fixedly mounted on the surface of the tablet 4. Therefore, the position and orientation of the position sensor 8a detected by the computer 2 represent the position and orientation of the tablet surface 4a in the virtual reality coordinate system.

[0028] The electronic pen 5 and glove unit 6 are used by the user to indicate their location within the virtual reality space. The electronic pen 5 is configured in a pen-like shape. The glove unit 6 is shaped like a glove that is worn on the user's hand.

[0029] Various sensing devices, such as switches, are provided on or inside the surface of the electronic pen 5. These sensing devices include not only toggle switches that take on or off states, but also sensors configured to detect any physical quantity. Examples of switches provided on the electronic pen 5 include side switches or tail switches configured to accept on / off operations by the user. Other examples of switches provided on the electronic pen 5 include capacitive sensors that detect pressure (pen pressure) applied to the pen tip. The electronic pen 5 is configured to detect the output of its switches (pressed state or detected physical quantity) and to notify the virtual reality display 3 of all or part of the detection result as its operation information.

[0030] The sensing device provided on the electronic pen 5 may include a force sensor (load sensor) that detects the user's gripping force on the electronic pen 5. In this case, although the output of the force sensor does not inherently represent pen pressure, the computer 2, upon receiving notification of the force sensor output, may treat it as data indicating pen pressure (pen pressure data). This makes it possible to reflect pen pressure in the drawing result even when the user manipulates the electronic pen 5 in the air.

[0031] If the tablet 4 has a touch sensor, the position of the electronic pen 5 is also detected by this touch sensor. The position detected by the touch sensor is not a position in the virtual reality space coordinate system, but a position in the tablet surface coordinate system defined on the tablet surface 4a. The touch sensor is configured to notify the computer 2 of the detected position of the electronic pen 5.

[0032] Generally, the position detected by the touch sensor is more accurate than the position detected using the position sensor 8c. Therefore, when the computer 2 receives notification of the position of the electronic pen 5 from the touch sensor, it is preferable to acquire the position notified by the touch sensor as the position of the electronic pen 5, rather than the position detected through the position sensor 8c. In this case, it is preferable for the computer 2 to convert the position notified by the touch sensor into a position in the virtual reality coordinate system by associating the tablet surface coordinate system with the virtual reality coordinate system based on the position and orientation of the tablet surface 4a detected using the position sensor 8a.

[0033] Figure 2 illustrates the relationship between the tablet surface coordinate system and the virtual reality space coordinate system. The figure shows the state where the tablet surface 4a is located in the virtual reality space 10. The virtual reality space coordinate system is defined by three axes VRX, VRY, and VRZ, and the tablet surface coordinate system is defined by three axes TRX, TRY, and TRZ. However, axis TRZ is the normal direction of the tablet surface 4a. When the illustrated position P is detected by the touch sensor as a position (x,y,z) in the tablet surface coordinate system (z represents, for example, the hover position), the computer 2 converts this (x,y,z) to a position (X,Y,Z) in the virtual reality space coordinate system through a predetermined conversion process. This makes it possible to convert the position notified from the touch sensor to a position in the virtual reality space coordinate system.

[0034] Here, the position detection of the electronic pen 5 by the touch sensor may be performed using either an electromagnetic induction method or an active electrostatic method. When using the active electrostatic method, the touch sensor is configured to send a beacon signal at predetermined time intervals from a sensor electrode (not shown) placed within the touch surface. The beacon signal contains a command for controlling the electronic pen 5 from the touch sensor. The content of the control by the command includes, for example, sending pressure data (detected by a capacitive sensor) indicating the pressure applied to the tip of the electronic pen 5, sending the pressed state of various switches (not shown) provided on the electronic pen 5, and sending a unique ID pre-stored in the electronic pen 5.

[0035] The electronic pen 5, which supports the active electrostatic method, sends a pen signal as a response signal when it detects the beacon signal. The pen signal is a signal that includes a burst signal, which is an unmodulated carrier wave, and a data signal obtained by modulating the carrier wave with data corresponding to the command. The touch sensor attempts to detect the burst signal with the sensor electrodes and detects the position of the electronic pen 5 based on the detection result. It also receives the data transmitted by the electronic pen 5 in response to the command by detecting and demodulating the data signal with the sensor electrodes. The tablet 4 is configured to transmit the position of the electronic pen 5 and the data transmitted by the electronic pen 5, which it has thus acquired, to the computer 2. The computer 2 is configured to convert the notified position into a position in the virtual reality space coordinate system as described above, and to acquire the notified data as part of the operation information described above.

[0036] Lightning houses 7a and 7b are signal transmitters for position detection used in the 3D object rendering system 1. Each is configured to emit a signal, in this example laser light, while changing direction according to control by computer 2. Position sensors 8a to 8d are each composed of multiple light receiving sensors. Each light receiving sensor receives the signal (laser light) emitted by each of the lightning houses 7a and 7b, and is configured to acquire light receiving level information, including the respective light receiving levels. As described above, the acquired light receiving level information is notified to computer 2 from each position sensor 8a to 8d and used to detect their positions and orientations.

[0037] The above describes the overall overview of the 3D object rendering system 1. In such a 3D object rendering system 1, when a user inputs a new 3D object, they have previously either used the electronic pen 5 and glove unit 6 for 3D input, or used the tablet 4 as a tablet computer and inputted 2D objects to this tablet computer using the electronic pen 5. However, as mentioned above, 3D input has the advantage of allowing intuitive operation but the disadvantage of insufficient accuracy, while 2D input can achieve high accuracy but the disadvantage of being difficult to operate intuitively.

[0038] In view of the problems with conventional input methods, the 3D object rendering system 1 according to this embodiment enables intuitive and highly accurate rendering in the virtual reality space by allowing the user to switch between displaying 3D objects in the virtual reality space (3D display or 2D display). This point will be explained in detail below with reference to the flowchart of the processing performed by the control unit 2a of the computer 2.

[0039] Figure 3 is a flowchart showing the processing performed by the control unit 2a of computer 2. This processing is performed when the user inputs a 3D object using at least one of the electronic pen 5 and the glove unit 6, and is initiated when the user performs a predetermined operation using the electronic pen 5 and the glove unit 6, which is detected by the control unit 2a.

[0040] As shown in Figure 3, the control unit 2a first allocates a memory area in memory 2b for a 3D object that indicates the shape, position, and orientation of the input object in the virtual reality coordinate system (step S1). The specific format of the 3D object is not particularly limited, but it is preferable to use data in VRML format or X3D format, for example.

[0041] Next, the control unit 2a executes the process of acquiring location information, etc. (step S2).

[0042] Figure 4 is a flowchart showing the details of the location information acquisition process performed in step S2. As shown in the figure, the control unit 2a performs the location information acquisition process by executing the following steps: acquiring viewpoint information (step S20), acquiring tablet surface information (step S21), acquiring controller information (first information) (step S22; controller information acquisition step; first information acquisition process), and acquiring electronic pen information (second information) (step S23; electronic pen information acquisition step; second information acquisition process). Note that the execution order of steps S20 to S23 is not particularly limited.

[0043] Viewpoint information represents the user's viewpoint in the virtual reality spatial coordinate system, and is specifically indicated by the position and orientation of the virtual reality display 3. The control unit 2a is configured to acquire viewpoint information based on the position and orientation detected by the position sensor 8b. The specific viewpoint information is composed of, for example, vector information starting from a single 3D coordinate.

[0044] The tablet surface information is information indicating the shape, position, and orientation of the tablet surface 4a in the virtual reality coordinate system, and is stored in memory 2b as one of the 3D objects. The control unit 2a acquires the tablet surface information based on the position and orientation detected by the position sensor 8a and the shape of the tablet 4 that is stored in advance.

[0045] Controller information refers to the position and orientation of the 3D controller (including the electronic pen 5 and the glove unit 6) in the virtual reality coordinate system, as well as information on the operation of the 3D controller. The control unit 2a acquires controller information based on the position and orientation detected by the position sensors 8c and 8d, and the operation information of the electronic pen 5 and the glove unit 6, respectively, received via the virtual reality display 3.

[0046] The electronic pen information is information indicating the indicated position of the electronic pen 5 in the tablet surface coordinate system and the operation information of the electronic pen 5. If the tablet 4 has a touch sensor, the control unit 2a obtains the indicated position of the electronic pen 5 in the tablet surface coordinate system from the touch sensor. On the other hand, if the tablet 4 does not have a touch sensor, the control unit 2a obtains the indicated position of the electronic pen 5 in the tablet surface coordinate system by performing a conversion process (the reverse of the conversion process explained with reference to Figure 2) on the position obtained from the position sensor 8c (position in the virtual reality space coordinate system). Furthermore, if the control unit 2a can obtain operation information of the electronic pen 5 from the touch sensor (for example, if the electronic pen 5 supports an active electrostatic method), it obtains operation information of the electronic pen 5 (including the output of the capacitive sensor) from the touch sensor. On the other hand, if operation information of the electronic pen 5 cannot be obtained from the touch sensor, the control unit 2a obtains operation information of the electronic pen 5 (including the output of the force sensor) via the virtual reality display 3.

[0047] Returning to Figure 3, the control unit 2a, which has performed the position information acquisition process, then performs the tablet surface display process (step S3). This process is for displaying a tablet surface image, which shows the tablet surface, and a display surface image, which shows the display surface for displaying the 3D object being input in 2D, within the virtual reality space.

[0048] Figure 5 is a flowchart showing the details of the tablet surface display processing performed in step S3. As shown in the figure, the control unit 2a first renders the display surface image based on the position selected by the user in the virtual reality space or the position in contact with the 3D object being input (step S30. Display surface image rendering step (processing)). The user's selection of position is performed, for example, by the user pressing a switch provided on the electronic pen 5, and operation information indicating this is notified to the control unit 2a. The display surface image may be a representation of a display or a simple rectangular frame. Preferably, the control unit 2a renders the display surface image based on the viewpoint information acquired in step S20 of Figure 4, such that the normal direction of the display surface image matches the user's line of sight.

[0049] Next, the control unit 2a acquires first correspondence information indicating the correspondence between the virtual reality space coordinate system and the display surface coordinate system (first planar coordinate system) defined on the display surface (step S31). Specifically, the first correspondence information is a transformation rule for converting between the virtual reality space coordinate system and the display surface coordinate system.

[0050] Next, the control unit 2a acquires second correspondence information that shows the correspondence between the virtual reality space coordinate system and the tablet plane coordinate system (second planar coordinate system) (step S32). Specifically, the second correspondence information is a transformation rule for converting between the virtual reality space coordinate system and the tablet plane coordinate system.

[0051] Finally, the control unit 2a renders a tablet surface image representing the tablet surface 4a based on the tablet surface information and viewpoint information (step S33. Tablet surface image rendering step (processing)). The tablet surface image may be a representation of the touch surface of the tablet device, or it may be a simple rectangular frame. Preferably, the control unit 2a renders the tablet surface image based on the viewpoint information acquired in step S20 of Figure 4, such that the angle (downward angle) between the user's line of sight and the tablet surface 4a is a predetermined value.

[0052] Returning to Figure 3, the control unit 2a, which has performed the tablet surface display processing, then accepts a user's operation to select an operating mode (step S4). This selection may be performed, for example, by the user pressing a switch provided on the electronic pen 5, or by the user changing the distance between the tablet surface 4a and the electronic pen 5. For example, the display may switch to 2D when the distance between the tablet surface 4a and the electronic pen 5 becomes closer than a predetermined distance, and automatically switch to 3D display when the distance between the tablet surface 4a and the electronic pen 5 becomes further than a predetermined distance. In the latter case, the distance between the tablet surface 4a and the electronic pen 5 may be detected by a touch sensor, or by the control unit 2a based on the display position of the tablet surface image and the position of the electronic pen 5 detected using the position sensor 8c. The control unit 2a, having received the selection operation in step S4, performs a process to enter either the 3D display mode or the 2D display mode (i.e., a display selection process to select either 3D display or 2D display) according to the selection (step S5, mode selection step).

[0053] When the user enters 3D display mode in step S5, the control unit 2a performs a 3D rendering step (process) to render the 3D object in the virtual reality space coordinate system onto the virtual reality display 3. Specifically, the control unit 2a first renders the input object and other 3D objects based on the 3D objects stored in memory 2b and the viewpoint information acquired in step S20 of Figure 4 (step S6). Then, it updates the output (display) to the virtual reality display 3 based on the result (step S7, display update step (process)). At this time, other displays in the virtual reality space, such as the display surface image rendered in step S30 of Figure 5 and the tablet surface image rendered in step S33 of Figure 5, are also updated simultaneously. This makes it possible for the user to edit the input object using 3D input.

[0054] The control unit 2a further updates the 3D object of the input object stored in memory 2b based on the controller information acquired in step S22 of Figure 4 (step S8. 3D object update step (processing)). After that, it returns to step S2 and continues processing.

[0055] Here, the updating of the 3D object in step S8 and step S14 (described later) is also performed based on the operation information notified from the electronic pen 5. For example, if data indicating the output of the capacitive sensor or the force sensor is notified from the electronic pen 5, the computer 2 acquires this data as pressure data and determines the line width and transparency of the object being input based on the acquired pressure data. Then, it reflects the result of that determination in the 3D object.

[0056] When the user enters 2D display mode in step S5, the control unit 2a performs a 2D rendering step (process) to render the 3D object as a 3D object for 2D display in the display plane coordinate system on the virtual reality display 3. Specifically, first, based on the first correspondence information acquired in step S31 in Figure 5, the 3D object of the input object stored in memory 2b is converted into a 2D object that shows the shape, position, and orientation of the input object in the display plane coordinate system (step S10, first conversion step). Then, based on the obtained 2D object and the viewpoint information acquired in step S20 in Figure 4, the input object is rendered (step S11), and the output (display) to the virtual reality display 3 is updated based on the result (step S12, display update step (process)). At this time, as with step S7, other displays in the virtual reality space, such as the display plane image rendered in step S30 in Figure 5 and the tablet plane image rendered in step S33 in Figure 5, are simultaneously updated. This makes it possible for the user to edit the input object using 2D input.

[0057] Next, the control unit 2a converts the indicated position shown by the electronic pen information acquired in step S23 of Figure 4 into a position in the virtual reality space coordinate system based on the second correspondence information acquired in step S32 of Figure 5 (step S13, second conversion step). Then, based on the electronic pen information including the obtained indicated position, it updates the 3D object of the input object stored in memory 2b (step S14, 3D object update step). As described above, this update is also performed based on the operation information notified from the electronic pen 5. The control unit 2a then returns to step S2 and continues processing.

[0058] Figure 6 shows the state in which a 3D object is being input in 2D in the virtual reality space (the state in step S4 of Figure 3 where the user has selected the 2D display mode), and Figure 7 shows the state in which a 3D object is being input in 3D in the virtual reality space (the state in step S4 of Figure 3 where the user has selected the 3D display mode).

[0059] As shown in Figure 6, in 2D display mode, the input object 13 is displayed in 2D within a rectangular display surface image 11 shown in the virtual reality space 10. The three axes DRX, DRY, and DRZ shown in the display surface image 11 represent the display surface coordinate system. Of the three axes, axis DRZ is the normal direction of the display surface image 11. A rectangular tablet surface image 12 is also displayed, and the user edits the input object 13 by moving the electronic pen 5 within this tablet surface image 12. This editing is performed by moving the electronic pen 5 on a plane, so it is 2D input. Although invisible to the user wearing the virtual reality display 3, a tablet surface 4a actually exists at the location where the tablet surface image 12 is displayed, so the user can move the electronic pen 5 within the tablet surface image 12 while feeling the tactile feedback of the tablet surface 4a.

[0060] On the other hand, as shown in Figure 7, in 3D display mode, the input object 13 is displayed in 3D within the virtual reality space 10. The user edits the input object 13 by moving the electronic pen 5 and the glove unit 6 within this virtual reality space 10. This editing is performed using a 3D input controller, and is therefore 3D input.

[0061] As described above, the 3D object rendering system 1 according to this embodiment allows the user to switch between displaying 3D objects in virtual reality space (3D display or 2D display), thereby enabling intuitive and highly accurate rendering in virtual reality space. Furthermore, editing of 3D objects using 2D input is possible while 2D display is enabled, and editing of 3D objects using 3D input is possible while 3D display is enabled, making it possible to edit 3D objects using an input method appropriate to the display method.

[0062] Furthermore, according to the 3D object rendering system 1 of this embodiment, it becomes possible to input 3D objects using an electronic pen 5 in a virtual reality space.

[0063] Next, a 3D object rendering system 1 according to a second embodiment of the present invention will be described. The 3D object rendering system 1 according to this embodiment differs from the 3D object rendering system 1 according to the first embodiment, in that the display plane coordinate system and the tablet plane coordinate system are the same coordinate system. In other respects, it is the same as the 3D object rendering system 1 according to the first embodiment, so the same reference numerals are used for the same components as in the first embodiment, and the following description will focus on the differences from the first embodiment.

[0064] Figure 8 shows the state in which a 3D object is input in 2D in the virtual reality space according to this embodiment. First, referring to Figure 8, the outline of this embodiment will be explained. The control unit 2a in this embodiment is configured to render a tablet terminal image 14, which represents the tablet 4 (tablet terminal) shown in Figure 1, in the virtual reality space. The tablet surface image 14a included in the tablet terminal image 14 corresponds to both the tablet surface image and the display surface image described in the first embodiment. Therefore, in this embodiment, the tablet surface coordinate system (axis TRX, TRY, TRZ) also serves as the display surface coordinate system. As a result, in 2D display mode, as shown in Figure 8, the input object 13 is displayed in 2D within the tablet surface image 14a.

[0065] Figure 9 is a flowchart showing part of the processing performed by the control unit 2a according to this embodiment. Figure 9(a) replaces the flowchart shown in Figure 5. On the other hand, Figure 9(b) replaces step S10 shown in Figure 3.

[0066] As shown in Figure 9(a), the control unit 2a in this embodiment is configured to execute step S33a instead of step S33, instead of steps S30 and S31 shown in Figure 5. In step S33a, based on the tablet surface information and viewpoint information, a tablet terminal image representing the tablet 4 (tablet terminal) shown in Figure 1 is rendered (tablet terminal image rendering step (processing)). As a result, as shown in Figure 8, a tablet terminal (tablet terminal image 14) similar to the real tablet 4 (tablet terminal) appears in the virtual reality space 10.

[0067] Furthermore, as shown in Figure 9(b), the control unit 2a in this embodiment performs a process to convert the 3D object of the input object stored in memory 2b into a 2D object that indicates the shape, position, and orientation of the input object in the display plane coordinate system, based on the second correspondence information obtained in step S32 rather than the first correspondence information, in order to obtain 2D object information to be used when rendering the input object in 2D display mode (step S10a). As a result, as shown in Figure 8, the input object 13 (i.e., the position of the electronic pen 5 in the virtual reality space coordinate system obtained by the controller information acquisition step (step S22)) is displayed in the tablet surface image 14a.

[0068] The 3D object rendering system 1 according to this embodiment also allows the user to switch the display method of 3D objects in the virtual reality space (3D display or 2D display) according to their selection, thereby enabling intuitive and highly accurate rendering in the virtual reality space. Furthermore, editing of 3D objects is possible using 2D input while in 2D display mode, and editing of 3D objects is possible using 3D input while in 3D display mode, making it possible to edit 3D objects using an input method appropriate to the display mode.

[0069] Furthermore, according to this embodiment, users can experience inputting data into a tablet device within a virtual reality space. Therefore, it becomes possible to perform input operations on 3D objects as if using a regular tablet device.

[0070] Although preferred embodiments of the present invention have been described above, the present invention is not limited in any way to these embodiments, and it goes without saying that the present invention can be implemented in various forms without departing from its essence. [Explanation of symbols]

[0071] 1. 3D Object Rendering System 2 Computers 2a Control section 2b memory 3. Virtual reality display 4 tablets 4a Tablet surface 5 Electronic pen 6 Glove Units 7a,7b Lightning House 8a~8d Position Sensor 10 Virtual reality space 11 Display surface 12 Tablet screen images 13. Object being entered 14. Tablet device image 14a Tablet screen image

Claims

1. A computer having a control unit that renders objects in a virtual space, The control unit, A rendering process that renders a planar image representing a plane in real space used when inputting a position indicated by a stylus into the virtual space based on the position and orientation of the plane in the real space, A rendering process that renders a display image, which is a display image that displays the aforementioned object and is different from the planar image, into the virtual space, The system performs a display process that displays the object, which is generated based on the position indicated by the stylus on the plane, on the display image. computer.

2. A computer having a control unit for rendering objects in a virtual space, The control unit, A rendering process that renders a planar image in the virtual space representing a plane in real space used when inputting a position indicated by a stylus, which is configured not to have the function of detecting the position indicated by the stylus, A rendering process that renders a display image, which is a display image that displays the aforementioned object and is different from the planar image, into the virtual space, The system performs a display process that displays the object, which is generated based on the position indicated by the stylus on the plane, on the display image. computer.

3. The control unit, Further rendering is performed to render the stylus image, which moves in conjunction with the movement of the stylus on the plane, into the virtual space. The computer according to claim 1 or 2.

4. The position indicated by the stylus is detected by a position detection sensor located below the plane. The computer according to claim 1 or 2.

5. The control unit, Further, a transformation process is performed to convert the position detected by the position detection sensor into a position in the coordinate system of the virtual space. The computer according to claim 4.

6. The control unit, The system performs a display process that displays the object, which is generated based on the pressure detected by the sensing device of the stylus, on the display image. The computer according to claim 1 or 2.

7. The aforementioned pressure is the pressure applied to the stylus by the plane when the stylus is in contact with the plane. The computer according to claim 6.

8. The aforementioned planar image is a tablet device image showing a tablet device, The plane in the real space is the tablet surface of the tablet device. The computer according to claim 1 or 2.

Images