Computer

Abstract

To allow 2D input of a 3D object in a virtual reality space with the same use feeling as that when a drawing operation is carried out using a so-called plate tablet.SOLUTION: A computer according to the present invention is a computer having a control unit for rendering an object in a virtual space. The control unit executes a rendering process for rendering, in a virtual space, a plane image indicating a plane in a real space used when a position instructed by a stylus is input, a rendering process for rendering, in the virtual space, a display image as a display image for displaying the object and different from the plane image, and a display process for displaying, in the display image, an object generated based on the position instructed by the stylus on the plane.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a rendering device and a rendering method for rendering a 3D object in a virtual reality (including VR: Virtual Reality, AR: Augmented Reality, and MR: Mixed Reality) space. [Background technology]

[0002] In recent years, there has been an increasing need to design various products while viewing them in a virtual reality space.

[0003] Patent Document 1 discloses a technology for generating a 3D object based on a 2D object input to a tablet terminal using an electronic pen in an AR space. Hereinafter, the input method performed by moving the electronic pen on a flat surface will be referred to as "2D input."

[0004] Non-Patent Document 1 discloses a technique for inputting a 3D object by moving a 3D input controller in a VR space. Hereinafter, such an input method using a 3D input controller will be referred to as "3D input". [Prior art documents] [Patent documents]

[0005] [Patent Document 1] US Patent Application Publication No. 2016 / 0343174 [Non-patent literature]

[0006] [Non-Patent Document 1] Google, "Tilt Brush: Painting from a new perspective", [online], May 3, 2016, [Retrieved October 5, 2017], Internet〈URL:https: / / www.youtube.com / watch?v=TckqNdrdbgk〉 Summary of the Invention [Problem to be solved by the invention]

[0007] The 2D input described above can achieve higher accuracy than 3D input because the position of the electronic pen is fixed within a known plane. However, since 2D input is limited to drawing within a plane, it has a problem that intuitive operation is more difficult than 3D input.

[0008] On the other hand, 3D input allows intuitive operation. However, because the controller position has a high degree of freedom, the technology is not precise enough for design purposes.

[0009] Therefore, one object of the present invention is to provide a rendering device and a rendering method that can realize intuitive and highly accurate drawing in a virtual reality space. [Means for solving the problem]

[0010] A first aspect of the present invention relates to a rendering device that renders a 3D object in a virtual reality space displayed on a virtual reality display, and executes a 3D rendering step of rendering the 3D object as a 3D object of 3D display in a virtual reality space coordinate system, a 2D rendering step of rendering the 3D object as a 3D object of 2D display in a planar coordinate system, and a display update step having a 3D display mode of updating a display of the virtual reality display with a rendering result of the 3D rendering step, and a 2D display mode of updating a display of the virtual reality display with a rendering result of the 2D rendering step.

[0011] A second aspect of the present invention relates to a rendering method for causing a computer to function as a rendering device that renders a 3D object in a virtual reality space displayed on a virtual reality display, the rendering method causing the computer to execute a 3D rendering step of rendering the 3D object as a 3D object of 3D display in a virtual reality space coordinate system, a 2D rendering step of rendering the 3D object as a 3D object of 2D display in a planar coordinate system, and a display update step having a 3D display mode of updating the display of the virtual reality display with the rendering result of the 3D rendering step, and a 2D display mode of updating the display of the virtual reality display with the rendering result of the 2D rendering step. Effect of the Invention

[0012] According to the present invention, the display method (3D display or 2D display) of a 3D object in a virtual reality space can be switched according to user selection, making it possible to realize intuitive and highly accurate drawing in the virtual reality space. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram showing a configuration of a 3D object rendering system 1 according to a first embodiment of the present invention. [Diagram 2] 3 is a diagram illustrating the relationship between a tablet surface coordinate system and a virtual reality space coordinate system. FIG. [Diagram 3] 2 is a flow chart showing a process performed by a control unit 2a shown in FIG. [Figure 4] 3 is a flow diagram showing details of the location information etc. acquisition process shown in FIG. 2. [Diagram 5] 3 is a flow diagram showing details of the tablet terminal display process shown in FIG. 2. [Figure 6] FIG. 2 is a diagram showing a state in which a 3D object is being input in 2D in a virtual reality space according to the first embodiment of the present invention. [Figure 7]FIG. 2 is a diagram showing a state in which a 3D object is being 3D input in a virtual reality space according to the first embodiment of the present invention. [Figure 8] FIG. 11 is a diagram showing a state in which a 3D object is being input in 2D in a virtual reality space according to a second embodiment of the present invention. [Figure 9] FIG. 11 is a flow chart showing a part of the processing performed by a control unit 2a according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

[0015] FIG. 1 is a diagram showing a configuration of a 3D object rendering system 1 according to a first embodiment of the present invention. As shown in the figure, the 3D object rendering system 1 according to the present embodiment includes a computer 2, a virtual reality display 3, a tablet 4, an electronic pen 5, a glove unit 6, lightning houses 7a, 7b, and position sensors 8a to 8d. The position sensors 8a, 8c, and 8d are attached to the tablet 4, the electronic pen 5, and the glove unit 6, respectively, and the position sensor 8b is provided on the virtual reality display 3. By attaching the position sensor 8c, the electronic pen 5 functions as a stylus and also as a controller. The position sensor 8c may be built into the electronic pen 5.

[0016] 1 are arranged in a room in principle. In the 3D object rendering system 1, almost the entire room can be used as a virtual reality space.

[0017] The computer 2 includes a control unit 2a (controller) and a memory 2b that cooperates with the control unit 2a. Each process performed by the computer 2, which will be described later, can be realized by the control unit 2a cooperating with the memory 2b (more specifically, by reading and executing a program stored in the memory 2b).

[0018] The computer 2 is connected to the virtual reality display 3 and the lightning houses 7a and 7b by wire or wirelessly. FIG. 1 shows an example in which the computer 2 is connected to the virtual reality display 3 and the lightning houses 7a and 7b by a wired communication standard such as USB. In addition, as will be described in detail later, when the tablet 4 is a device having a communication function, the computer 2 is also connected to the tablet 4 by wire or wirelessly. FIG. 1 shows an example in which the computer 2 and the tablet 4 are connected by a short-range wireless communication standard such as Bluetooth (registered trademark). In addition, when the tablet 4 or the virtual reality display 3 has a built-in function as a computer, the computer 2 may be configured by that computer.

[0019] The computer 2 is configured with a function of displaying a virtual reality space on the virtual reality display 3. This virtual reality space may be a VR (Virtual Reality) space, an AR (Augmented Reality) space, or an MR (Mixed Reality) space. When a VR space is displayed, a user wearing the virtual reality display 3 recognizes virtual reality, which is separated from the real world. On the other hand, when an AR space or an MR space is displayed, a user wearing the virtual reality display 3 recognizes a space in which virtual reality and the real world are mixed.

[0020] The computer 2 is configured to function as a rendering device that sets a virtual reality space based on the positions of the lightning houses 7a and 7b and renders various 3D objects in the set virtual reality space. The computer 2 updates the display of the virtual reality display 3 based on the rendering results. As a result, various 3D objects appear in the virtual reality space displayed on the virtual reality display 3.

[0021] Rendering by the computer 2 is performed based on 3D objects stored in the memory 2b. The 3D object is information indicating the shape, position, and orientation of a 3D object in a virtual reality space coordinate system indicating a virtual reality space set by the computer 2, and is stored in the memory 2b for each 3D object to be rendered.

[0022] The 3D objects rendered by the computer 2 include 3D objects representing the tablet 4, electronic pen 5, and glove unit 6 shown in Fig. 1. In rendering these 3D objects, the computer 2 first detects the position and orientation of each of the position sensors 8a to 8d in the virtual reality space coordinate system. Then, the computer 2 acquires viewpoint information indicating the user's viewpoint based on the detected position and orientation of the position sensor 8b, and is configured to render 3D objects representing each of the tablet 4, electronic pen 5, and glove unit 6 in the virtual reality space based on the acquired viewpoint information, the shape of each stored 3D object, and the detected positions and orientations of the position sensors 8a, 8c, and 8d.

[0023] The computer 2 is further configured to detect operations performed by the user in the virtual reality space by detecting the positions of the position sensors 8c and 8d, and to create a new 3D object or update a 3D object that is already held based on the results.

[0024] The virtual reality display 3 is a VR display (head-mounted display) that is worn on a human head. There are various types of virtual reality displays generally available on the market, such as "transparent" or "non-transparent," "glasses-type" or "hat-type," and any of these can be used as the virtual reality display 3.

[0025] The virtual reality display 3 is connected to each of the position sensor 8a, the electronic pen 5 (including the position sensor 8c), and the glove unit 6 (including the position sensor 8d) by wire or wirelessly. The position sensors 8a, 8c, and 8d are configured to notify the virtual reality display 3 of light reception level information described later through this connection. The virtual reality display 3 is configured to notify the computer 2 of the light reception level information notified from each of the position sensors 8a, 8c, and 8d together with the light reception level information of the position sensor 8b built in the virtual reality display 3. The computer 2 detects the position and orientation of each of the position sensors 8a to 8d in the virtual reality space coordinate system based on the light reception level information thus notified. In addition, the electronic pen 5 and the glove unit 6 are configured to notify the virtual reality display 3 of operation information described later through the above connection. The virtual reality display 3 is configured to transfer the operation information thus notified to the computer 2.

[0026] The tablet 4 has a tablet surface 4a. The tablet surface 4a is preferably a flat surface and may be made of a material suitable for sliding the pen tip of the electronic pen 5. In one example, the tablet 4 is a so-called digitizer, and is configured with a touch sensor that detects the pointing position of the electronic pen 5 in the touch surface and a communication function that notifies the computer 2 of the detected pointing position. In this case, the tablet surface 4a is configured by the touch surface of the digitizer. In another example, the tablet 4 is a so-called tablet computer, and is configured with a display, a touch sensor that detects the pointing position of the electronic pen 5 in the display surface of the display, and a communication function that notifies the computer 2 of the detected pointing position. In this case, the tablet surface 4a is configured by the display surface of the display. In yet another example, the tablet 4 is a physical object that does not have the function of detecting the pointing position of the electronic pen 5 (including a simple board, a table, a display or a computer that does not have the function of detecting the pointing position of the electronic pen 5, etc.). In this case, the tablet surface 4a is configured by a plane provided on the surface of the tablet 4.

[0027] The position sensor 8a is fixedly installed on the surface of the tablet 4. Therefore, the position and orientation of the position sensor 8a detected by the computer 2 indicates the position and orientation of the tablet surface 4a in the virtual reality space coordinate system.

[0028] The electronic pen 5 and the glove unit 6 are used by the user to indicate a position in the virtual reality space. The electronic pen 5 is configured to have a pen-like shape. The glove unit 6 is shaped like a glove to be worn on the user's hand.

[0029] Various sensing devices such as switches are provided on the surface or inside of the electronic pen 5. The sensing devices referred to here include a changeover switch that can be in either an on or off state, as well as a sensor configured to be capable of detecting any physical quantity. Examples of the switches provided on the electronic pen 5 include a side switch or a tail switch configured to be capable of receiving an on / off operation by a user. Other examples of the switches provided on the electronic pen 5 include a capacitive sensor that detects the pressure (writing pressure) applied to the pen tip of the electronic pen 5. The electronic pen 5 is configured to detect the output (pressed state or detected physical quantity) of the switch provided on the electronic pen 5 and notify the virtual reality display 3 of all or a part of the detection result as its own operation information.

[0030] The sensing device provided in the electronic pen 5 may include a force sensor (load sensor) that detects the gripping force of the electronic pen 5 by the user. In this case, the output of the force sensor does not originally indicate the writing pressure, but the computer 2 that has received a notification of the output of the force sensor may handle it as data indicating the writing pressure (writing pressure data). In this way, even when the user operates the electronic pen 5 in the air, the writing pressure can be reflected in the drawing result.

[0031] If the tablet 4 has a touch sensor, the position of the electronic pen 5 is also detected by the touch sensor. The position detected by the touch sensor is not a position in the virtual reality space coordinate system, but a position in the tablet surface coordinate system defined on the tablet surface 4a. The touch sensor is configured to notify the computer 2 of the detected position of the electronic pen 5.

[0032] In general, the position detected by the touch sensor is more accurate than the position detected by the position sensor 8c. Therefore, when the position of the electronic pen 5 is notified from the touch sensor, it is preferable for the computer 2 to acquire the position notified from the touch sensor as the position of the electronic pen 5, instead of the position detected through the position sensor 8c. In this case, it is preferable for the computer 2 to convert the position notified from the touch sensor into a position in the virtual reality space coordinate system by associating the tablet surface coordinate system with the virtual reality space coordinate system based on the position and orientation of the tablet surface 4a detected by the position sensor 8a.

[0033] FIG. 2 is a diagram for explaining the relationship between the tablet surface coordinate system and the virtual reality space coordinate system. The diagram shows a state in which the tablet surface 4a is located in the virtual reality space 10. The virtual reality space coordinate system is defined by three axes VRX, VRY, and VRZ, and the tablet surface coordinate system is defined by three axes TRX, TRY, and TRZ. However, the axis TRZ is the normal direction of the tablet surface 4a. When the illustrated position P is detected by the touch sensor as a position (x, y, z) in the tablet surface coordinate system (z indicates, for example, a hover position), the computer 2 converts this (x, y, z) into a position (X, Y, Z) in the virtual reality space coordinate system by a predetermined conversion process. This makes it possible to convert the position notified by the touch sensor into a position in the virtual reality space coordinate system.

[0034] Here, the position detection of the electronic pen 5 by the touch sensor may be performed by an electromagnetic induction method or an active electrostatic method. When the active electrostatic method is used, the touch sensor is configured to transmit a beacon signal at a predetermined time interval from a sensor electrode (not shown) arranged in the touch surface. The beacon signal includes a command for controlling the electronic pen 5 from the touch sensor. The contents of the control by the command include, for example, transmitting writing pressure data (detected by a capacitance sensor) indicating the pressure applied to the pen tip of the electronic pen 5, transmitting the pressed state of various switches (not shown) provided on the electronic pen 5, transmitting a unique ID previously stored in the electronic pen 5, and the like.

[0035] When the electronic pen 5 corresponding to the active electrostatic method detects the beacon signal, it transmits a pen signal as a response signal. The pen signal is a signal including a burst signal, which is an unmodulated carrier wave, and a data signal obtained by modulating the carrier wave with data corresponding to the command. The touch sensor attempts to detect the burst signal by the sensor electrode, and detects the position of the electronic pen 5 based on the detection result. Also, the touch sensor receives data transmitted by the electronic pen 5 in response to the command by detecting and demodulating the data signal by the sensor electrode. The tablet 4 is configured to transmit the position of the electronic pen 5 thus obtained and the data transmitted by the electronic pen 5 to the computer 2. The computer 2 is configured to convert the position thus notified into a position in the virtual reality space coordinate system as described above, and to obtain the notified data as part of the above-mentioned operation information.

[0036] The lightning houses 7a and 7b are signal transmitting devices for position detection used in the 3D object rendering system 1, and are each configured to emit a signal, in this example, a laser beam, while changing the direction according to the control of the computer 2. The position sensors 8a to 8d are each configured to receive the signal (laser beam) emitted by each of the lightning houses 7a and 7b with each light receiving sensor, and to obtain light receiving level information including each light receiving level. As described above, the obtained light receiving level information is notified to the computer 2 from each of the position sensors 8a to 8d, and is used to detect their positions and orientations.

[0037] The above describes the overall overview of the 3D object rendering system 1. When a user inputs a new 3D object in such a 3D object rendering system 1, up until now, the user has either performed 3D input using the electronic pen 5 and glove unit 6, or used the tablet 4 as a tablet computer and performed 2D input using the electronic pen 5 for this tablet computer. However, as described above, 3D input has the advantage of enabling intuitive operation but the disadvantage of insufficient accuracy, and 2D input has the disadvantage of being difficult to intuitively operate while being highly accurate.

[0038] In consideration of the problems of the conventional input method, the 3D object rendering system 1 according to the present embodiment allows the user to select the display method (3D display or 2D display) of the 3D object in the virtual reality space, thereby enabling intuitive and highly accurate drawing in the virtual reality space. This will be described in detail below with reference to a flow chart of the process performed by the control unit 2a of the computer 2.

[0039] 3 is a flow diagram showing a process performed by the control unit 2a of the computer 2. This process is executed when the user inputs a 3D object using at least one of the electronic pen 5 and the glove unit 6, and is started when the user performs a predetermined operation using the electronic pen 5 and the glove unit 6 and the control unit 2a detects the operation.

[0040] 3, the control unit 2a first secures in the memory 2b a storage area for a 3D object indicating the shape, position, and orientation of an object being input in the virtual reality space coordinate system (step S1). The specific format of the 3D object is not particularly limited, but it is preferable to use data in, for example, VRML format or X3D format.

[0041] Next, the control unit 2a executes a process of acquiring position information and the like (step S2).

[0042] 4 is a flow diagram showing details of the position information, etc., acquisition process executed in step S2. As shown in the figure, control unit 2a executes the position information, etc., acquisition process by executing each of the steps of acquiring viewpoint information (step S20), acquiring tablet surface information (step S21), acquiring controller information (first information) (step S22: controller information acquisition step; first information acquisition process), and acquiring electronic pen information (second information) (step S23: electronic pen information acquisition step; second information acquisition process). Note that the order of execution of steps S20 to S23 is not particularly limited.

[0043] The viewpoint information is information that represents the user's viewpoint in the virtual reality space coordinate system, and is specifically indicated by the position and orientation of the virtual reality display 3. The control unit 2a is configured to acquire the viewpoint information based on the position and orientation detected by the position sensor 8b. Specific viewpoint information is configured by, for example, vector information starting from one three-dimensional coordinate.

[0044] The tablet surface information is information indicating the shape, position, and orientation of the tablet surface 4a in the virtual reality space coordinate system, and is stored in the memory 2b as one of the 3D objects. The control unit 2a acquires the tablet surface information based on the position and orientation detected by the position sensor 8a and the shape of the tablet 4 stored in advance.

[0045] The controller information is information indicating the position and orientation of the 3D controller (including the electronic pen 5 and the glove unit 6) in the virtual reality space coordinate system, and operation information of the 3D controller. The control unit 2a acquires the controller information based on the positions and orientations detected by the position sensors 8c and 8d, and the operation information of the electronic pen 5 and the glove unit 6 received via the virtual reality display 3.

[0046] The electronic pen information is information indicating the indicated position of the electronic pen 5 in the tablet surface coordinate system and the operation information of the electronic pen 5. When the tablet 4 has a touch sensor, the control unit 2a acquires the indicated position of the electronic pen 5 in the tablet surface coordinate system from the touch sensor. On the other hand, when the tablet 4 does not have a touch sensor, the control unit 2a acquires the indicated position of the electronic pen 5 in the tablet surface coordinate system by performing a conversion process (the reverse process of the conversion process described with reference to FIG. 2) on the position acquired from the position sensor 8c (the position in the virtual reality space coordinate system). Furthermore, when the control unit 2a can acquire the operation information of the electronic pen 5 from the touch sensor (for example, when the electronic pen 5 supports the active electrostatic method), the control unit 2a acquires the operation information of the electronic pen 5 (including the output of the capacitive sensor) from the touch sensor. On the other hand, when the control unit 2a cannot acquire the operation information of the electronic pen 5 from the touch sensor, the control unit 2a acquires the operation information of the electronic pen 5 (including the output of the force sensor) via the virtual reality display 3.

[0047] 3, the control unit 2a, having executed the position information acquisition process, then executes a tablet surface display process (step S3). This process is for displaying, in the virtual reality space, a tablet surface image showing the tablet surface and a display surface image showing the display surface for 2D display of the 3D object being input.

[0048] FIG. 5 is a flow diagram showing the details of the tablet surface display process executed in step S3. As shown in the figure, the control unit 2a first performs rendering of a display surface image based on a position selected by the user in the virtual reality space or a position in contact with a 3D object being input (step S30. Display surface image rendering step (processing)). The user selects a position, for example, by pressing a switch provided on the electronic pen 5, and operation information indicating this is notified to the control unit 2a. The display surface image may be an image imitating a display, or may simply be a rectangular frame. It is preferable that the control unit 2a renders the display surface image so that the normal direction of the display surface image coincides with the line of sight of the user, based on the viewpoint information acquired in step S20 of FIG. 4.

[0049] Next, the control unit 2a acquires first correspondence information indicating a correspondence relationship between the virtual reality space coordinate system and a display surface coordinate system (first planar coordinate system) defined on the display surface (step S31). The first correspondence information is specifically a conversion rule for converting between the virtual reality space coordinate system and the display surface coordinate system.

[0050] Next, control unit 2a acquires second correspondence information indicating the correspondence relationship between the virtual reality space coordinate system and the tablet surface coordinate system (second planar coordinate system) (step S32). The second correspondence information is specifically a conversion rule for converting between the virtual reality space coordinate system and the tablet surface coordinate system.

[0051] Finally, the control unit 2a renders a tablet surface image showing the tablet surface 4a based on the tablet surface information and the viewpoint information (step S33. Tablet surface image rendering step (processing)). The tablet surface image may imitate the touch surface of a tablet terminal, or may simply be a rectangular frame. It is preferable that the control unit 2a renders the tablet surface image based on the viewpoint information acquired in step S20 of Figure 4 so that the angle (depression angle) between the user's line of sight and the tablet surface 4a is a predetermined value.

[0052] Returning to FIG. 3, the control unit 2a that has executed the tablet surface display process subsequently accepts a selection operation of an operation mode by the user (step S4). This selection may be executed, for example, by the user pressing a switch provided on the electronic pen 5, or may be executed by the user changing the distance between the tablet surface 4a and the electronic pen 5. For example, when the distance between the tablet surface 4a and the electronic pen 5 becomes closer than a predetermined distance, the display may be switched to 2D display, and when the distance between the tablet surface 4a and the electronic pen 5 becomes farther than the predetermined distance, the display may be automatically switched to 3D display. In the latter case, the distance between the tablet surface 4a and the electronic pen 5 may be detected by a touch sensor, or may be detected by the control unit 2a based on the display position of the tablet surface image and the position of the electronic pen 5 detected using the position sensor 8c. The control unit 2a that has accepted the selection operation in step S4 performs a process of entering either the 3D display mode or the 2D display mode (i.e., a display selection process of selecting either the 3D display or the 2D display) according to the selection content (step S5. Mode selection step).

[0053] When the 3D display mode is entered in step S5, the control unit 2a executes a 3D rendering step (processing) of rendering a 3D object in the virtual reality space coordinate system on the virtual reality display 3. Specifically, the control unit 2a first performs rendering of the input object and other 3D objects based on the 3D object stored in the memory 2b and the viewpoint information acquired in step S20 of FIG. 4 (step S6). Then, the output (display) to the virtual reality display 3 is updated based on the result (step S7, display update step (processing)). At this time, other displays in the virtual reality space, such as the display surface image rendered in step S30 of FIG. 5 and the tablet surface image rendered in step S33 of FIG. 5, are also updated at the same time. This enables the user to edit the input object by 3D input.

[0054] The control unit 2a further updates the 3D object of the object being input stored in the memory 2b based on the controller information acquired in step S22 of Fig. 4 (step S8, 3D object update step (processing)). After that, the process returns to step S2 and continues.

[0055] Here, the updating of the 3D object in step S8 and step S14 described later is also performed based on the operation information notified from the electronic pen 5. For example, when data indicating the output of a capacitance sensor or a force sensor is notified from the electronic pen 5, the computer 2 acquires this data as pen pressure data, and determines the line width and transparency of the object being input based on the acquired pen pressure data. Then, the determination result is reflected in the 3D object.

[0056] When the control unit 2a enters the 2D display mode in step S5, the control unit 2a executes a 2D rendering step (processing) of rendering the 3D object on the virtual reality display 3 as a 2D displayed 3D object in the display surface coordinate system. Specifically, first, based on the first correspondence information acquired in step S31 of FIG. 5, the 3D object of the object being input stored in the memory 2b is converted into a 2D object indicating the shape, position, and orientation of the object being input in the display surface coordinate system (step S10, first conversion step). Then, based on the obtained 2D object and the viewpoint information acquired in step S20 of FIG. 4, rendering of the object being input is performed (step S11), and the output (display) to the virtual reality display 3 is updated based on the result (step S12, display update step (processing)). At this time, similarly to step S7, other displays in the virtual reality space, such as the display surface image rendered in step S30 of FIG. 5 and the tablet surface image rendered in step S33 of FIG. 5, are updated at the same time. This enables the user to edit the object being input by 2D input.

[0057] Next, the control unit 2a converts the indicated position indicated by the electronic pen information acquired in step S23 of FIG. 4 into a position in the virtual reality space coordinate system based on the second correspondence information acquired in step S32 of FIG. 5 (step S13, second conversion step). Then, based on the electronic pen information including the obtained indicated position, the 3D object of the object being input stored in the memory 2b is updated (step S14, 3D object update step). As described above, this update is also performed based on the operation information notified from the electronic pen 5. The control unit 2a then returns to step S2 and continues the process.

[0058] FIG. 6 is a diagram showing a state in which a 3D object is being input in 2D into a virtual reality space (a state in which the user has selected 2D display mode in step S4 of FIG. 3), and FIG. 7 is a diagram showing a state in which a 3D object is being input in 3D into a virtual reality space (a state in which the user has selected 3D display mode in step S4 of FIG. 3).

[0059] As shown in FIG. 6, in the 2D display mode, an object 13 being inputted is displayed in 2D in a rectangular display surface image 11 displayed in a virtual reality space 10. The three axes DRX, DRY, and DRZ shown in the display surface image 11 represent a display surface coordinate system. The axis DRZ among the three axes is a normal direction of the display surface image 11. A rectangular tablet surface image 12 is also displayed, and the user edits the object 13 being inputted by moving the electronic pen 5 within the tablet surface image 12. This editing is performed by moving the electronic pen 5 on a plane, so it is a 2D input. Although it is not visible to the user wearing the virtual reality display 3, the tablet surface 4a actually exists at the position where the tablet surface image 12 is displayed, so the user can move the electronic pen 5 within the tablet surface image 12 while feeling the response of the tablet surface 4a.

[0060] 7, in the 3D display mode, the object 13 being input is displayed in 3D in the virtual reality space 10. The user edits the object 13 being input by moving the electronic pen 5 and the glove unit 6 in this virtual reality space 10. This editing is performed using a controller for 3D input, and is therefore 3D input.

[0061] As described above, according to the 3D object rendering system 1 of this embodiment, the display method (3D display or 2D display) of a 3D object in a virtual reality space can be switched by the user's selection, so that intuitive and highly accurate drawing can be realized in the virtual reality space. Also, while the 2D display is being performed, editing of the 3D object is possible by 2D input, and while the 3D display is being performed, editing of the 3D object is possible by 3D input, so that editing of the 3D object can be performed by an input method suitable for the display method.

[0062] Furthermore, according to the 3D object rendering system 1 of this embodiment, it becomes possible to input a 3D object with the electronic pen 5 in the virtual reality space.

[0063] Next, a 3D object rendering system 1 according to a second embodiment of the present invention will be described. The 3D object rendering system 1 according to this embodiment differs from the 3D object rendering system 1 according to the first embodiment in that the display surface coordinate system and the tablet surface coordinate system are the same coordinate system, in that these are different coordinate systems. Since the 3D object rendering system 1 according to the second embodiment is otherwise the same as the 3D object rendering system 1 according to the first embodiment, the same components as those in the first embodiment are given the same reference numerals, and the following description will focus on the differences from the first embodiment.

[0064] FIG. 8 is a diagram showing a state in which a 3D object is being input in 2D in a virtual reality space according to this embodiment. First, an overview of this embodiment will be described with reference to FIG. 8. The control unit 2a according to this embodiment is configured to render a tablet terminal image 14 showing the tablet 4 (tablet terminal) shown in FIG. 1 in a virtual reality space. The tablet surface image 14a included in the tablet terminal image 14 corresponds to both the tablet surface image and the display surface image described in the first embodiment, and therefore, in this embodiment, the tablet surface coordinate system (axes TRX, TRY, TRZ) also serves as the display surface coordinate system. As a result, in the 2D display mode, the input object 13 is displayed in 2D in the tablet surface image 14a as shown in FIG. 8.

[0065] Fig. 9 is a flow diagram showing a part of the process performed by the control unit 2a according to this embodiment. Fig. 9(a) replaces the flow diagram shown in Fig. 5. Meanwhile, Fig. 9(b) replaces step S10 shown in Fig. 3.

[0066] As shown in Fig. 9(a), the control unit 2a according to this embodiment is configured to execute step S33a instead of step S33, without executing steps S30 and S31 shown in Fig. 5. In step S33a, rendering of a tablet terminal image showing the tablet 4 (tablet terminal) shown in Fig. 1 is performed based on the tablet surface information and viewpoint information (tablet terminal image rendering step (processing)). As a result, a tablet terminal (tablet terminal image 14) similar to the real tablet 4 (tablet terminal) appears in the virtual reality space 10, as shown in Fig. 8.

[0067] In addition, as shown in Fig. 9(b), in order to obtain 2D object information used when rendering the object being input in the 2D display mode, the control unit 2a according to this embodiment performs a process of converting the 3D object of the object being input stored in the memory 2b into a 2D object indicating the shape, position, and orientation of the object being input in the display surface coordinate system based on the second correspondence information acquired in step S32 instead of the first correspondence information (step S10a). As a result, the object being input 13 (i.e., the position of the electronic pen 5 in the virtual reality space coordinate system acquired in the controller information acquisition step (step S22)) is displayed in the tablet surface image 14a as shown in Fig. 8.

[0068] The 3D object rendering system 1 according to this embodiment also allows the user to select the display method (3D display or 2D display) of a 3D object in a virtual reality space, making it possible to realize intuitive and highly accurate drawing in the virtual reality space. In addition, since the 3D object can be edited by 2D input during 2D display and the 3D object can be edited by 3D input during 3D display, it becomes possible to edit the 3D object using an input method suitable for the display method.

[0069] In addition, according to the present embodiment, the user can experience inputting data to a tablet device in a virtual reality space, and can therefore perform input operations for 3D objects as if the user were performing input operations on a normal tablet device.

[0070] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and it goes without saying that the present invention can be embodied in various forms without departing from the spirit of the present invention. [Explanation of symbols]

[0071] 1. 3D Object Rendering System 2. Computer 2a Control section 2b Memory 3 Virtual reality displays 4 Tablet 4a Tablet surface 5 Electronic pen 6 Globe Unit 7a,7b Lightning House 8a~8d Position sensors 10 Virtual Reality Space 11 Display surface 12 Tablet surface image 13 Input Object 14 Tablet device images 14a Tablet surface image

Claims

1. A computer having a control unit for rendering an object in a virtual space, The control unit is a rendering process for rendering, in the virtual space, a plane image representing a plane in the real space used when inputting a position designated by a stylus; a rendering process for rendering a display image showing the object in the virtual space, the display image being different from the planar image; and displaying, on the display image, the object that is generated based on a position on the plane indicated by the stylus. computer.

2. The control unit is a rendering process for rendering, in the virtual space, a stylus image that moves in accordance with the movement of the stylus on the plane; The computer of claim 1.

3. The position indicated by the stylus is detected by a position detection sensor located below the plane. The computer of claim 1.

4. The control unit is and further performing a conversion process for converting the position detected by the position detection sensor into a position in a coordinate system of the virtual space. The computer of claim 3.

5. The plane is configured not to have a function of detecting a position indicated by the stylus. The computer of claim 1.

6. The control unit is executing a display process for displaying, on the display image, the object that is generated based on the pressure detected by a sensing device of the stylus; The computer of claim 1.

7. the pressure being the pressure exerted by the flat surface on the stylus when the stylus contacts the flat surface; The computer of claim 6.

8. the flat image is a tablet terminal image showing a tablet terminal, The plane in the real space is a tablet surface of the tablet terminal. The computer of claim 1.

Images