Computer, rendering method, and program

Abstract

To perform an input operation on a 3D object in a virtual space as if performing an input operation with a stylus on a plane in a real space.SOLUTION: A computer according to the present invention has a control unit that renders an object in a virtual space. The control unit executes: a rendering process of rendering in the virtual space a plane image indicating a plane in a real space used when inputting a position indicated by a stylus; and a display process of displaying on the plane image an object image generated based on a position on the plane in the real space indicated by the stylus.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to a rendering device and a rendering method for rendering 3D objects in a virtual reality (including VR: Virtual Reality, AR: Augmented Reality, MR: Mixed Reality) space.

Background Art

[0002] In recent years, there has been an increasing need to design various products while viewing them stereoscopically in a virtual reality space.

[0003] Patent Document 1 discloses a technique for generating a 3D object based on a 2D object input to a tablet terminal using an electronic pen in an AR space. Hereinafter, an input method performed by moving an electronic pen on a plane is referred to as "2D input".

[0004] Non-Patent Document 1 discloses a technique for inputting a 3D object by moving a 3D input controller in a VR space. Hereinafter, an input method using such a 3D input controller is referred to as "3D input".

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] According to the above-mentioned 2D input, since the position of the electronic pen is fixed within a known plane, higher accuracy can be obtained compared to 3D input. However, on the other hand, 2D input is limited to drawing within a plane, so there is a problem that intuitive operation is difficult compared to 3D input.

[0008] On the contrary, according to 3D input, intuitive operation becomes possible. On the other hand, since the degree of freedom of the position of the controller is high, there is a problem that the accuracy is insufficient as a technology used for design applications.

[0009] Therefore, one of the objects of the present invention is to provide a rendering device and a rendering method capable of realizing intuitive and highly accurate drawing in a virtual reality space.

Means for Solving the Problems

[0010] A first aspect of the present invention relates to a rendering device that renders a 3D object in a virtual reality space displayed on a virtual reality display, the 3D rendering step of rendering the 3D object as a 3D object in 3D display in a virtual reality space coordinate system, the 2D rendering step of rendering the 3D object as a 3D object in 2D display in a plane coordinate system, a 3D display mode for updating the display of the virtual reality display based on the rendering result of the 3D rendering step, and a 2D display mode for updating the display of the virtual reality display based on the rendering result of the 2D rendering step, and a rendering device that executes a display update step having the above.

[0011] A second aspect of the present invention relates to a rendering method for causing a computer to function as a rendering device that renders a 3D object in a virtual reality space displayed on a virtual reality display. The method includes causing the computer to execute a 3D rendering step of rendering the 3D object as a 3D object for 3D display in a virtual reality space coordinate system, a 2D rendering step of rendering the 3D object as a 3D object for 2D display in a plane coordinate system, a 3D display mode of updating the display of the virtual reality display based on the rendering result of the 3D rendering step, and a 2D display mode of updating the display of the virtual reality display based on the rendering result of the 2D rendering step, and a display update step having the above.

Advantages of the Invention

[0012] According to the present invention, since the display method (3D display or 2D display) of the 3D object in the virtual reality space can be switched according to the user's selection, it becomes possible to realize intuitive and highly accurate drawing in the virtual reality space.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0015] FIG. 1 is a diagram showing the configuration of a 3D object rendering system 1 according to the first embodiment of the present invention. As shown in the figure, the 3D object rendering system 1 according to the present embodiment includes a computer 2, a virtual reality display 3, a tablet 4, an electronic pen 5, a glove unit 6, lighting houses 7a and 7b, and position sensors 8a to 8d. The position sensors 8a, 8c, and 8d are attached to the tablet 4, the electronic pen 5, and the glove unit 6, respectively, and the position sensor 8b is provided on the virtual reality display 3. By attaching the position sensor 8c, the electronic pen 5 functions as a stylus and also as a controller. Note that the position sensor 8c may be built into the electronic pen 5.

[0016] Each device shown in FIG. 1 is generally arranged in a room. In the 3D object rendering system 1, almost the entire room can be used as a virtual reality space.

[0017] The computer 2 includes a control unit 2a (controller) and a memory 2b that cooperates with the control unit 2a. Each process performed by the computer 2 described later can be realized by the control unit 2a cooperating with the memory 2b (more specifically, by reading and executing a program stored in the memory 2b).

[0018] The computer 2 is connected to each of the virtual reality display 3 and the lighting houses 7a, 7b, either by wire or wirelessly. FIG. 1 shows an example in which the computer 2 is connected to each of the virtual reality display 3 and the lighting houses 7a, 7b by a wired communication standard such as USB. Further, although details will be described later, when the tablet 4 is a device having a communication function, the computer 2 is also connected to the tablet 4, either by wire or wirelessly. FIG. 1 shows an example in which the computer 2 and the tablet 4 are connected by a short-range wireless communication standard such as Bluetooth (registered trademark). When the tablet 4 or the virtual reality display 3 incorporates the function of a computer, the computer may constitute the computer 2.

[0019] The computer 2 is configured to have a function of displaying a virtual reality space on the virtual reality display 3. This virtual reality space may be a VR (Virtual Reality) space, an AR (Augmented Reality) space, or an MR (Mixed Reality) space. When displaying a VR space, a user wearing the virtual reality display 3 recognizes virtual reality and is separated from the real world. On the other hand, when displaying an AR space or an MR space, a user wearing the virtual reality display 3 recognizes a space in which virtual reality and the real world are mixed.

[0020] The computer 2 is configured to function as a rendering device that sets a virtual reality space based on the positions of the lighting houses 7a, 7b and renders various 3D objects within the set virtual reality space. The computer 2 updates the display of the virtual reality display 3 according to the result of rendering. As a result, various 3D objects will appear in the virtual reality space displayed on the virtual reality display 3.

[0021] Rendering by computer 2 is executed based on the 3D objects stored in memory 2b. The 3D objects are information indicating the shape, position, and orientation of the 3D objects in the virtual reality space coordinate system showing the virtual reality space set by computer 2, and are stored in memory 2b for each 3D object to be rendered.

[0022] The 3D objects rendered by computer 2 include 3D objects representing the tablet 4, the electronic pen 5, and the glove unit 6 shown in FIG. 1 respectively. In rendering these 3D objects, computer 2 first detects the position and orientation of each of the position sensors 8a to 8d in the virtual reality space coordinate system. Then, based on the detected position and orientation of the position sensor 8b, viewpoint information indicating the user's viewpoint is obtained, and based on the obtained viewpoint information, the shape of each stored 3D object, and the detected positions and orientations of the position sensors 8a, 8c, 8d respectively, the 3D objects representing the tablet 4, the electronic pen 5, and the glove unit 6 are configured to be rendered in the virtual reality space.

[0023] Computer 2 is further configured to detect an operation performed by the user in the virtual reality space by detecting the positions of the position sensors 8c and 8d, and based on the result, newly create a 3D object or update the 3D object already held.

[0024] The virtual reality display 3 is a VR display (head-mounted display) worn on a human head. Generally commercially available virtual reality displays include various types such as "transmissive" or "non-transmissive", "eyeglass type" or "hat type", etc., but any of them can be used as the virtual reality display 3.

[0025] The virtual reality display 3 is connected to each of the position sensor 8a, the electronic pen 5 (including the position sensor 8c), and the glove unit 6 (including the position sensor 8d) by wire or wirelessly. The position sensors 8a, 8c, and 8d are configured to notify the virtual reality display 3 of light reception level information described later through this connection. The virtual reality display 3 is configured to notify the computer 2 of the light reception level information notified from each of the position sensors 8a, 8c, and 8d together with the light reception level information of the position sensor 8b built in the virtual reality display 3. The computer 2 detects the position and orientation of each of the position sensors 8a to 8d in the virtual reality space coordinate system based on the light reception level information thus notified. In addition, the electronic pen 5 and the glove unit 6 are configured to notify the virtual reality display 3 of operation information described later through the above connection. The virtual reality display 3 is configured to transfer the operation information thus notified to the computer 2.

[0026] The tablet 4 has a tablet surface 4a. The tablet surface 4a is preferably a flat surface and may be made of a material suitable for sliding the pen tip of the electronic pen 5. In one example, the tablet 4 is a so-called digitizer, and is configured with a touch sensor that detects the pointing position of the electronic pen 5 in the touch surface and a communication function that notifies the computer 2 of the detected pointing position. In this case, the tablet surface 4a is configured by the touch surface of the digitizer. In another example, the tablet 4 is a so-called tablet computer, and is configured with a display, a touch sensor that detects the pointing position of the electronic pen 5 in the display surface of the display, and a communication function that notifies the computer 2 of the detected pointing position. In this case, the tablet surface 4a is configured by the display surface of the display. In yet another example, the tablet 4 is a physical object that does not have the function of detecting the pointing position of the electronic pen 5 (including a simple board, a table, a display or a computer that does not have the function of detecting the pointing position of the electronic pen 5, etc.). In this case, the tablet surface 4a is configured by a plane provided on the surface of the tablet 4.

[0027] The position sensor 8a is fixedly installed on the surface of the tablet 4. Therefore, the position and orientation of the position sensor 8a detected by the computer 2 indicate the position and orientation of the tablet surface 4a in the virtual reality space coordinate system.

[0028] The electronic pen 5 and the glove unit 6 are for the user to indicate positions within the virtual reality space. The electronic pen 5 is configured to have a pen-like shape. The glove unit 6 has the shape of a glove to be worn on the user's hand.

[0029] Various sensing devices such as switches are provided on or inside the surface of the electronic pen 5. The sensing devices mentioned here include, in addition to a toggle switch that takes either an on or off state, sensors configured to be able to detect any physical quantity. Examples of switches provided on the electronic pen 5 include a side switch or a tail switch configured to receive an on / off operation by the user. Also, examples of other switches provided on the electronic pen 5 include a capacitance sensor that detects the pressure (pen pressure) applied to the tip of the electronic pen 5. The electronic pen 5 is configured to detect the output (depressed state or detected physical quantity) of the switch provided on itself and notify all or part of the detection result to the virtual reality display 3 as its own operation information.

[0030] The sensing devices provided on the electronic pen 5 may also include a force sensor (load sensor) that detects the gripping force of the electronic pen 5 by the user. In this case, although the output of the force sensor does not originally indicate the pen pressure, the computer 2 that receives the notification of the output of the force sensor may handle this as data indicating the pen pressure (pen pressure data). By doing so, even when the user operates the electronic pen 5 in the air, it becomes possible to reflect the pen pressure in the drawing result.

[0031] When the tablet 4 has a touch sensor, the position of the electronic pen 5 is also detected by this touch sensor. The position detected by the touch sensor is not the position in the virtual reality space coordinate system, but the position in the tablet surface coordinate system defined on the tablet surface 4a. The touch sensor is configured to notify the computer 2 of the detected position of the electronic pen 5.

[0032] Generally, the position detected by the touch sensor is more accurate than the position detected using the position sensor 8c. Therefore, when the position of the electronic pen 5 is notified from the touch sensor, it is preferable for the computer 2 to acquire the position notified from the touch sensor as the position of the electronic pen 5 instead of the position detected through the position sensor 8c. In this case, it is preferable for the computer 2 to convert the position notified from the touch sensor into the position in the virtual reality space coordinate system by associating the tablet surface coordinate system with the virtual reality space coordinate system based on the position and orientation of the tablet surface 4a detected using the position sensor 8a.

[0033] FIG. 2 is a diagram for explaining the relationship between the tablet surface coordinate system and the virtual reality space coordinate system. The figure shows a state where the tablet surface 4a is located in the virtual reality space 10. The virtual reality space coordinate system is defined by three axes VRX, VRY, VRZ, and the tablet surface coordinate system is defined by three axes TRX, TRY, TRZ. However, the axis TRZ is the normal direction of the tablet surface 4a. When the illustrated position P is detected by the touch sensor as the position (x, y, z) in the tablet surface coordinate system (z indicates, for example, the hover position), the computer 2 converts this (x, y, z) into the position (X, Y, Z) in the virtual reality space coordinate system by a predetermined conversion process. Thereby, it becomes possible to convert the position notified from the touch sensor into the position in the virtual reality space coordinate system.

[0034] Here, for the position detection of the electronic pen 5 by the touch sensor, either an electromagnetic induction method or an active electrostatic method may be used. When the active electrostatic method is used, the touch sensor is configured to send a beacon signal at a predetermined time interval from sensor electrodes (not shown) arranged in the touch surface. The beacon signal includes a command for controlling the electronic pen 5 from the touch sensor. The content of the control by the command includes, for example, causing the pen pressure data (detected by a capacitance sensor) indicating the pressure applied to the pen tip of the electronic pen 5 to be transmitted, causing the pressed state of various switches (not shown) provided on the electronic pen 5 to be transmitted, and causing the unique ID pre-stored in the electronic pen 5 to be transmitted, etc.

[0035] When the electronic pen 5 corresponding to the active electrostatic method detects the above beacon signal, it sends a pen signal as a response signal. The pen signal is a signal including a burst signal that is an unmodulated carrier wave and a data signal obtained by modulating the carrier wave with data corresponding to the above command. The touch sensor attempts to detect the burst signal by the above sensor electrodes, and detects the position of the electronic pen 5 based on the detection result. Also, by detecting and demodulating the data signal by the above sensor electrodes, the data transmitted by the electronic pen 5 in response to the command is received. The tablet 4 is configured to transmit the position of the electronic pen 5 and the data transmitted by the electronic pen 5 thus obtained to the computer 2. The computer 2 is configured to convert the position thus notified into the position in the virtual reality space coordinate system as described above, and to acquire the notified data as a part of the operation information described above.

[0036] The lightning houses 7a and 7b are signal transmitting devices for position detection used in the 3D object rendering system 1, and are each configured to emit a signal, in this example laser light, while changing the direction according to the control by the computer 2. The position sensors 8a to 8d are each composed of a plurality of light receiving sensors, and are configured to receive the signals (laser light) irradiated by each of the lightning houses 7a and 7b by the respective light receiving sensors and acquire light reception level information including the respective light reception levels. The acquired light reception level information is notified from each of the position sensors 8a to 8d to the computer 2 as described above and is used to detect these positions and orientations.

[0037] The overall outline of the 3D object rendering system 1 has been described above. When a user inputs a new 3D object in such a 3D object rendering system 1, hitherto, 3D input has been performed using the electronic pen 5 and the glove unit 6, or the tablet 4 has been used as a tablet computer and 2D input has been performed on this tablet computer using the electronic pen 5. However, as described above, 3D input has the advantage of enabling an intuitive operation while having the disadvantage of insufficient accuracy, and 2D input has the advantage of being able to obtain high accuracy while having the disadvantage of being difficult to operate intuitively.

[0038] The 3D object rendering system 1 according to the present embodiment, in view of the problems of such conventional input methods, enables intuitive and highly accurate drawing in the virtual reality space by making it possible to switch the display method (3D display or 2D display) of the 3D object in the virtual reality space according to the user's selection. Hereinafter, this point will be described in detail with reference to the flowchart of the processing performed by the control unit 2a of the computer 2.

[0039] Figure 3 is a flowchart showing the processing performed by the control unit 2a of the computer 2. This processing is executed when the user inputs a 3D object using at least one of the electronic pen 5 and the glove unit 6, and is started when the user performs a predetermined operation using the electronic pen 5 and the glove unit 6 and the control unit 2a detects it.

[0040] As shown in FIG. 3, the control unit 2a first secures in the memory 2b a storage area for a 3D object indicating the shape, position, and orientation of the object being input in the virtual reality space coordinate system (step S1). The specific format of the 3D object is not particularly limited, but for example, data in the VRML format or the X3D format is preferably used.

[0041] Next, the control unit 2a executes position information acquisition processing (step S2).

[0042] Figure 4 is a flowchart showing the details of the position information acquisition processing executed in step S2. As shown in the figure, the control unit 2a executes the position information acquisition processing by executing each of a step of acquiring viewpoint information (step S20), a step of acquiring tablet surface information (step S21), a step of acquiring controller information (first information) (step S22. Controller information acquisition step. First information acquisition processing), and a step of acquiring electronic pen information (second information) (step S23. Electronic pen information acquisition step. Second information acquisition processing). Note that the execution order of steps S20 to S23 is not particularly limited.

[0043] The viewpoint information is information representing the user's viewpoint in the virtual reality space coordinate system, and specifically is indicated by the position and orientation of the virtual reality display 3. The control unit 2a is configured to acquire the viewpoint information based on the position and orientation detected by the position sensor 8b. The specific viewpoint information is constituted by, for example, vector information starting from one three-dimensional coordinate.

[0044] The tablet surface information is information indicating the shape, position, and orientation of the tablet surface 4a in the virtual reality space coordinate system, and is stored in the memory 2b as one of the 3D objects. The control unit 2a acquires the tablet surface information based on the position and orientation detected by the position sensor 8a and the shape of the tablet 4 stored in advance.

[0045] The controller information is information indicating the position and orientation of the 3D controller (including the electronic pen 5 and the glove unit 6) in the virtual reality space coordinate system, and the operation information of the 3D controller. The control unit 2a acquires the controller information based on the position and orientation detected by the position sensors 8c and 8d and the operation information of each of the electronic pen 5 and the glove unit 6 received via the virtual reality display 3.

[0046] The electronic pen information is information indicating the pointing position of the electronic pen 5 in the tablet surface coordinate system and the operation information of the electronic pen 5. When the tablet 4 has a touch sensor, the control unit 2a acquires the pointing position of the electronic pen 5 in the tablet surface coordinate system from the touch sensor. On the other hand, when the tablet 4 does not have a touch sensor, the control unit 2a performs a conversion process (the reverse process of the conversion process described with reference to FIG. 2) on the position acquired from the position sensor 8c (the position in the virtual reality space coordinate system) to acquire the pointing position of the electronic pen 5 in the tablet surface coordinate system. Further, when the control unit 2a can acquire the operation information of the electronic pen 5 from the touch sensor (for example, when the electronic pen 5 corresponds to the active electrostatic method), the control unit 2a acquires the operation information of the electronic pen 5 (including the output of the capacitance sensor) from the touch sensor. On the other hand, when the operation information of the electronic pen 5 cannot be acquired from the touch sensor, the control unit 2a acquires the operation information of the electronic pen 5 (including the output of the force sensor) via the virtual reality display 3.

[0047] Returning to FIG. 3, the control unit 2a that has executed the position information acquisition process and the like then executes a tablet surface display process (step S3). This process is for displaying a tablet surface image indicating the tablet surface and a display surface image indicating a display surface for two-dimensionally displaying the 3D object being input in the virtual reality space.

[0048] FIG. 5 is a flowchart showing details of the tablet surface display process executed in step S3. As shown in the figure, the control unit 2a first performs rendering of the display surface image based on the position selected by the user in the virtual reality space or the position in contact with the 3D object being input (step S30, display surface image rendering step (process)). The selection of the position by the user is executed, for example, by the user pressing a switch provided on the electronic pen 5 and operation information indicating this being notified to the control unit 2a. The display surface image may be imitative of a display or may simply be a rectangular frame. Note that it is preferable for the control unit 2a to render the display surface image based on the viewpoint information acquired in step S20 of FIG. 4 so that the normal direction of the display surface image coincides with the user's line of sight direction.

[0049] Next, the control unit 2a acquires first correspondence information indicating the correspondence between the virtual reality space coordinate system and the display surface coordinate system (first planar coordinate system) defined on the display surface (step S31). Specifically, the first correspondence information is a conversion rule for mutually converting the virtual reality space coordinate system and the display surface coordinate system.

[0050] Subsequently, the control unit 2a acquires second correspondence information indicating the correspondence between the virtual reality space coordinate system and the tablet surface coordinate system (second planar coordinate system) (step S32). Specifically, the second correspondence information is a conversion rule for mutually converting the virtual reality space coordinate system and the tablet surface coordinate system.

[0051] Finally, the control unit 2a performs rendering of a tablet surface image indicating the tablet surface 4a based on the tablet surface information and the viewpoint information (step S33. Tablet surface image rendering step (process)). The tablet surface image may imitate the touch surface of the tablet terminal, or may be a simple rectangular frame. Note that the control unit 2a preferably renders the tablet surface image such that the angle (depression angle) formed between the user's line of sight direction and the tablet surface 4a becomes a predetermined value based on the viewpoint information acquired in step S20 of FIG. 4.

[0052] Returning to FIG. 3, the control unit 2a that has executed the tablet surface and the like display process then accepts a selection operation of the operation mode by the user (step S4). This selection may be executed, for example, by the user pressing a switch provided on the electronic pen 5, or may be executed by the user changing the distance between the tablet surface 4a and the electronic pen 5. For example, when the distance between the tablet surface 4a and the electronic pen 5 approaches a predetermined distance, it may be switched to 2D display, and when the distance between the tablet surface 4a and the electronic pen 5 moves away from the predetermined distance or more, it may be automatically switched to 3D display. In the latter case, the distance between the tablet surface 4a and the electronic pen 5 may be detected by the touch sensor, or may be detected by the control unit 2a based on the display position of the tablet surface image and the position of the electronic pen 5 detected using the position sensor 8c. The control unit 2a that has accepted the selection operation in step S4 performs a process of entering either the 3D display mode or the 2D display mode according to the selection content (that is, a display selection process of selecting either 3D display or 2D display) (step S5. Mode selection step).

[0053] When entering the 3D display mode in step S5, the control unit 2a executes a 3D rendering step (process) of rendering a 3D object in the virtual reality space coordinate system onto the virtual reality display 3. Specifically, the control unit 2a first performs rendering of the object being input and other 3D objects based on the 3D object stored in the memory 2b and the viewpoint information acquired in step S20 of FIG. 4 (step S6). Then, the output (display) to the virtual reality display 3 is updated based on the result (step S7. Display update step (process)). At this time, other displays in the virtual reality space, such as the display surface image rendered in step S30 of FIG. 5 and the tablet surface image rendered in step S33 of FIG. 5, are also updated simultaneously. As a result, the user can edit the object being input by 3D input.

[0054] The control unit 2a further updates the 3D object of the object being input stored in the memory 2b based on the controller information acquired in step S22 of FIG. 4 (step S8. 3D object update step (process)). Then, the process returns to step S2 and continues.

[0055] Here, the update of the 3D object in step S8 and step S14 described later is also executed based on the operation information notified from the electronic pen 5. For example, when data indicating the output of the capacitance sensor or the output of the force sensor is notified from the electronic pen 5, the computer 2 acquires this data as pen pressure data, determines the line width and transparency of the object being input based on the acquired pen pressure data, and reflects the determination result in the 3D object.

[0056] When entering the 2D display mode in step S5, the control unit 2a executes a 2D rendering step (process) of rendering a 3D object on the virtual reality display 3 as a 2D object of the 3D object in the display plane coordinate system. Specifically, first, based on the first correspondence information acquired in step S31 of FIG. 5, the 3D object of the object being input stored in the memory 2b is converted into a 2D object indicating the shape, position, and orientation of the object being input in the display plane coordinate system (step S10. First conversion step). Then, based on the obtained 2D object and the viewpoint information acquired in step S20 of FIG. 4, rendering of the object being input is performed (step S11), and the output (display) to the virtual reality display 3 is updated based on the result (step S12. Display update step (process)). Also at this time, similar to step S7, other displays in the virtual reality space, such as the display plane image rendered in step S30 of FIG. 5 and the tablet plane image rendered in step S33 of FIG. 5, are updated simultaneously. Thereby, the user can edit the object being input by 2D input.

[0057] Subsequently, the control unit 2a converts the instruction position indicated by the electronic pen information acquired in step S23 of FIG. 4 into a position in the virtual reality space coordinate system based on the second correspondence information acquired in step S32 of FIG. 5 (step S13. Second conversion step). Then, based on the electronic pen information including the obtained instruction position, the 3D object of the object being input stored in the memory 2b is updated (step S14. 3D object update step). As described above, this update is also executed based on the operation information notified from the electronic pen 5. The control unit 2a then returns to step S2 to continue the process.

[0058] FIG. 6 is a diagram showing a state in which a 3D object is being 2D input in a virtual reality space (a state in which the user has selected the 2D display mode in step S4 of FIG. 3), and FIG. 7 is a diagram showing a state in which a 3D object is being 3D input in a virtual reality space (a state in which the user has selected the 3D display mode in step S4 of FIG. 3).

[0059] As shown in FIG. 6, in the 2D display mode, the object 13 being input is 2D displayed within the rectangular display surface image 11 displayed in the virtual reality space 10. Note that the three axes DRX, DRY, and DRZ shown in the display surface image 11 represent the display surface coordinate system. Among the three axes, the axis DRZ is the normal direction of the display surface image 11. Also, the rectangular tablet surface image 12 is displayed, and the user edits the object 13 being input by moving the electronic pen 5 within this tablet surface image 12. Since this editing is performed by moving the electronic pen 5 on a plane, it is 2D input. Although not visible to the user wearing the virtual reality display 3, since the tablet surface 4a actually exists at the position where the tablet surface image 12 is displayed, the user can move the electronic pen 5 within the tablet surface image 12 while feeling the tactile sensation of the tablet surface 4a.

[0060] On the other hand, as shown in FIG. 7, in the 3D display mode, the object 13 being input is 3D displayed within the virtual reality space 10. The user edits the object 13 being input by moving the electronic pen 5 and the glove unit 6 within this virtual reality space 10. Since this editing is performed using a 3D input controller, it is 3D input.

[0061] As described above, according to the 3D object rendering system 1 according to the present embodiment, since the display method (3D display or 2D display) of the 3D object in the virtual reality space can be switched by the user's selection, it is possible to realize intuitive and highly accurate rendering in the virtual reality space. Further, while performing 2D display, it is possible to edit the 3D object by 2D input, and while performing 3D display, it is possible to edit the 3D object by 3D input. Therefore, it is possible to edit the 3D object by an input method suitable for the display method.

[0062] Further, according to the 3D object rendering system 1 according to the present embodiment, it is possible to input a 3D object with the electronic pen 5 in the virtual reality space.

[0063] Next, the 3D object rendering system 1 according to the second embodiment of the present invention will be described. The 3D object rendering system 1 according to the present embodiment is different from the 3D object rendering system 1 according to the first embodiment in that the display surface coordinate system and the tablet surface coordinate system are the same coordinate system, and these are different coordinate systems from each other. Since it is the same as the 3D object rendering system 1 according to the first embodiment in other respects, the same components as those in the first embodiment are denoted by the same reference numerals, and the differences from the first embodiment will be mainly described below.

[0064] FIG. 8 is a diagram showing a state in which a 3D object is 2D-input in a virtual reality space according to the present embodiment. First, the outline of the present embodiment will be described with reference to FIG. 8. The control unit 2a according to the present embodiment is configured to render a tablet terminal image 14 showing the tablet 4 (tablet terminal) shown in FIG. 1 in the virtual reality space. The tablet surface image 14a included in the tablet terminal image 14 corresponds to both the tablet surface image and the display surface image described in the first embodiment. Therefore, in the present embodiment, the tablet surface coordinate system (axes TRX, TRY, TRZ) also serves as the display surface coordinate system. As a result, in the 2D display mode, as shown in FIG. 8, the object 13 being input is 2D-displayed within the tablet surface image 14a.

[0065] FIG. 9 is a flowchart showing a part of the processing performed by the control unit 2a according to the present embodiment. FIG. 9(a) replaces the flowchart shown in FIG. 5. On the other hand, FIG. 9(b) replaces step S10 shown in FIG. 3.

[0066] As shown in FIG. 9(a), the control unit 2a according to the present embodiment is configured to execute step S33a instead of steps S30 and S31 shown in FIG. 5. In step S33a, based on the tablet surface information and the viewpoint information, rendering of a tablet terminal image showing the tablet 4 (tablet terminal) shown in FIG. 1 is performed (tablet terminal image rendering step (processing)). As a result, as shown in FIG. 8, a tablet terminal (tablet terminal image 14) similar to the actual tablet 4 (tablet terminal) appears in the virtual reality space 10.

[0067] Also, as shown in FIG. 9(b), the control unit 2a according to the present embodiment obtains the 2D object information used when rendering the input object in the 2D display mode. Based on the second correspondence information acquired in step S32 instead of the first correspondence information, the control unit 2a performs a process of converting the 3D object of the input object stored in the memory 2b into a 2D object indicating the shape, position, and orientation of the input object in the display plane coordinate system (step S10a). As a result, as shown in FIG. 8, the input object 13 (that is, the position of the electronic pen 5 in the virtual reality space coordinate system acquired by the controller information acquisition step (step S22)) is displayed in the tablet surface image 14a.

[0068] Also, with the 3D object rendering system 1 according to the present embodiment, since the display method (3D display or 2D display) of the 3D object in the virtual reality space can be switched according to the user's selection, it is possible to realize intuitive and highly accurate drawing in the virtual reality space. In addition, while the 2D display is being performed, the 3D object can be edited by 2D input, and while the 3D display is being performed, the 3D object can be edited by 3D input. Therefore, it is possible to edit the 3D object using an input method suitable for the display method.

[0069] In addition, according to the present embodiment, the user can experience input to the tablet terminal in the virtual reality space. Therefore, it is possible to perform an input operation on the 3D object as if performing an input operation on a normal tablet terminal.

[0070] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to such embodiments, and it goes without saying that the present invention can be implemented in various modes without departing from the gist thereof.

Description of Reference Numerals

[0071] 1 3D object rendering system 2 Computers 2a Control Unit 2b Memory 3 Virtual Reality Display 4 Tablet 4a Tablet Surface 5 Electronic Pen 6 Glove Unit 7a, 7b Lighting House 8a~8d Position Sensor 10 Virtual Reality Space 11 Display Surface 12 Tablet Surface Image 13 Object Being Input 14 Tablet Terminal Image 14a Tablet Surface Image

Claims

1. A computer having a control unit for rendering an object in a virtual space, wherein the control unit, performs a rendering process of rendering a plane image indicating a plane in the real space, which is used when inputting a line composed of a plurality of positions indicated by a stylus, into the virtual space, and a display process of displaying a line image generated based on the line input on the plane in the real space on the plane image in the virtual space, a computer.

2. wherein the control unit, further performs a rendering process of rendering a moving stylus image into the virtual space as the stylus moves on the plane, the computer according to claim 1.

3. wherein the plane image is a tablet terminal image indicating a tablet terminal, and the plane in the real space is the tablet surface of the tablet terminal, the computer according to claim 1.

4. wherein a sensing device is provided on the stylus, and the control unit generates the line image based on the pen pressure data acquired by the sensing device, the computer according to claim 1.

5. A computer having a control unit for rendering an object in a virtual space, wherein the control unit, performs a rendering process of rendering a plane image indicating a plane in the real space, which is used when inputting a position indicated by a stylus, into the virtual space, and a display process of displaying an object image generated based on the position on the plane in the real space indicated by the stylus on the plane image, The position on the plane indicated by the stylus is detected by a position detection sensor provided under the plane, The control unit, further performs a conversion process of converting the position detected by the position detection sensor into a position in the coordinate system of the virtual space, computer.

6. The control unit, further performs a rendering process of rendering a moving stylus image in the virtual space as the stylus moves on the plane, The computer according to claim 5.

7. The plane image is a tablet terminal image showing a tablet terminal, The plane in the real space is the tablet surface of the tablet terminal, The computer according to claim 5.

8. A sensing device is provided on the stylus, The control unit generates the object image based on the pen pressure data acquired by the sensing device, The computer according to claim 5.

9. A rendering method for rendering an object in a virtual space, comprising: rendering a plane image showing a plane in the real space used when inputting a line composed of a plurality of positions indicated by a stylus into the virtual space; displaying a line image generated based on the line input on the plane in the real space on the plane image in the virtual space; and a rendering method including the above.

10. further including the step of rendering a moving stylus image in the virtual space as the stylus moves on the plane, The rendering method according to claim 9.

11. The planar image is a tablet terminal image showing a tablet terminal, and the plane in the real space is the tablet surface of the tablet terminal. The rendering method according to claim 9.

12. The stylus is provided with a sensing device, and the line image is generated based on the pen pressure data acquired by the sensing device. The rendering method according to claim 9.

13. A rendering method for rendering an object in a virtual space, comprising the step of rendering in the virtual space a planar image showing a plane in the real space used when inputting a position indicated by a stylus; and the step of displaying an object image generated based on the position on the plane in the real space indicated by the stylus on the planar image. The position on the plane indicated by the stylus is detected by a position detection sensor provided under the plane, and further comprising the step of converting the position detected by the position detection sensor into a position in the coordinate system of the virtual space. Rendering method.

14. The step of rendering a moving stylus image in the virtual space as the stylus moves on the plane, The rendering method according to claim 13, further comprising this step.

15. The planar image is a tablet terminal image showing a tablet terminal, and the plane in the real space is the tablet surface of the tablet terminal. The rendering method according to claim 13.

16. The stylus is provided with a sensing device, The object image is generated based on the pen pressure data acquired by the sensing device. The rendering method according to claim 13.

17. A program for causing a computer to execute a process of rendering an object in a virtual space, a step of rendering, in the virtual space, a plane image indicating a plane in the real space used when inputting a line composed of a plurality of positions indicated by a stylus; a step of displaying, on the plane image in the virtual space, a line image generated based on the line input on the plane in the real space; A program for causing the computer to execute the steps.

18. A step of rendering, in the virtual space, a moving stylus image as the stylus moves on the plane, The program according to claim 17, for further causing the computer to execute the step.

19. The plane image is a tablet terminal image indicating a tablet terminal, The plane in the real space is the tablet surface of the tablet terminal, The program according to claim 17.

20. The stylus is provided with a sensing device, The line image is generated based on the pen pressure data acquired by the sensing device, The program according to claim 17.

21. A program for causing a computer to execute a process of rendering an object in a virtual space, a step of rendering, in the virtual space, a plane image indicating a plane in the real space used when inputting a position indicated by a stylus; Causing the computer to execute a step of displaying, on the planar image, an object image generated based on a position on a plane within the real space indicated by the stylus. The position on the plane indicated by the stylus is detected by a position detection sensor provided under the plane. Further causing the computer to execute a step of converting the position detected by the position detection sensor into a position in the coordinate system of the virtual space. A program therefor. Claim 22 A step of rendering, within the virtual space, a moving stylus image as the stylus moves on the plane. The program according to claim 21, further causing the computer to execute the step. Claim 23 The planar image is a tablet terminal image showing a tablet terminal. The plane within the real space is the tablet surface of the tablet terminal. The program according to claim 21. Claim 24 The stylus is provided with a sensing device. The object image is generated based on pen pressure data acquired by the sensing device. The program according to claim 21.

Images