Method for generating a three dimensional object based on the operation of a pointing device

Abstract

To enable a user to intuitively and dynamically specify the shape, direction, dimensions, etc., of a multidimensional object represented by digital data by using a single input device.SOLUTION: The method includes defining a two-dimension shape based on a movement of a tip of the position indicator on an inputting surface, determining a direction in which the two-dimension shape extends based on a pressure applied to the inputting surface by the position indicator, an acceleration of the position indicator, or both of them, generating a three dimensional object pushed out from the two-dimension shape in the determined direction, and displaying the three dimensional object on a display apparatus.SELECTED DRAWING: Figure 6A

Description

[Technical Field]

[0001] The present disclosure relates to specifying dimensions of multidimensional objects represented by digital data, and more particularly to systems and methods for dynamically sketching the shape of such multidimensional objects based on intuitive user manipulations performed using a pointing device as an input device. [Background technology]

[0002] Conventionally, a user must perform a complex set of operations with multiple input devices to specify the dimensions of a multidimensional object represented by digital data. For example, conventional systems that enable a user to specify the shape of a multidimensional object represented by digital data may require the user to operate one or more keys on a keyboard with one hand while simultaneously moving a computer mouse and operating buttons on the computer mouse with the other hand to specify the shape, orientation, dimensions, etc. Therefore, it is desirable to provide a system and method that enables a user to intuitively specify the shape, orientation, dimensions, etc. of a multidimensional object represented by digital data using a single input device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent Application Publication No. 2012 / 0206452 [Patent Document 2] U.S. Patent No. 9,939,931 [Patent Document 3] U.S. Patent No. 9,964,395 [Patent Document 4] U.S. Patent No. 9,600,096 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure teaches systems and methods that allow a user to intuitively and dynamically specify the shape, orientation, dimensions, etc. of a multi-dimensional object represented by digital data using a single input device. [Means for solving the problem]

[0005] A system according to a first embodiment of the present disclosure can be generally summarized as including a position indicator and a processing device. The position indicator includes a housing having a plurality of reference tags on an outer surface thereof, a core body provided within the housing and having a tip end protruding from the housing through an opening in the housing, a pressure detector that detects pressure applied to the tip end of the core body during operation, and a transmitter coupled to the pressure detector that transmits one or more signals indicative of the pressure applied to the tip end of the core body during operation. The processing device includes at least one receiver that, in operation, receives one or more signals indicating pressure applied to the tip of the core body of the position indicator and one or more signals indicating one or more respective positions of one or more of the reference tags; at least one processor coupled to the at least one receiver; and at least one memory device storing instructions that, when executed by the at least one processor, cause the processing device to generate visualization data based on the one or more signals indicating pressure applied to the tip of the core body of the position indicator and the one or more signals indicating one or more respective positions of one or more of the reference tags, the visualization data describing an object extending from a predetermined position in a direction based on the one or more signals indicating pressure applied to the tip of the core body of the position indicator.

[0006] When one or more signals indicating pressure applied to the tip of the core body of the position indicator indicate that the pressure is greater than a predetermined threshold, the instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data such that the object, when displayed by the visualization device, extends from a predetermined position in a first predetermined direction.

[0007] When one or more signals indicating pressure applied to the tip of the core body of the position indicator indicate that the pressure is less than a predetermined threshold, the instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data such that the object, when displayed by the visualization device, extends from the predetermined position in a second predetermined direction that is opposite to the first predetermined direction.

[0008] The processing device may include a sensor having an input surface, which, in operation, may detect the position indicator and output a signal indicative of the position of the position indicator on the input surface detected by the sensor, and instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data based on one or more signals indicative of pressure applied to the tip of the core body of the position indicator, one or more signals indicative of one or more respective positions of one or more of the reference tags, and a signal indicative of the position of the position indicator on the input surface detected by the sensor.

[0009] The position indicator may include a switch that is in one of a plurality of positions when operated, and one or more signals transmitted by the transmitter may indicate the pressure applied to the tip of the core body and the position of the switch, and instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data based on the one or more signals indicating the pressure applied to the tip of the core body of the position indicator and the position of the switch, and one or more signals indicating one or more respective positions of one or more of the reference tags.

[0010] The position indicator may include an accelerometer that outputs a signal indicative of the acceleration of the processing device during operation.

[0011] The one or more signals transmitted by the transmitter may indicate the pressure applied to the tip of the core body and the acceleration of the position indicator, and the instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data based on the one or more signals indicating the pressure applied to the tip of the core body of the position indicator and the acceleration of the processing device, and one or more signals indicating one or more respective positions of one or more of the reference tags.

[0012] A system according to a second embodiment of the present disclosure can be generally summarized as including a position indicator and a processing device. The position indicator includes a housing having a core body disposed therein, the core body having a tip end protruding from the housing through an opening in the housing, a pressure detector that detects pressure applied to the tip end of the core body in operation, and a transmitter coupled to the pressure detector that transmits one or more signals indicative of the pressure applied to the tip end of the core body detected by the pressure detector in operation. The processing device includes a sensor having an input surface that, in operation, detects the position indicator and outputs a signal indicating the position of the tip of the core body relative to the sensor's input surface; at least one receiver that, in operation, receives one or more signals indicating pressure applied to the tip of the core body of the position indicator; at least one processor coupled to the sensor and the at least one receiver; and at least one memory device storing instructions that, when executed by the at least one processor, cause the processing device to generate visualization data based on the signal indicating the position of the tip of the core body of the position indicator on the input surface and the one or more signals indicating pressure applied to the tip of the core body of the position indicator, the visualization data describing an object extending from a predetermined position in a direction based on the one or more signals indicating the pressure applied to the tip of the core body of the position indicator, and the visualization data is provided for display by the visualization device.

[0013] When one or more signals indicating pressure applied to the tip of the core body of the position indicator indicate that the pressure is greater than a predetermined threshold, the instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data such that the object, when displayed by the visualization device, extends from a predetermined position in a first predetermined direction.

[0014] When one or more signals indicating pressure applied to the tip of the core body of the position indicator indicate that the pressure is less than a predetermined threshold, the instructions stored by the at least one memory device, when executed by the at least one processor, may cause the process to generate visualization data such that the object, when displayed by the visualization device, extends from the predetermined position in a second predetermined direction that is opposite to the first predetermined direction.

[0015] The processing device may include a switch that is in one of a plurality of positions during operation, and one or more signals transmitted by the transmitter may indicate pressure applied to the tip of the core body and the position of the switch, and instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data based on the signal indicating the position of the tip of the core body relative to the input surface of the sensor and the one or more signals indicating the pressure applied to the tip of the core body of the position indicator and the position of the switch.

[0016] The processing device may include an accelerometer that outputs a signal indicative of an acceleration of the processing device during operation, and the one or more signals transmitted by the transmitter may indicate a pressure applied to the tip of the core body and an acceleration of the processing device, and instructions stored by the at least one memory device, when executed by the at least one processor, may cause the processing device to generate visualization data based on the signal indicative of the position of the tip of the core body relative to the input surface of the sensor, the one or more signals indicative of a pressure applied to the tip of the core body of the position indicator, and the acceleration of the processing device.

[0017] A method according to a third embodiment of the present disclosure may be summarized as including receiving one or more signals indicative of one or more spatial positions of a position indicator in three-dimensional space relative to a surface of a sensor; receiving a signal indicative of a pressure applied to the tip of the core body of the position indicator; generating visualization data based on the one or more signals indicative of the one or more positions of the position indicator and the signal indicative of the pressure applied to the tip of the core body of the position indicator, wherein the visualization data describes an object that, when displayed, extends in a direction away from the plane of the surface of the sensor based on the signal indicative of the pressure applied to the tip of the core body of the position indicator; and presenting the visualization data for display.

[0018] The object may extend from the predetermined position in a first predetermined direction when the signal indicative of the pressure applied to the tip of the core body of the position indicator indicates that the pressure is greater than a predetermined threshold.

[0019] When a signal indicating pressure applied to the tip of the core body of the position indicator indicates that the pressure is less than a predetermined threshold, the object may extend from the predetermined position in a second predetermined direction opposite to the first predetermined direction.

[0020] The method may further include receiving a signal indicating the position of a switch of the position indicator, and generating the visualization data includes generating the visualization data based on (i) a signal indicating the pressure applied to the tip of the core body of the position indicator, (ii) one or more signals indicating one or more spatial positions of the position indicator, and (iii) a signal indicating the position of the switch of the position indicator.

[0021] The method may further include receiving a signal indicating an acceleration of the position indicator, and generating the visualization data includes generating the visualization data based on (i) a signal indicating a pressure applied to the tip of the core body of the position indicator, (ii) one or more signals indicating one or more spatial positions of the position indicator, and (iii) a signal indicating the acceleration of the position indicator.

[0022] The one or more signals indicating one or more spatial positions of the position indicator may include one or more signals indicating one or more respective positions of one or more of a plurality of reference tags provided on the position indicator.

[0023] The one or more signals indicative of the one or more spatial positions of the position indicator may include a signal indicative of the position of the tip of the position indicator relative to the surface of the sensor.

[0024] The method may further include displaying a representation of the object based on the visualization data. The representation of the object may be displayed, at least in part, by a head-mounted display. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 illustrates a block diagram of a visualization system in accordance with one or more embodiments of the present disclosure. [Figure 2] FIG. 2 shows a block diagram of a position indicator used as an input device in accordance with one or more embodiments of the present disclosure. [Figure 3] FIG. 3 illustrates a block diagram of a processing device that receives input via the pointing device shown in FIG. 2 in accordance with one or more embodiments of the present disclosure. [Figure 4] FIG. 4 illustrates a flowchart of a method that may be performed by the visualization system shown in FIG. 1 in accordance with one or more embodiments of the present disclosure. [Figure 5] FIG. 5 illustrates a flowchart of a method that may be performed by the visualization system shown in FIG. 1 in accordance with one or more embodiments of the present disclosure. [Figure 6A] FIG. 6A shows a perspective view of an object that may be displayed by the visualization system shown in FIG. [Figure 6B] FIG. 6B shows a side view of the object shown in FIG. 6A in accordance with one or more embodiments of the present disclosure. [Figure 7A]FIG. 7A shows a perspective view of an object that may be displayed by the visualization system shown in FIG. [Figure 7B] FIG. 7B shows a side view of the object shown in FIG. 7A in accordance with one or more embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0026] 1 shows a block diagram of a visualization system 100 in accordance with one or more embodiments of the present disclosure. The visualization system 100 includes a position indicator 102, a processing device 104, multiple tracking devices 106a and 106b, a visualization device 108, and a sensor 109.

[0027] In the illustrated embodiment, the position indicator 102 includes a hollow, generally cylindrical housing 110 with an opening 112 formed at one end, although the housing of the position indicator 102 may have other different configurations. The tip of the core 114 protrudes from the housing 110 through the opening 112. In one or more embodiments, the core 114 is a rod-shaped member that transmits pressure corresponding to pressure applied to a portion of the position indicator (e.g., the tip of the core 114) to a pressure detector 118, which is described below with reference to FIG. 2 . In one or more embodiments, the core 114 is formed of a conductive material. In one or more embodiments, the core 114 is non-conductive and formed of a resin.

[0028] Alternatively, or in combination, in one or more embodiments, opening 112 is formed in a side of housing 110, and core 114 extends through opening 112 to allow a user's finger to apply pressure to the core to provide input to processing device 104. As described below with reference to FIG. 2 , position indicator 102 transmits a signal to processing device 104 indicative of the amount of pressure applied to the tip of core 114. Position indicator 102 can be used as an input device for processing device 104.

[0029] The processing device 104 includes an input surface 116 formed from a transparent material, such as glass. In one or more embodiments, the processing device 104 is a tablet computer. As described below with reference to FIG. 3 , a sensor 140 that tracks the current position of the position indicator 102 and a display device 138 may be provided below the input surface 116. The processing device 104 generates visualization data based on a user's manipulation of the position indicator 102 and transmits the visualization data to the visualization device 108, which displays an image based on the visualization data. Additionally or alternatively, the display device 138 of the processing device 104 may display an image based on the visualization data.

[0030] In one or more embodiments, visualization device 108 and display device 138 each process portions of the visualization data generated by processing device 104 and simultaneously display images. In one or more embodiments, visualization device 108 and display device 138 operate at different screen refresh rates. Thus, it may be desirable to offload the processing of the device operating at the higher screen refresh rate to a device operating at a lower screen refresh rate. For example, visualization device 108 may operate at a 90 Hz screen refresh rate, and display device 138 may operate at a 60 Hz screen refresh rate; in such cases, it may be desirable to offload some or all of visualization device 108's processing of the visualization data to display device 138. Thus, processing device 104 may divide the visualization data such that the processing load of visualization device 108 is offloaded to display device 138.

[0031] In one or more embodiments, processing device 104 receives a signal from visualization device 108 indicating the current processing load of visualization device 108, and processing device 104 dynamically adjusts the amount of visualization data sent to visualization device 108 and display device 138 based on the current processing load. In one or more embodiments, processing device 104 estimates the current processing load of visualization device 108 and dynamically adjusts the amount of visualization data sent to visualization device 108 and display device 138 based on the estimated current processing load. For example, if the indicated or estimated current processing load of visualization device 108 is equal to or greater than a predetermined threshold, processing device 104 decreases the amount of visualization data sent to visualization device 108 and increases the amount of visualization data sent to display device 138. Additionally or alternatively, processing device 104 may offload processing from display device 138 to visualization device 108 in a similar manner.

[0032] The tracking devices 106a and 106b track the position and / or orientation of the position indicator 102, and in particular, in some embodiments, the tip of the core 114 of the position indicator 102. The tracking devices 106a and 106b are collectively referred to as tracking devices 106. While the embodiment shown in FIG. 1 includes two devices 106, the visualization system 100 may include a different number of tracking devices 106 without departing from the scope of the present disclosure. For example, the visualization system 100 may include three, four, or more tracking devices 106 in accordance with the present disclosure. In one or more embodiments, the visualization system 100 does not include any tracking devices 106, and the position of the tip of the core 114 of the position indicator 102 is tracked using only the sensor 140 of the processing device 104.

[0033] In one or more embodiments, the tracking device 106 employs well-known optical motion tracking techniques to track the position and / or orientation of the tip of the core body 114 of the position indicator 102. In one or more embodiments, the position indicator 102 has reference tags in the form of optical markers attached to the exterior surface of the housing 110. The optical markers are passive devices each having a unique, visually distinguishable color or pattern formed thereon that can be optically detected. Each of the tracking devices 106 may include a camera that captures images of one or more of the optical markers and transmits corresponding image data to the processing device 104. The processing device 104 stores data indicating the spatial relationship between each of the optical markers and the tip of the core body 114 of the position indicator 102 and processes the image data according to well-known techniques to determine the current position and / or orientation of the tip of the core body 114 of the position indicator 102. In one or more embodiments, the optical markers are active devices each having a light-emitting device (e.g., a light-emitting diode) that emits light having a different wavelength. In one or more embodiments, tracking device 106 is a Constellation sensor that is part of the Oculus Rift system available from Oculus VR. In one or more embodiments, tracking device 106 is a laser-based tracking device. For example, tracking device 106 is a SteamVR 2.0 Base Station that is part of the HTC Vive system available from HTC Corporation.

[0034] Visualization device 108 processes the visualization data generated by processing device 104 and displays corresponding images. In one or more embodiments, visualization device 108 is a head-mounted display device. In one or more embodiments, visualization device 108 is an HTC Vive Pro virtual reality headset, which is part of the HTC Vive system available from HTC Corporation. In one or more embodiments, visualization device 108 is an Oculus Rift virtual reality headset, which is part of the Oculus Rift system available from Oculus VR. In one or more embodiments, visualization device 108 is a HoloLens augmented reality headset available from Microsoft Corporation.

[0035] In one or more embodiments, visualization device 108 includes sensor 109 and uses sensor 109 to track the position of physical objects within the field of view of sensor 109. For example, visualization device 108 is a head-mounted display, and sensor 109 includes a pair of cameras, each positioned near one eye of a user of visualization device 108 and having a field of view that is substantially the same as that eye. In addition, visualization device 108 includes a transmitter that transmits image data corresponding to images captured by the cameras to processing device 104, which processes the image data and determines the coordinates of objects captured by the cameras, e.g., using conventional image processing techniques. For example, in one or more embodiments, processing device 104 includes object recognition software configured in a manner similar to the object recognition engine described in U.S. Patent Application Publication No. 2010 / 0129990 (see, e.g., paragraph 87), which is incorporated herein by reference in its entirety. Instead, the visualization device 108 includes a processor and memory storing instructions that, when executed by the processor, cause the visualization device 108 to determine coordinates of objects imaged by the camera and transmit those coordinates to the processing device 104.

[0036] Having generally described the visualization system 100, the position indicator 102 will now be described in more detail with reference to FIG. 2 , which illustrates a block diagram of the position indicator 102 in accordance with one or more embodiments of the present disclosure. The position indicator 102 includes a pressure detector 118 that, in operation, detects pressure applied to the tip of the core 114, for example, when a user presses the tip of the core 114 against an input surface 116 of the processing device 104. In one or more embodiments, the pressure detector 118 is configured in a manner similar to the pressure sensing component described in U.S. Patent Application Publication No. 2010 / 0129990 (see, e.g., column 13, line 49 to column 22, line 13), which is incorporated herein by reference in its entirety.

[0037] In one or more embodiments, the position indicator 102 includes a switch 120 that is in one of a plurality of positions during operation. A user can actuate the switch 120 to change the position of the switch 120 to provide input to the processing device 104. For example, the switch 120 is in a “closed” or “on” position while the user is pressing the switch 120, and is in an “open” or “off” position while the user is not pressing the switch 120. In one or more embodiments, the switch 120 is configured in a manner similar to the side switch described in U.S. Patent Application Publication No. 2009 / 0129990 (see, e.g., column 11, lines 24-49). In one or more embodiments, the position indicator 102 includes two switches 120 that a user can operate to provide inputs similar to those provided by operating the left and right buttons of a computer mouse.

[0038] In one or more embodiments, the position indicator 102 includes an accelerometer 122 that, during operation, outputs a signal indicative of the acceleration of the position indicator 102. In one or more embodiments, the accelerometer 122 is configured as a micromachined micro-electromechanical system (MEMS).

[0039] The position indicator 102 also includes a transmitter 124 coupled to the pressure detector 118, which, in operation, transmits a signal indicative of pressure applied to the tip of the core 114 as detected by the pressure detector 118. In one or more embodiments, the transmitter 124 operates according to one or more of the Bluetooth communication standards. In one or more embodiments, the transmitter 124 operates according to one or more of the IEEE 802.11 family of communication standards. In one or more embodiments, the transmitter 124 electromagnetically guides a signal through the tip of the core 114 and the sensor 140 of the processing device 104. In one or more embodiments, the transmitter 124 is coupled to the switch 120, which, in operation, transmits a signal indicative of the position of the switch 120. In one or more embodiments, the transmitter 124 is coupled to the accelerometer 122, which, in operation, transmits a signal indicative of the acceleration of the processing device 102 as detected by the accelerometer 122.

[0040] In one or more embodiments, the position indicator 102 includes multiple reference tags 126a, 126b, and 126c. The reference tags 126a, 126b, and 126c are collectively referred to herein as reference tags 126. The reference tags 126 are tracked by the tracking device 106. In one or more embodiments, the reference tags 126 are passive optical markers affixed to the exterior surface of the housing 110 of the position indicator 102, as described above in connection with FIG. 1. Alternatively, or in addition, in one or more embodiments, the reference tags 126 actively emit light or radio waves that are detected by the tracking device 106. While the embodiment shown in FIG. 2 includes three reference tags 126, the position indicator 102 may include a different number of reference tags 126. For example, the position indicator 102 may include four, five, six, or more reference tags 126 in accordance with the present disclosure.

[0041] Having described the position indicator 102 in more detail, the processing device 104 will now be described in more detail with reference to Figure 3, which shows a block diagram of the processing device 104, in accordance with one or more embodiments of the present disclosure. The processing device 104 includes a microprocessor 128 having a memory 130 and a central processing unit (CPU) 132, a memory 134, input / output (I / O) circuitry 136, a display device 138, a sensor 140, a transmitter 142, and a receiver 144.

[0042] Memory 134 stores processor-executable instructions that, when executed by CPU 132, cause processing device 104 to perform the operations of processing device 104 described in connection with Figures 4, 6A, 6B, and 7. CPU 132 uses memory 130 as working memory while executing instructions. In one or more embodiments, memory 130 comprises one or more random access memory (RAM) modules and / or one or more non-volatile random access memory (NVRAM) modules, such as, for example, electrically erasable programmable read-only memory (EEPROM) or Flash memory modules.

[0043] In one or more embodiments, I / O circuitry 136 may include buttons, switches, dials, knobs, a microphone, or other user interface elements for inputting commands to processing device 104. I / O circuitry 136 may also include one or more speakers, one or more lighting devices, or other user interface elements for outputting information or instructions from processing device 104.

[0044] Display device 138 graphically displays information to the operator. Microprocessor 128 controls display device 138 to display information based on visualization data generated by processing device 104. In one or more embodiments, display device 138 is a liquid crystal display (LCD) device. In one or more embodiments, display device 138 displays two images simultaneously so that a user wearing appropriate eyewear can perceive a multidimensional image in a manner similar to viewing a three-dimensional (3D) image through a 3D-enabled television, for example.

[0045] The sensor 140 detects the position indicator 102 and outputs a signal indicative of the position of the position indicator 102 relative to an input surface (e.g., surface 116) of the sensor 140. In one or more embodiments, the microprocessor 128 processes the signal received from the sensor 140 to obtain (X, Y) coordinates on the input surface of the sensor 140 corresponding to the position indicated by the position indicator 102. In one or more embodiments, the microprocessor 128 processes the signal received from the sensor 140 to obtain (X, Y) coordinates on the input surface of the sensor 140 corresponding to the position indicated by the position indicator 102, in addition to the height (e.g., Z coordinate) above the input surface of the sensor 140 at which the position indicator 102 is located. In one or more embodiments, the sensor 140 is an inductive type sensor configured in a manner similar to the position detection sensor described in U.S. Patent No. 6,229,149 (see, e.g., column 7, line 35 to column 10, line 27), which is incorporated herein by reference in its entirety. In one or more embodiments, sensor 140 is a capacitive type sensor configured in a manner similar to the position detection sensor described in U.S. Patent No. 6,273,693 (see, e.g., column 6, lines 5 to 8, line 17), which is incorporated herein by reference in its entirety.

[0046] The transmitter 142 is coupled to the microprocessor 128, and during operation, the transmitter 142 transmits visualization data generated by the microprocessor 128 to the visualization device 108. For example, in one or more embodiments, the transmitter 142 operates according to one or more of the Bluetooth and / or IEEE 802.11 family of communication standards. The receiver 144 is coupled to the microprocessor 128, and during operation, the receiver 144 receives signals from the tracking device 106 and the visualization device 108. For example, in one or more embodiments, the receiver 144 operates according to one or more of the Bluetooth and / or IEEE 802.11 family of communication standards. In one or more embodiments, the receiver 144 receives signals from the position indicator 102. In one or more embodiments, the receiver 144 is included in the sensor 140 and receives one or more signals from the tip of the core 114 of the position indicator 102 via electromagnetic induction.

[0047] Having described the structure of visualization system 100, an example method 200 performed by visualization system 100 will now be described in connection with Figure 4, which illustrates a flowchart of method 200, in accordance with one or more embodiments of the present disclosure. Method 200 begins at 202, for example, upon power-on of processing device 104.

[0048] At 202, one or more signals indicative of one or more positions of the position indicator 102 are received. For example, the receiver 144 of the processing device 104 receives one or more signals from the tracking device 106. Additionally or alternatively, the microprocessor 128 receives one or more signals from the sensor 140 of the processing device 104. The method 200 then proceeds to 204.

[0049] At 204, a signal indicative of the position of the switch 120 of the position indicator 102 is received. For example, the receiver 144 of the processing device 104 receives the signal indicative of the position of the switch 120 from the transmitter 124 of the position indicator 102. The method 200 then proceeds to 206.

[0050] Optionally, at 206, a signal indicative of the acceleration of the position indicator 102 is received. For example, the receiver 144 of the processing device 104 receives the signal indicative of the acceleration of the position indicator 102 from the transmitter 124 of the position indicator 102. The method 200 then proceeds to 208.

[0051] At 208, a signal indicative of the pressure applied to the tip of the core 114 is received. For example, the receiver 144 of the processing device 104 receives the signal indicative of the pressure applied to the tip of the core 114 from the transmitter 124 of the position indicator 102. Additionally or alternatively, the sensor 140 of the processing device 104 receives the signal indicative of the pressure applied to the tip of the core 114 from the tip of the core 114 of the position indicator 102 via electromagnetic induction. The method 200 then proceeds to 210.

[0052] At 210, one or more signals indicative of one or more physical objects located in the vicinity of a user of visualization system 100 are received. In one or more embodiments, receiver 144 of processing device 104 receives signals indicative of one or more physical objects located in the vicinity of the user from sensor 109 of visualization device 108. For example, receiver 144 receives image data generated by a pair of cameras of sensor 109, and microprocessor 128 processes the image data to obtain coordinates corresponding to the exterior surfaces of the objects imaged by the cameras. Method 200 then proceeds to 212.

[0053] At 212, the signals received at 202, 204, 206, 208, and 210 are processed. In one or more embodiments, the data transmitted by these signals is time-stamped and stored in memory 130 of processing device 104, and CPU 132 processes this data chronologically based on the timestamps associated with the data. Processing corresponding to the flowchart shown in FIG. 5 may be performed at 212, as described below. Method 200 then proceeds to 214.

[0054] At 214, a determination is made whether a quit command has been received. For example, the microprocessor 128 determines whether the position indicator 102 has been used to select a predetermined icon or object displayed by the display device 138 of the processing device 104. As another example, the microprocessor 128 determines at 214 whether a voice command corresponding to a quit operation has been received. If a determination is made at 214 that a quit operation has been received, the method 200 ends. Otherwise, the method 200 returns to 202.

[0055] 5 illustrates a flowchart of a method 300 that may be performed by the visualization system 100 at 212 of the method 200 described above, in accordance with one or more embodiments of the present disclosure. The method 300 provides a "push" operation that results in a particular visual display, as described below. The method 300 begins at 302 in response to the microprocessor 128 determining that an instruction to perform a push operation has been received. For example, the microprocessor 128 determines that the position indicator 102 has been used to select a predetermined icon or object displayed by the display device 138 of the processing device 104. As another example, the method 300 begins at 302 in response to the microprocessor 128 determining that a voice command corresponding to an instruction to perform a push operation has been received.

[0056] At 302, the pressure applied to the distal end of the core body 114 is compared to a threshold pressure value. In the illustrated embodiment, a determination is made as to whether the pressure applied to the distal end of the core body 114 is greater than or equal to the threshold pressure value. For example, the memory 134 stores a predetermined threshold pressure value, and the microprocessor 128 determines whether the pressure applied to the distal end of the core body 114, as indicated by the signal received at 208 of method 200 described above, is greater than or equal to the threshold pressure value. If at 302 it is determined that the pressure applied to the distal end of the core body 114 is greater than or equal to the threshold pressure value, the method 300 proceeds to 304. Otherwise, the method 300 proceeds to 306.

[0057] In one or more embodiments, the user may indicate to the processing device 104 that the pushing operation is to be performed by lifting the tip of the core 114 relatively slowly off the input surface 116 of the processing device 104, as opposed to lifting the position indicator 102 relatively quickly off the input surface 116 to perform another input operation on a different portion of the input surface 106. Accordingly, at 302 of the method 300, an additional determination may be made as to whether the acceleration of the position indicator 102 is less than a threshold acceleration value. For example, the memory 134 may store a predetermined threshold acceleration value, and the microprocessor 128 may determine whether the acceleration of the position indicator 102, as indicated by the signal received at 206 of the method 200 described above, is greater than zero and less than or equal to the threshold acceleration value. If at 302 the acceleration of the position indicator 102 is not determined to be greater than zero and less than or equal to the threshold acceleration value (and the pressure applied to the tip of the core 114 is determined to be greater than or equal to the threshold pressure value), the method 300 proceeds to 304. If not, the method 300 proceeds to 306 .

[0058] At 304, visualization data is generated that describes an object extending away from a predetermined location in a first direction. For example, the predetermined location corresponds to a plane having a Z coordinate of zero, such as input surface 116 of processing device 104, and the first direction corresponds to increasing negative Z coordinate values ​​that are perpendicular to the plane of input surface 116. Method 300 then proceeds to 308.

[0059] At 306, visualization data is generated that describes an object extending away from a predetermined location in a second direction. For example, the predetermined location corresponds to a plane having a Z coordinate of zero, such as input surface 116 of processing device 104, and the second direction corresponds to increasing positive Z coordinate values ​​that are perpendicular to the plane of input surface 116. Method 300 then proceeds to 308.

[0060] At 308, the visualization data generated at 304 or 306 is stored. For example, the microprocessor 128 of the processing device 104 stores the visualization data in the memory 134. The method 300 then proceeds to 310.

[0061] At 310, the visualization data generated at 304 or 306 is transmitted. In one or more embodiments, the microprocessor 128 of the processing device 104 causes the transmitter 142 to transmit the visualization data to the visualization device 108. In one or more embodiments, the microprocessor 128 transmits the visualization data to the display device 138 of the processing device 104. The method 300 then proceeds to 312.

[0062] At 312, the visualization data is processed and an object is displayed based on the visualization data. In one or more embodiments, the visualization device 108 performs rendering of a two-dimensional image to obtain a three-dimensional (3D) representation of the object described by the visualization data. In one or more embodiments, the visualization device 108 performs rendering of a two-dimensional image to obtain a two-and-a-half-dimensional (2.5D) representation of the object described by the visualization data, and the 3D environment of an observer viewing the output of the visualization device 108 is projected onto the 2D plane of the observer's retina. In one or more embodiments, the microprocessor 128 causes the display device 138 of the processing device 104 to render the visualization data and display the object. Method 300 then ends.

[0063] Figure 6A shows a perspective view of an object 146 that may be displayed at 312 of method 300, in accordance with one or more embodiments of the present disclosure. Figure 6B shows a side view of the object 146 shown in Figure 6A.

[0064] In the illustrated example, the hand 148 of the user of the visualization system 100 holds the position indicator 102 and uses the tip of the core 114 to outline a square shape 150 on the input surface 116 of the processing device 104. The processing device 104 receives one or more signals indicating a corresponding position of the position indicator 102 at 202 of the method 200 described above. The user also indicates to the processing device 104 that a push operation based on the shape 150 will be performed by moving the switch 120 of the position indicator 102 to a closed or on position while keeping the position indicator 102 positioned over the outline of the shape 150. The processing device 104 receives the signal indicating the position of the switch 120 at 204 of the method 200 described above. Additionally, the user indicates a direction in which a push operation will be performed by pressing the tip of the core 114 downward against the input surface 116 of the processing device 104. The processing device 104 receives a signal indicative of the pressure applied to the distal end of the core 114 at 208 of the method 200 described above.

[0065] Furthermore, because the pressure applied to the tip of the core 114 is determined to be equal to or greater than the threshold value at 302 of the method 300, the processing device 104 generates visualization data at 304 of the method 300, showing the object 146 extending downward from the plane corresponding to the input surface 116 of the processing device 104 in a direction away from the user. The degree to which the object 146 extends downward is based on the magnitude of the pressure applied to the tip of the core 114 and / or the time the pressure is applied to the tip of the core 114. That is, the greater the pressure the user applies to the tip of the core 114, the greater the distance the object 146 extends downward. Similarly, the longer the user applies pressure to the tip of the core 114, the greater the distance the object 146 extends downward.

[0066] For example, input surface 116 corresponds to a Z coordinate of zero, and the Z coordinate increases as one moves upward from input surface 116, and the Z coordinate decreases as one moves downward from the input surface. Microprocessor 128 also generates coordinates of object 146 such that the X and Y coordinates of object 146 correspond to the X and Y coordinates of shape 150, respectively, and the Z coordinate of object 146 ranges from zero to a negative value corresponding to the magnitude of pressure applied to the tip of core 114. Therefore, when the visualization data generated by processing device 104 in 304 of method 300 described above is displayed by visualization device 108 in 312 of method 300 described above, object 146 is displayed extending downward from a surface of processing device 104 corresponding to input surface 116 in a direction away from the user, as shown in FIG. 6B .

[0067] Figure 7A shows a perspective view of an object 152 that may be displayed at 312 of the method 300 described above, in accordance with one or more embodiments of the present disclosure. Figure 7B shows a side view of the object 152 shown in Figure 7A.

[0068] In the illustrated example, the hand 148 of the user of the visualization system 100 holds the position indicator 102 and uses the tip of the core 114 to outline a square shape 154 on the input surface 116 of the processing device 104. The processing device 104 receives one or more signals indicating a corresponding position of the position indicator 102 at 202 of the method 200 described above. Furthermore, the user indicates to the processing device 104 that a push operation based on the shape 154 will be performed by moving the switch 120 of the position indicator 102 from an open or off position to a closed or on position while keeping the position indicator 102 positioned over the outline of the shape 154. The processing device 104 receives the signal indicating the position of the switch 120 at 204 of the method 200 described above.

[0069] Additionally or alternatively, the user indicates to the processing device 104 that a push operation will be performed by slowly moving the tip of the core 114 upward, away from the input surface 116 of the processing device 104. The processing device 104 receives a signal indicative of the acceleration of the position indication 102 at 206 of the above-described method 200. Furthermore, because the acceleration of the position indication 102 is determined to be greater than zero and less than or equal to the threshold acceleration value, and because the pressure applied to the tip of the core 114 is not determined to be greater than or equal to the threshold pressure value at 302 of the above-described method 300, the processing device 104 generates visualization data at 306 of the above-described method 300 showing the object 152 extending upward from the plane corresponding to the input surface 116 of the processing device 104 in a direction toward the user. The extent to which the object 152 extends upward is based on the magnitude of the distance between the tip of the core 114 and the input surface 116 of the processing device 104. That is, the greater the distance between the tip of the core 114 and the input surface 116 of the processing device 104, the greater the distance that the object 152 extends upward.

[0070] For example, again, input surface 116 corresponds to a Z coordinate of zero, and the Z coordinate increases with increasing upward distance from input surface 116, and the Z coordinate decreases with decreasing downward distance from input surface 116. Microprocessor 128 also generates coordinates of object 152 such that the X and Y coordinates of object 152 correspond to the X and Y coordinates of shape 154, respectively, and the Z coordinate of object 152 ranges from zero to a positive value corresponding to the distance between the tip of core 114 and input surface 116 of processing device 104. Thus, when the visualization data generated by processing device 104 in 306 of method 300 described above is displayed by visualization device 108 in 312 of method 300 described above, object 152 is displayed extending upward from a surface corresponding to input surface 116 of processing device 104 in a direction toward the user, as shown in FIG. 7B .

[0071] Using the present invention, a user can intuitively specify the shape, orientation, dimensions, etc. of an object in digital data and have this object rendered with a multi-dimensional appearance above or below the plane of the input surface using a single input device as described.

[0072] The various embodiments described above can be combined to provide further embodiments, and aspects of the embodiments can be modified to provide still further embodiments, where necessary to employ concepts from various patents in light of this specification.

[0073] These and other changes can be made to the embodiments in light of the above-detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims, but should be construed to include all possible embodiments, along with the full range of equivalents to which such claims are entitled. Accordingly, the claims are not limited by this disclosure.

Claims

1. A method for generating a three-dimensional object based on an operation of a position indicator, comprising: defining a two-dimensional shape based on movement of the tip of the position indicator on an input surface; determining a direction in which the two-dimensional shape extends based on a pressure applied to the input surface by the position indicator, an acceleration of the position indicator, or both; generating a three-dimensional object extruded from the two-dimensional shape in the determined direction; displaying the three-dimensional object on a display device; method.

2. 10. The method of claim 1, determining a distance that the three-dimensional object extends from the two-dimensional shape based on a magnitude of pressure applied to the input surface by the position indicator; method.

3. 10. The method of claim 1, determining a distance that the three-dimensional object extends from the two-dimensional shape based on a time that the position indicator applies pressure to the input surface; method.

4. 10. The method of claim 1, determining a direction in which the two-dimensional shape extends to be a direction upward and away from the input surface when the pressure applied to the input surface by the position indicator is less than a predetermined threshold or when the acceleration of the position indicator is within a predetermined range; method.

5. 10. The method of claim 1, starting to push the three-dimensional object in response to detecting a change in the position of the switch of the position indicator; method.

6. 10. The method of claim 1, a sensor disposed below the input surface tracks the movement of the tip of the pointer over the input surface; method.

7. 10. The method of claim 1, tracking the movement of the position indicator using one or more reference tags disposed on an outer surface of the position indicator and one or more tracking devices configured to detect the reference tags; method.

Images