Information processing device, information processing method, and program

Abstract

Provided is an information processing device that allows a user to confirm, before performing an operation, a range in which a device can be operated. The present invention comprises: an image display unit 13 that displays a stereoscopic image creating an illusion of a 3D virtual shape, which is three-dimensional, for a user by causing the right eye and the left eye of the user to respectively see different images; a tactile sensation generation unit 23 that generates a tactile sensation in a tactile sensation range which has a similar shape to that of the 3D virtual shape; and a control unit (corresponding to an image control unit 12) that executes an operation on the stereoscopic image when the user performs an operation in the tactile sensation range during generation of the tactile sensation in the tactile sensation range by the tactile sensation generation unit 23.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Conventionally, devices have been proposed that use the parallax between a user's left and right eyes to create the illusion of a three-dimensional image on a display. The angle control device described in Patent Document 1 detects the movement of the user's hand using a non-contact sensor and rotates the three-dimensional image displayed on the display based on the angle of the user's hand movement. The angle control device described in Patent Document 1 allows the user to manipulate the displayed image without directly touching the display.

[0003] JP 2019-219926 A

[0004] However, in the invention described in Patent Document 1, the user cannot determine whether or not the user's hand is within the detection range of the non-contact sensor before performing an operation. In other words, the invention described in Patent Document 1 has a problem in that the user cannot check the operable range of the device before performing an operation.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to allow a user to check the operable range of a device before performing an operation.

[0006] The information processing device according to the present disclosure includes an image display unit that displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a three-dimensional figure, by having the user's right eye and left eye respectively view different images; a tactile generation unit that generates a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure; and a control unit that executes an operation on the stereoscopic image when the user performs an operation within the tactile range while the tactile generation unit is generating a tactile sensation in the tactile range.

[0007] In addition, the information processing method and program disclosed herein include a first step in which the image display unit 13 displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a three-dimensional figure, by having the user's right eye and left eye respectively visualize different images; a second step in which the tactile sensation generation unit generates a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure; and a third step in which the control unit executes an operation on the stereoscopic image when the user performs an operation within the tactile range after the processing of the second step is completed.

[0008] The information processing device, information processing method, and program according to the present disclosure have the advantage of allowing a user to check the operable range before performing an operation.

[0009] 1 is a schematic diagram of a three-dimensional model manipulation device according to a first embodiment of the present disclosure. FIG. 2 is a block diagram showing a functional configuration of the three-dimensional model manipulation device according to the first embodiment of the present disclosure. FIG. 3 is a block diagram showing a hardware configuration of the three-dimensional model manipulation device according to the first embodiment of the present disclosure. FIG. 4 is a flowchart showing a process performed by the three-dimensional model manipulation device according to the first embodiment of the present disclosure when starting to display a stereoscopic image and generate a haptic sensation. FIG. 5 is a flowchart showing a process performed by the three-dimensional model manipulation device according to the first embodiment of the present disclosure when ending to display a stereoscopic image and stopping to generate a haptic sensation. FIG. 6 is a flowchart showing a process performed by the three-dimensional model manipulation device according to the first embodiment of the present disclosure when performing a rotation operation of a stereoscopic image and a haptic range. FIG. 7 is a flowchart showing a rotation process of a stereoscopic image performed by an image control unit of the three-dimensional model manipulation device according to the first embodiment of the present disclosure. FIG. 8 is a flowchart showing a rotation process of a haptic range performed by a haptic control unit of the three-dimensional model manipulation device according to the first embodiment of the present disclosure. FIG. 9 is a block diagram showing a functional configuration of a three-dimensional model manipulation device according to a second embodiment of the present disclosure. FIG. 10 is a diagram showing an example of a three-dimensional virtual figure displayed on an image display unit of an image display device of the three-dimensional model manipulation device according to the second embodiment of the present disclosure. FIG. 11 is a flowchart showing a process performed by the three-dimensional model manipulation device according to the second embodiment of the present disclosure when performing a rotation operation of a stereoscopic image and a haptic range. Fig. 10 is a block diagram showing the functional configuration of a three-dimensional model manipulation device according to Embodiment 3 of the present disclosure, and Fig. 11 is a flowchart showing the processing performed when the three-dimensional model manipulation device according to Embodiment 3 of the present disclosure performs an operation on a stereoscopic image and a tactile range.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the present disclosure is not limited to the following embodiments, and modifications or omissions are possible without departing from the spirit of the present disclosure. Furthermore, common elements in each drawing are designated by the same reference numerals, and redundant explanations will be omitted.

[0011] Embodiment 1. Fig. 1 is a schematic diagram of a three-dimensional model manipulation device 100 according to embodiment 1 of the present disclosure. An overview of the three-dimensional model manipulation device 100 will be described using Fig. 1. As shown in Fig. 1, the three-dimensional model manipulation device 100 has an image display device 10, a haptic sense generating device 20, a detection device 30, a determination device 40, an input device 50, and a storage device 60. Furthermore, the network 800 shown in Fig. 1 is shown for the purpose of explanation and is not included in the three-dimensional model manipulation device 100.

[0012] The image display device 10, the tactile sense generating device 20, the detection device 30, the determination device 40, the input device 50, and the storage device 60 are connected via a network 800. The connection to the network 800 can be made via, for example, a LAN, a WAN, the Internet, Bluetooth (registered trademark), a dedicated circuit, or infrared communication.

[0013] The image display device 10 displays a stereoscopic image. The stereoscopic image gives the user the illusion of a three-dimensional virtual figure by visualizing different images in each of the user's right and left eyes. The three-dimensional virtual figure is perceived by the user as protruding from the screen toward the user's eyes by a distance determined by the protrusion amount. The protrusion amount is determined by the stereoscopic image.

[0014] Furthermore, when the determination device 40 determines that the user has performed an operation on the three-dimensional model operating device 100, the image display device 10 rotates the stereoscopic image by a predetermined reference rotation angle in the direction in which the user performed the operation. The value of the reference rotation angle is stored in the storage device 60. In the first embodiment, a case will be described in which the user performs a rotation operation by a predetermined reference rotation angle.

[0015] The tactile sense generating device 20 provides tactile feedback to the user by generating ultrasonic waves in a predetermined tactile range when a stereoscopic image is displayed on the screen display unit 10. Specifically, the tactile sense generating device 20 forms a tactile focus in the tactile range by utilizing interference of ultrasonic waves. The tactile range will be described later.

[0016] By using ultrasound to provide tactile feedback, it is possible to generate tactile feedback with high precision even if the tactile area has a complex shape. Therefore, the three-dimensional model manipulation device 100 has the effect of being able to handle even tactile areas with complex shapes.

[0017] Furthermore, the tactile range has the same shape as the three-dimensional virtual figure. In other words, the tactile range has a similarity relationship with the three-dimensional virtual figure. By making the tactile range similar to the three-dimensional virtual figure, the user can check the tactile range before performing an operation. Therefore, the three-dimensional model operating device 100 has the effect of allowing the user to check the operable range before performing an operation. Furthermore, since the tactile range is a three-dimensional range, the three-dimensional operable range can be checked.

[0018] Furthermore, in the first embodiment, the position of the tactile range is the same as the position where the illusion of the three-dimensional virtual figure is created. Also, the size of the tactile range is the same as the size of the three-dimensional virtual figure. By making the position and size of the tactile range the same as the position and size of the three-dimensional virtual figure, the operability of the three-dimensional model manipulation device 100 can be further improved.

[0019] Furthermore, when the determination device 40 determines that the user has performed an operation on the three-dimensional model operating device 100, the tactile sense generating device 20 rotates the tactile range by the reference rotation angle in the direction in which the user performed the operation. By configuring the tactile range to rotate by the same angle as the stereoscopic image when an operation is performed on the three-dimensional model operating device 100, the effect is achieved that the user can confirm the operable range even after the rotation operation.

[0020] The detection device 30 detects operations performed by the user on the three-dimensional model manipulation device 100. More specifically, the detection device 30 detects the position and movement of the user's hand. The detection device 30 transmits a signal indicating the detection result to the determination device 40.

[0021] The determination device 40 determines the operation performed by the user on the three-dimensional model manipulation device 100. Specifically, the determination device 40 determines whether the user has performed an operation on the three-dimensional model manipulation device 100 and the rotation direction of the user's operation. The determination device 40 receives the detection result from the detection device 30, and determines from the detection result of the detection device 30 whether the user has performed an operation on the three-dimensional model manipulation device 100. More specifically, if the user has moved their hand within the tactile range by more than a predetermined first threshold distance, it is determined that the user has performed an operation on the three-dimensional model manipulation device 100. The rotation direction is determined from the direction in which the user moved their hand when it is determined that the user has performed an operation on the three-dimensional model manipulation device 100.

[0022] In this way, by configuring the three-dimensional model operating device 100 to determine that an operation has been performed when the user moves their hand more than the first threshold distance within the tactile range, the three-dimensional model operating device 100 has the effect of preventing false detection when the user moves their hand less than the first threshold distance, that is, when they move their hand only a short distance.

[0023] Furthermore, by configuring the determination device 40 to determine the direction of rotation, the user can rotate the stereoscopic image in the same direction as the direction in which they move their hand, which has the effect of further improving the operability of the three-dimensional model operating device 100.

[0024] In addition, the determination device 40 transmits to the image display device 10 and the tactile generating device 20 a signal indicating that the user has performed an operation on the three-dimensional model operating device 100, and a signal indicating the direction of rotation.

[0025] The input device 50 receives input of operations of the three-dimensional model operating device 100 by the user. More specifically, an operation to start displaying a stereoscopic image and an operation to end displaying the stereoscopic image are input to the input device 50. When an operation to start displaying a stereoscopic image is input, the input device 50 transmits a signal to the image display device 10 to start displaying the stereoscopic image, and transmits a signal to the haptic sense generating device 20 to start generating a haptic sense in the haptic range. Furthermore, when an operation to end displaying the stereoscopic image is input, the input device 50 transmits a signal to the image display device 10 to end displaying the stereoscopic image, and transmits a signal to the haptic sense generating device 20 to stop generating a haptic sense in the haptic range.

[0026] The storage device 60 stores various types of information necessary for operating the three-dimensional model manipulation device 100. More specifically, the storage device 60 stores stereoscopic images and corresponding tactile ranges in association with each other. The storage device 60 also stores a reference rotation angle and a first threshold distance.

[0027] 2 is a block diagram showing the functional configuration of the three-dimensional model manipulation device 100 according to the first embodiment of the present disclosure. The functional configuration of the three-dimensional model manipulation device 100 will be described with reference to FIG.

[0028] The image display device 10 includes a receiving unit 11 , an image control unit 12 , and an image display unit 13 .

[0029] The receiving unit 11 receives signals from the determination device 40, the input device 50, and the storage device 60. More specifically, the receiving unit 11 receives from the determination device 40 a signal indicating that the user has performed an operation on the three-dimensional model operating device 100, and a signal indicating the rotation direction. The receiving unit 11 also receives from the input device 50 a signal to start displaying a stereoscopic image and a signal to end displaying the stereoscopic image. The receiving unit 11 also receives from the storage device 60 a signal indicating a reference rotation angle.

[0030] The image control unit 12 controls the stereoscopic image displayed on the image display unit 13. More specifically, the image control unit 12 controls the start and end of display of the stereoscopic image based on the signal received by the receiving unit 11. Furthermore, when the receiving unit 11 receives a signal from the determination device 40 indicating that the user has performed an operation on the three-dimensional model operating device 100, the image control unit 12 controls the stereoscopic image to rotate by a reference rotation angle in the direction in which the user performed the operation. The image display unit 13 displays the stereoscopic image.

[0031] The tactile sense generating device 20 includes a receiving unit 21 , a tactile sense control unit 22 , and a tactile sense generating unit 23 .

[0032] The receiving unit 21 receives signals from the determination device 40, the input device 50, and the storage device 60. More specifically, the receiving unit 21 receives from the determination device 40 a signal indicating that the user has performed an operation on the three-dimensional model operating device 100, and a signal indicating the rotation direction. The receiving unit 21 also receives from the input device 50 a signal to start generating ultrasound in the tactile range and a signal to stop generating ultrasound in the tactile range. The receiving unit 21 also receives from the storage device 60 a signal indicating the tactile range corresponding to the stereoscopic image and a signal indicating the reference rotation angle.

[0033] The tactile control unit 22 controls the ultrasonic waves generated by the tactile generation unit 23. More specifically, the tactile control unit 22 controls the start and stop of generation of ultrasonic waves for the tactile range based on the signal received by the receiving unit 21. The tactile control unit 22 controls the ultrasonic waves so that the position coordinates of the tactile range stored in the memory unit match the position coordinates of the ultrasonic waves. Furthermore, when the receiving unit 21 receives a signal from the determination device 40 indicating that the user has performed an operation on the three-dimensional model operating device 100, the tactile control unit 22 controls the tactile range to rotate by a reference rotation angle in the direction of the user's operation.

[0034] The tactile sense generating unit 23 generates ultrasonic waves. Specifically, the tactile sense generating unit 23 has a plurality of ultrasonic vibrators and forms a tactile focus in the tactile range by utilizing interference of the ultrasonic waves.

[0035] The detection device 30 includes a transmission unit 31 and a detection unit 32 .

[0036] The transmitter 31 transmits a signal to the determination device 40. More specifically, the transmitter 31 transmits to the determination device 40 a signal indicating the position of the user's hand at the start and end of an operation when the user's hand moves within the tactile range, and a signal indicating the movement of the user's hand, which are acquired from the detector 32.

[0037] The detection unit 32 detects the position and movement of the user's hand at predetermined intervals. The detection unit 32 is, for example, a microwave sensor. Because microwaves and ultrasonic waves use different frequency bands, the detection unit 32 can detect the position and movement of the user's hand even when ultrasonic waves are generated in the tactile range.

[0038] The determination device 40 includes a transmitter / receiver 41 and a determination unit 42 .

[0039] The transceiver 41 receives signals from the detection device 30 and the storage device 60. More specifically, the transceiver 41 receives signals indicating the position and movement of the user's hand from the detection device 30. The transceiver 41 also receives signals indicating the tactile range corresponding to the stereoscopic image and the first threshold distance from the storage device 60.

[0040] The transmitting / receiving unit 41 also transmits signals to the image display device 10 and the tactile sense generating device 20. More specifically, the transmitting / receiving unit 41 transmits to the image display device 10 and the tactile sense generating device 20 a signal indicating that the user has performed an operation on the three-dimensional model operating device 100 and a signal indicating the rotation direction.

[0041] The determination unit 42 determines whether the user has performed an operation on the three-dimensional model operation device 100 within the tactile range. More specifically, the determination unit 42 determines whether the user has performed an operation on the three-dimensional model operation device 100 from a signal indicating the detection result of the detection unit 32, which is received by the transmission / reception unit 41. Specifically, the determination unit 42 determines that the user has performed an operation on the three-dimensional model operation device 100 when the user's hand has moved a first threshold distance or more within the tactile range. The determination unit 42 determines that the user has not performed an operation on the three-dimensional model operation device 100 when the user's hand has moved outside the tactile range, or when the distance the user's hand has moved is less than the first threshold distance. If the determination unit 42 determines that the user has performed an operation on the three-dimensional model operation device 100, it determines the direction of rotation from the direction in which the user's hand moved.

[0042] The input device 50 includes an input unit 51 and a transmission unit 52. A signal to start displaying a stereoscopic image and a signal to end displaying the stereoscopic image are input to the input unit 51. When the signal to start displaying the stereoscopic image is input to the transmission unit 52, the transmission unit 52 transmits a signal to start displaying the stereoscopic image to the image display device 10 and transmits a signal to start generating a haptic sensation in the haptic range to the haptic generation device 20. When the signal to end displaying the stereoscopic image is input to the transmission unit 52, the transmission unit 52 transmits a signal to end displaying the stereoscopic image to the image display device 10 and transmits a signal to stop generating a haptic sensation in the haptic range to the haptic generation device 20.

[0043] The storage device 60 includes a storage unit 61 and a transmission unit 62. The storage unit 61 stores a tactile range corresponding to a stereoscopic image, a reference rotation angle, and a first threshold distance. The transmission unit 62 transmits a signal indicating the tactile range corresponding to the stereoscopic image to the tactile generation device 20 and the determination device 40. The transmission unit 62 also transmits a signal indicating the reference rotation angle to the image display device 10 and the tactile generation device 20. The transmission unit 62 also transmits a signal indicating the first threshold distance to the determination device 40.

[0044] 3 is a block diagram showing the hardware configuration of the three-dimensional model manipulation device 100 according to the first embodiment of the present disclosure. The hardware configuration of the three-dimensional model manipulation device 100 will be described with reference to FIG.

[0045] The image display device 10 includes a communication interface 101, a processor 102, a memory 103, and a naked-eye 3D display 104. The components of the image display device 10 are connected to each other via a bus 105.

[0046] The communication interface 101 receives, wirelessly or via a wired connection, a signal indicating that the user has performed an operation on the three-dimensional model operating device 100 and a signal indicating the rotation direction from the communication interface 401 of the determination device 40. The communication interface 101 also receives, wirelessly or via a wired connection, a signal to start displaying a stereoscopic image and a signal to end displaying a stereoscopic image from the communication interface 501 of the input device 50. The communication interface 101 also receives, wirelessly or via a wired connection, a signal indicating a reference rotation angle from the communication interface 601 of the storage device 60. The receiving unit 11 is realized by the communication interface 101.

[0047] The processor 102 executes a program stored in the memory 103. Specifically, the processor 102 starts and ends the display of the stereoscopic image, and performs rotation operations. The processor 102 is, for example, a CPU (Central Processing Unit).

[0048] The memory 103 stores programs executed by the processor 102. The memory 103 is also used as a work area for the processor 102. The memory 103 may be, for example, a volatile memory such as a random access memory (RAM), a non-volatile memory such as a read only memory (ROM), or both a volatile memory and a non-volatile memory.

[0049] The image control unit 12 is realized by a processor 102 and a memory 103 .

[0050] The naked-eye 3D display 104 displays a stereoscopic image. By including the naked-eye 3D display 104, the 3D model operating device 100 has the effect of allowing the user to have the illusion of a three-dimensional virtual figure without wearing any special equipment. The naked-eye 3D display 104 displays the stereoscopic image using a parallax barrier method or a lenticular lens method. The image display unit 13 is realized by the naked-eye 3D display 104.

[0051] The tactile sense generating device 20 includes a communication interface 201, a processor 202, a memory 203, and an ultrasonic transducer array 204. The components of the tactile sense generating device 20 are connected to each other via a bus 205.

[0052] The communication interface 201 receives, wirelessly or via a wired connection, a signal indicating that the user has performed an operation on the 3D model manipulation device 100 and a signal indicating the rotation direction from the communication interface 401 of the determination device 40. The communication interface 101 also receives, wirelessly or via a wired connection, a signal to generate a haptic sensation in the tactile range and a signal to stop the generation of a haptic sensation in the tactile range from the communication interface 501 of the input device 50. The communication interface 201 also receives, wirelessly or via a wired connection, a signal indicating a reference rotation angle from the communication interface 601 of the storage device 60. The receiving unit 21 is realized by the communication interface 201.

[0053] The processor 202 executes a program stored in the memory 203. Specifically, the processor 202 starts and stops the generation of ultrasonic waves and performs rotation operations for the tactile range. The processor 202 is, for example, a CPU (Central Processing Unit).

[0054] The memory 203 stores programs executed by the processor 202. The memory 203 is also used as a work area for the processor 202. The memory 203 may be, for example, a volatile memory such as a random access memory (RAM), a non-volatile memory such as a read only memory (ROM), or both a volatile memory and a non-volatile memory.

[0055] The haptic control unit 22 is realized by a processor 202 and a memory 203 .

[0056] The ultrasonic transducer array 204 has a plurality of ultrasonic transducers arranged in a fixed pattern. The ultrasonic transducer array 204 forms a tactile focus by causing interference between ultrasonic waves generated by the plurality of ultrasonic transducers. The tactile generation unit 23 is realized by the ultrasonic transducer array 204.

[0057] The detection device 30 includes a communication interface 301 and a microwave sensor 302. The communication interface 301 and the microwave sensor 302 are connected via a bus 303.

[0058] The communication interface 301 transmits, wirelessly or via a wired connection, a signal indicating the position of the user's hand and a signal indicating the movement of the user's hand to the communication interface 401 of the determination device 40. The transmission unit 31 is realized by the communication interface 301.

[0059] The microwave sensor 302 detects the position and movement of the user's hand. The microwave sensor 302 detects the position and movement of the user's hand by comparing the transmitted wave with the reflected wave. Specifically, when the user's hand moves within the tactile range, the microwave sensor 302 acquires the start and end positions of the user's hand movement. The detection unit 32 is realized by the microwave sensor 302.

[0060] The determination device 40 includes a communication interface 401, a processor 402, and a memory 403. The components of the determination device 40 are connected via a bus 404.

[0061] The communication interface 401 receives, wirelessly or via a wired connection, a signal indicating the position of the user's hand and a signal indicating the movement of the user's hand from the communication interface 301 of the detection device 30. The communication interface 401 also receives, wirelessly or via a wired connection, a signal indicating a tactile range corresponding to the stereoscopic image and a signal indicating the first threshold distance from the communication interface 601 of the storage device 60.

[0062] The communication interface 401 also transmits, wirelessly or via a wired connection, to the communication interface 101 of the image display device 10 and the communication interface 201 of the haptic generation device 20, a signal indicating whether the user has performed an operation on the three-dimensional model operation device 100 and a signal indicating the direction in which the stereoscopic image has been rotated. The transmission / reception unit 41 is realized by the communication interface 401.

[0063] The processor 402 executes a program stored in the memory 203. The processor 402 executes processing to determine whether the user has performed an operation on the three-dimensional model manipulation device 100. Specifically, the processor 402 determines whether the user's hand has been moved a distance equal to or greater than a first threshold distance within the tactile range. Furthermore, if the processor 402 determines that the user has performed an operation on the three-dimensional model manipulation device 100, it executes processing to determine the direction of rotation. The processor 402 is, for example, a CPU (Central Processing Unit).

[0064] The memory 403 stores programs executed by the processor 402. The memory 403 is also used as a work area for the processor 402. The memory 403 is, for example, a volatile memory such as a random access memory (RAM), a non-volatile memory such as a read only memory (ROM), or both a volatile memory and a non-volatile memory.

[0065] The determination unit 42 is realized by a processor 402 and a memory 403 .

[0066] The input device 50 includes a communication interface 501 and a display 502. The communication interface 501 and the display 502 are connected via a bus 503.

[0067] The communication interface 501 transmits, wirelessly or via a wired connection, a signal input to the display 502 to the communication interface 101 of the image display device 10 and the communication interface 201 of the haptic generation device 20. The transmission unit 52 is realized by the communication interface 501.

[0068] The display 502 receives a signal input by the user. Specifically, the display 502 has a button, and the user can perform an operation to start and end the display of a stereoscopic image by pressing the button on the display 502. The input unit 51 is realized by the display 502.

[0069] The storage device 60 includes a communication interface 601 and a storage 602. The communication interface 601 and the storage 602 are connected via a bus 603.

[0070] The communication interface 601 transmits, wirelessly or via a wired connection, signals indicating information stored in the storage 602 to the image display device 10, the haptic generation device 20, and the determination device 40. The communication interface 601 transmits, wirelessly or via a wired connection, a signal indicating a reference rotation angle to the image display device 10. The communication interface 601 also transmits, wirelessly or via a wired connection, a signal indicating a tactile range corresponding to a stereoscopic image and a signal indicating the reference rotation angle to the haptic generation device 20. The communication interface 601 also transmits, wirelessly or via a wired connection, a signal indicating a tactile range corresponding to a stereoscopic image and a signal indicating a first threshold distance to the determination device 40. The transmission unit 62 is realized by the communication interface 601.

[0071] The storage 602 stores the stereoscopic image, the corresponding tactile range, the reference rotation angle, and the first threshold distance. The storage 304 is, for example, a solid state drive (SSD) or a hard disk.

[0072] Fig. 4 is a flowchart showing processing performed by the three-dimensional model operating device 100 according to the first embodiment of the present disclosure when it starts displaying a stereoscopic image and generating a haptic sensation. Fig. 5 is a flowchart showing processing performed by the three-dimensional model operating device 100 according to the first embodiment of the present disclosure when it ends displaying a stereoscopic image and stops generating a haptic sensation. The processing performed by the three-dimensional model operating device 100 when it starts displaying a stereoscopic image and generating a haptic sensation, and the processing performed when it stops displaying a stereoscopic image and generating a haptic sensation will be described using Figs. 4 and 5 .

[0073] The process of step S101 starts when the user inputs a signal to start displaying a stereoscopic image to the input device 50. In step S101, the image control unit 12 of the image display device 10 causes the image display unit 13 to display the stereoscopic image. When the image control unit 12 displays the stereoscopic image on the image display unit 13, the process of step S101 ends.

[0074] Step S102 is performed after the processing of step S101. In step S102, the receiving unit 21 of the haptic sense generating device 20 receives and acquires a signal indicating a tactile range corresponding to the stereoscopic image displayed in step S101 from the storage device 60. When the receiving unit 21 acquires the tactile range corresponding to the stereoscopic image from the storage device 60, the processing of step S102 ends.

[0075] Step S103 is performed after the processing of step S102. In step S103, the tactile generation unit 23 of the tactile generation device 20 generates a tactile sensation in the tactile range acquired in step S102. When the tactile generation unit 23 generates a tactile sensation in the tactile range, the processing of step S103 ends. Furthermore, when the processing of step S103 ends, the three-dimensional model operating device 100 ends the processing performed when starting the display of the stereoscopic image and the generation of a tactile sensation.

[0076] The process of step S111 is started when the user inputs a signal to terminate the display of the stereoscopic image to the input device 50. In step S111, the image control unit 12 of the image display device 10 terminates the display of the stereoscopic image on the image display unit 13. When the image control unit 12 terminates the display of the stereoscopic image on the image display unit 13, the process of step S111 is terminated.

[0077] Step S112 is performed after the processing of step S111. In step S112, the tactile generation unit 23 of the tactile generation device 20 stops generating the tactile sensation in the tactile range. When the tactile generation unit 23 stops generating the tactile sensation, the processing of step S112 ends. Furthermore, when the processing of step S112 ends, the three-dimensional model operating device 100 ends the processing performed when stopping the display of the stereoscopic image and the generation of the tactile sensation.

[0078] Fig. 6 is a flowchart showing processing when the three-dimensional model manipulation device 100 according to the first embodiment of the present disclosure performs a rotation operation of a stereoscopic image and a tactile area. Fig. 7 is a flowchart showing the rotation processing of a stereoscopic image and a tactile area performed by the image control unit 12 of the three-dimensional model manipulation device 100 according to the first embodiment of the present disclosure. Fig. 7 shows, in more detail, the processing content of step S123 in Fig. 6. Fig. 8 is a flowchart showing the rotation processing of a tactile area performed by the tactile control unit 22 of the three-dimensional model manipulation device 100 according to the first embodiment of the present disclosure. Fig. 8 shows, in more detail, the processing content of step S124 in Fig. 6. The processing when the three-dimensional model manipulation device 100 performs a rotation operation of a stereoscopic image and a tactile area will be described using Figs. 6 to 8.

[0079] Step S121 is initiated when the detection device 30 detects a movement of the user's hand within the tactile range while a stereoscopic image is being displayed and a tactile sensation is occurring within the tactile range. In step S121, the determination unit 42 of the determination device 40 determines whether the user has moved their hand a distance equal to or greater than a first threshold distance stored in the storage device 60. In step S121, the process ends when the determination unit 42 determines whether the condition is satisfied.

[0080] Step S122 is performed when it is determined in step S121 that the user has moved their hand a distance equal to or greater than the first threshold distance (step S121, Yes). In step S122, the determination unit 42 of the determination device 40 determines the rotation direction of the user's rotation operation. More specifically, the determination unit 42 determines the rotation direction from the start position and end position of the rotation operation. In step S122, the process ends when the determination unit 42 determines the rotation direction.

[0081] In step S121, if it is determined that the user has not moved their hand by the first threshold distance or more (step S121, No), the determination unit 42 executes the process of step S121. In other words, the three-dimensional model operating device 100 is configured not to perform the process of rotating the stereoscopic image and not to transmit a signal from the determination device 40 to another device unless the user moves their hand within the tactile range by the threshold or more.

[0082] Step S123 is performed after step S122. In step S123, the image control unit 12 of the image display device 10 rotates the stereoscopic image displayed on the image display unit 13. More specifically, the process of step S123 includes steps S131 and S132, as shown in Fig. 7. The processes of steps S131 and S132 will be described below.

[0083] Step S131 is performed after step S122. In step S131, the image control unit 12 receives and acquires a signal indicating the reference rotation angle from the storage device 60 via the receiving unit 11. When the image control unit 12 acquires the reference rotation angle, the process in step S131 ends.

[0084] Step S132 is performed after step S131. In step S132, the image control unit 12 rotates the stereoscopic image in the rotation direction determined in step S122 by the reference rotation angle acquired in step S131. In step S132, the image control unit 12 rotates the stereoscopic image, and the process ends.

[0085] It should be noted that the rotation of the stereoscopic image executed by the image control unit 12 in step S32 does not mean a physical rotation, but means an operation of rotating the stereoscopic image on software.

[0086] When the process of step S132 ends, the image control unit 12 ends the process of rotating the stereoscopic image and the tactile range, i.e., the process of step S123 ends.

[0087] Step S124 is performed after the processing of step S123. In step S124, the tactile control unit 22 of the tactile generating device 20 rotates the tactile range. More specifically, the processing of step S124 performs the processing of steps S141 and S142, as shown in Fig. 8. The processing of steps S141 and S142 will be described below.

[0088] Step S141 is performed after step S123. In step S141, the haptic control unit 22 receives and acquires the reference rotation angle from the storage device 60 via the receiving unit 21. In step S141, the haptic generation unit 23 acquires the reference rotation angle, and the process ends.

[0089] Step S142 is performed after step S141. In step S142, the tactile control unit 22 rotates the tactile range by the reference rotation angle acquired in step S141 in the rotation direction determined in step S122. In step S142, the tactile control unit 22 rotates the tactile range, and the process ends.

[0090] When the process of step S142 is completed, the haptic control unit 22 ends the rotation process of the tactile range. That is, the process of step S124 is completed. Also, when the process of step S124 is completed, the three-dimensional model operating device 100 ends the rotation operation of the stereoscopic image and the tactile range.

[0091] As described above, the information processing device according to the first embodiment (corresponding to the three-dimensional model operating device 100) includes the image display unit 13 that displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a three-dimensional figure, by having the user's right eye and left eye respectively see different images, the tactile sense generation unit 23 that generates a tactile sense in a tactile range that is similar in shape to the three-dimensional virtual figure, and a control unit (corresponding to the image control unit 12) that executes an operation on the stereoscopic image when the user performs an operation within the tactile range while the tactile sense generation unit 23 is generating a tactile sense in the tactile range. With this configuration, the information processing device according to the first embodiment has the effect of allowing the user to check the operable range of the device before performing an operation.

[0092] Furthermore, the information processing method and program according to the first embodiment of the present disclosure include a first step in which the image display unit 13 displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a stereoscopic figure, by causing the user's right eye and left eye to see different images, a second step in which the tactile sensation generation unit generates a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure, and a third step in which, after the processing of the second step is completed, a control unit (corresponding to the image control unit 12) executes an operation on the stereoscopic image when the user performs an operation within the tactile range. With this configuration, the information processing method and program according to the first embodiment have the effect of allowing the user to check the operable range of the device before performing an operation.

[0093] Furthermore, the information processing device according to the first embodiment (corresponding to the three-dimensional model operating device 100) has an additional configuration in which, when the user performs an operation within the tactile range while the tactile generation unit is generating a tactile sensation within the tactile range, the control unit (corresponding to the tactile control unit 22) executes the operation within the tactile range. With this additional configuration, the information processing device according to the first embodiment has the effect of allowing the user to check the operable range of the device even after a rotation operation.

[0094] Furthermore, the information processing method and program according to the first embodiment additionally includes a fourth step in which, if the user performs an operation within the tactile range after the second step is completed, a control unit (corresponding to the tactile control unit 22) executes an operation within the tactile range. With this additional configuration, the information processing method and program according to the first embodiment advantageously enable the user to check the operable range of the device even after a rotation operation.

[0095] Furthermore, the information processing device according to the first embodiment (corresponding to the three-dimensional model operating device 100) has an additional configuration in which the haptic generation unit 23 generates a haptic sensation when a stereoscopic image is displayed on the image display unit 13. With this additional configuration, the information processing device according to the first embodiment has the advantage that it is easier for the user to confirm haptic feedback, compared to a case in which a haptic sensation is generated after detecting that the user's hand is within the tactile range.

[0096] Furthermore, the information processing method and program according to embodiment 1 additionally include a configuration in which the second step is performed after the processing of the first step is completed. This additional configuration allows the information processing method and program according to embodiment 1 to have the advantage of making it easier for the user to check the haptic feedback compared to when a haptic sensation is generated after detecting that the user's hand is within the tactile range.

[0097] Furthermore, the information processing device according to the first embodiment (corresponding to the three-dimensional model operating device 100) has an additional configuration in which the position of the tactile range is the same as the position where the illusion of a three-dimensional virtual figure is created, and the size of the tactile range is the same as the size of the three-dimensional virtual figure. This additional configuration enables the information processing device according to the first embodiment to achieve the effect of improving the operability of the device.

[0098] Furthermore, the information processing method and program according to the first embodiment additionally include a configuration in which the position of the tactile range where the tactile sensation is generated in the second step is the same as the position where the illusion of the three-dimensional virtual figure is created, and the size of the tactile range is the same as the size of the three-dimensional virtual figure. With this additional configuration, the information processing method and program according to the first embodiment have the effect of improving the operability of the device.

[0099] Furthermore, the information processing device according to embodiment 1 (corresponding to three-dimensional model operating device 100) further includes, as an additional configuration, a determination unit (corresponding to determination device 40) that acquires the detection results of a detection unit (corresponding to detection device 30) that detects the movement of the user's hand and determines whether the user is performing a rotation operation to rotate a three-dimensional virtual figure, and if the determination unit determines that the user is performing a rotation operation, a control unit (corresponding to image control unit 12) rotates the stereoscopic image by a predetermined reference rotation angle. This additional configuration enables the information processing device according to embodiment 1 to achieve the effect of rotating the stereoscopic image based on the movement of the user's hand.

[0100] Furthermore, the information processing method and program according to embodiment 1 additionally include a third step in which, when the determination unit determines that the user is performing a rotation operation based on the result of a detection unit (corresponding to detection device 30) that detects the user's hand movement, a control unit (corresponding to image control unit 12) rotates the stereoscopic image by a predetermined reference rotation angle. With this additional configuration, the information processing method and program according to embodiment 1 achieve the effect of being able to rotate the stereoscopic image based on the user's hand movement.

[0101] Furthermore, the information processing device according to embodiment 1 (corresponding to the three-dimensional model operating device 100) has an additional configuration in which a determination unit (corresponding to the determination device 40) determines the direction of a rotation operation, and when the determination unit determines that the user is performing a rotation operation, a control unit (corresponding to the image control unit 12) rotates the stereoscopic image in the same direction as the rotation operation. This additional configuration allows the user to rotate the stereoscopic image in the same direction as the direction in which they moved their hands, and therefore the information processing device according to embodiment 1 has the effect of further improving operability.

[0102] Furthermore, the information processing device according to the first embodiment (corresponding to the three-dimensional model operating device 100) has an additional configuration in which the tactile sensation generating unit 23 generates a tactile sensation by outputting ultrasonic waves to the tactile range. This additional configuration enables the information processing device according to the first embodiment to achieve the effect of being able to handle tactile ranges with complex shapes.

[0103] Furthermore, the information processing method and program according to the first embodiment additionally include a second step in which ultrasonic waves are output to the tactile area to generate a tactile sensation. This additional configuration allows the information processing method and program according to the first embodiment to achieve the effect of being able to handle tactile areas with complex shapes.

[0104] Furthermore, the information processing device according to the first embodiment (corresponding to the three-dimensional model operating device 100) has an additional configuration in which the image display unit 13 is a naked-eye 3D display. With this additional configuration, the information processing device according to the first embodiment has the effect of enabling the user to have the illusion of a three-dimensional virtual figure without wearing any special equipment.

[0105] Furthermore, the information processing method and program according to the first embodiment additionally include a configuration in which the image display unit 13 is a naked-eye 3D display. With this additional configuration, the information processing method and program according to the first embodiment achieves the effect of enabling the user to have the illusion of a three-dimensional virtual figure without wearing any special equipment or the like.

[0106] Furthermore, in the three-dimensional model operating device according to the first embodiment, the determination of an operation is performed for each detection by the detection unit, but this is not limiting. The device may be configured to determine an operation based on multiple detections by the detection unit.

[0107] Embodiment 2. A three-dimensional model manipulation device 200 according to embodiment 2 will be described. In embodiment 2, the same components as those in embodiment 1 of the present disclosure are designated by the same reference numerals, and descriptions of the same or corresponding parts will be omitted. The three-dimensional model manipulation device 200 according to embodiment 2 differs from the three-dimensional model manipulation device 100 according to embodiment 1 in the angle by which the stereoscopic image is rotated. More specifically, in embodiment 2, the rotation angle of the stereoscopic image and the tactile range is changed according to the distance the user moves their hand during the rotation operation.

[0108] 9 is a block diagram showing the functional configuration of a three-dimensional model manipulation device 200 according to embodiment 2 of the present disclosure. As shown in Fig. 9, the three-dimensional model manipulation device 200 differs from the three-dimensional model manipulation device 100 in the determination device 90 and the storage device 110. The image display device 10, tactile sense generation device 20, detection device 30, and input device 50 are the same as those in embodiment 1, and therefore description thereof will be omitted.

[0109] Unlike the first embodiment, the determination device 90 includes a rotation angle calculation unit 91. The rotation angle calculation unit 91 calculates the rotation angle from the distance the user's hand moves when performing a rotation operation. The calculation details of the rotation angle calculation unit 91 will be described later.

[0110] The storage device 110 differs from the first embodiment in the information stored in the storage unit 112. More specifically, the storage unit 111 stores the radius of the sphere 122 indicating the tactile range in addition to the information stored in the storage unit 62. The storage unit 111 also stores the positions of the first and second portions corresponding to the stereoscopic image. The configurations of the tactile range, the first portion, and the second portion will be described later.

[0111] 10 is a diagram showing an example of a three-dimensional virtual figure displayed on the image display unit 13 of the image display device 10 of the three-dimensional model operating device 200 according to embodiment 2 of the present disclosure. The three-dimensional virtual figure and the tactile range displayed by the image display unit 13 will be described with reference to FIG.

[0112] 10 , the three-dimensional virtual figure has a three-dimensional model 121 and a sphere 122. The three-dimensional model 121 is a model that the user requests to be displayed on the screen display unit 13. The sphere 122 contains the three-dimensional model 121. In other words, the tactile range of the three-dimensional model operating device 200 in the second embodiment is also the interior of the sphere 122.

[0113] The three-dimensional model operating device 200 has the advantage of being able to improve operability by being configured to display, on the screen display unit 13, not only the three-dimensional model 121 but also the sphere 122 that contains the three-dimensional model 121. In addition, because the tactile range has the simple shape of a sphere, it also has the advantage of being easy for the user to operate.

[0114] In this way, by displaying the sphere 122 containing the three-dimensional model 121 as a three-dimensional virtual figure and defining the interior of the sphere as the tactile range, the three-dimensional model operating device 200 has the effect of improving operability when the user performs a rotation operation. Furthermore, the portion of the tactile range that includes the three-dimensional model 121 is defined as a first portion, and the portion of the tactile range that does not include the three-dimensional model 121 is defined as a second portion.

[0115] In the three-dimensional model manipulation device 200, the amplitude of the ultrasonic waves generated in the first portion is made different from the amplitude of the ultrasonic waves generated in the second portion. By changing the amplitude, it is possible to change the haptic feedback provided to the user in the first portion and the second portion, thereby improving the operability of the device. The haptic control unit 22 of the haptic generating device 20 generates ultrasonic waves of different amplitudes in the first portion and the second portion by acquiring signals indicating the first portion and the second portion stored in correspondence with the stereoscopic image from the storage unit 110.

[0116] Next, we will explain the calculations performed by the rotation angle calculation unit 91. The rotation angle calculation unit 91 calculates the rotation angle of the rotation operation from the radius of the sphere 122 stored in the storage unit 110 and the distance the user has moved their hand obtained from the determination unit 42. If the radius of the sphere 122 is r, the length of the arc when the user performs a rotation operation along the sphere 122 is a, and the rotation angle is θ, the rotation angle θ can be expressed by the mathematical formula 1.

[0117]

[0118] The rotation angle calculation unit 91 calculates the rotation angle using the relationship in Equation 1. By using this relationship, when the distance the user has moved their hand is equal to or greater than a second threshold distance that is greater than the first threshold distance, the three-dimensional model manipulation device 200 sets a larger operation rotation angle than when the distance the user has moved their hand in a rotation operation is less than the second threshold distance. With this configuration, the three-dimensional model manipulation device 200 can change the rotation angle depending on the amount of movement of the user's hand, thereby achieving the effect of improving operability.

[0119] 11 is a flowchart showing the processing performed when the three-dimensional model operating device 200 according to the second embodiment of the present disclosure performs a rotation operation on a stereoscopic image and a tactile range. The processing performed when the three-dimensional model operating device 200 performs a rotation operation on a stereoscopic image and a tactile range will be described with reference to FIG.

[0120] Steps S201 and S202 are similar to steps S121 and S122 in the first embodiment, and therefore description thereof will be omitted. Step S203 is performed after the processing of step S202. In step S203, the transmitter / receiver 41 of the determination device 90 receives a signal indicating the radius of the sphere 122 from the storage device 110. When the transmitter / receiver 41 receives the signal indicating the radius of the sphere 122, the processing of step S203 ends.

[0121] Step S204 is performed after step S203. In step S204, the rotation angle calculation unit 91 of the determination device 90 calculates the calculated rotation angle using the relationship in Equation 1 from the distance the user moved their hand, calculated in step S201, and the signal indicating the radius of the sphere, acquired in step S203. In step S204, the rotation angle calculation unit 91 calculates the calculated rotation angle, and the process ends.

[0122] Step S205 is performed after step S204. In step S205, the image control unit 12 of the image display device 10 rotates the stereoscopic image by the calculated rotation angle calculated in step S204 in the rotation direction determined in step S202. In step S205, the image control unit 12 rotates the stereoscopic image, and the process ends.

[0123] Step S206 is performed after step S205. In step S206, the haptic control unit 22 of the haptic generating device 20 rotates the haptic range by the calculated rotation angle calculated in step S204 in the rotation direction determined in step S202. When the haptic control unit 22 rotates the haptic range in step S206, the process ends.

[0124] As described above, the information processing device according to embodiment 2 (corresponding to the three-dimensional model operating device 200), like embodiment 1, has an image display unit 13 that displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a three-dimensional figure, by having the user's right eye and left eye respectively view different images, a tactile sense generation unit 23 that generates a tactile sense in a tactile range that is similar in shape to the three-dimensional virtual figure, and a control unit (corresponding to the image control unit 12) that executes an operation on the stereoscopic image when the user performs an operation within the tactile sense range while the tactile sense generation unit 23 is generating a tactile sense in the tactile sense range. With this configuration, the information processing device according to embodiment 2 achieves the same effects as those described in embodiment 1.

[0125] Furthermore, the information processing method and program according to embodiment 2 of the present disclosure, like embodiment 1, includes a first step of displaying a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a stereoscopic figure, by having the user's right eye and left eye respectively view different images, a second step of causing a tactile sensation generation unit to generate a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure, and a third step of causing a control unit (corresponding to image control unit 12) to perform an operation on the stereoscopic image when the user performs an operation within the tactile range after the processing of the second step is completed. With this configuration, the information processing method and program according to embodiment 2 achieve the same effects as those described in embodiment 1.

[0126] Furthermore, the information processing device (corresponding to the three-dimensional model operating device 200) according to the second embodiment of the present disclosure has, as an additional configuration, a control unit (corresponding to the image control unit 12 and the tactile control unit 22) that, when the determination unit (corresponding to the determination device 90) determines that the user is performing a rotation operation, increases the operation rotation angle when the distance the user has moved their hand in the rotation operation is equal to or greater than a predetermined threshold distance (corresponding to a second threshold distance) compared to when the distance the user has moved their hand in the rotation operation is less than the threshold distance. With this additional configuration, the information processing device according to the second embodiment has the effect of improving the operability of the device.

[0127] Furthermore, the information processing device according to the second embodiment of the present disclosure (corresponding to the three-dimensional model manipulation device 200) has an additional configuration in which the tactile sense generating unit 23 generates ultrasonic waves of a first amplitude in a predetermined first portion of the tactile range, and generates ultrasonic waves of an amplitude different from the first amplitude in a second portion of the tactile range that is different from the first portion. With this additional configuration, the information processing device according to the second embodiment has the effect of improving the operability of the device.

[0128] Furthermore, the information processing method and program according to the second embodiment of the present disclosure additionally include a second step in which the tactile generation unit generates ultrasonic waves of a first amplitude in a predetermined first portion of the tactile range, and generates ultrasonic waves of a second amplitude different from the first amplitude in a second portion of the tactile range that is different from the first portion. With this additional configuration, the information processing method and program according to the second embodiment have the effect of improving the operability of the device.

[0129] In the three-dimensional model manipulation device according to the second embodiment, the image display device is configured to display a three-dimensional model and a sphere containing the three-dimensional model, but this is not limited to this. It may be configured to display a shape other than a sphere containing the three-dimensional model, or it may be configured to display only the three-dimensional model manipulation device. It may also be configured to display the center position of the three-dimensional model. By displaying the center position of the three-dimensional model manipulation device, it is possible to achieve the effect of improving the operability of the user when performing rotation operations, even if the shape of the three-dimensional model manipulation device becomes complex.

[0130] Furthermore, the first and second portions where the amplitudes of ultrasonic waves generated are different are not limited to the example shown in embodiment 2. For example, a configuration in which different amplitudes of ultrasonic waves are generated in portions of the same model where different materials are assumed to be used may also be used. Furthermore, there may be three or more portions where the amplitudes are different.

[0131] Furthermore, in the three-dimensional model manipulation device according to the second embodiment, the rotation angle calculation unit is configured to calculate the rotation angle from the radius of the sphere and the distance the user has moved his / her hand, but this is not limiting. For example, the rotation angle may be calculated taking into account the position of the user's hand and the distance to the center of the three-dimensional model manipulation device.

[0132] Furthermore, the three-dimensional model operating device according to the second embodiment is configured to rotate the stereoscopic image and the tactile range by the rotation angle calculated by the rotation angle calculation unit, but is not limited to this. Any configuration may be used as long as, when the user moves their hand by a predetermined second threshold distance or more, the rotation angle of the stereoscopic image and the tactile range is made larger than when the user moves their hand by a distance less than the second threshold distance.

[0133] Embodiment 3. A three-dimensional model manipulation device 300 according to embodiment 3 will be described. In embodiment 3, the same components as those in embodiment 1 of the present disclosure will be designated by the same reference numerals, and descriptions of the same or corresponding parts will be omitted. The three-dimensional model manipulation device 300 according to embodiment 3 differs from the three-dimensional model manipulation device 100 according to embodiment 1 and the three-dimensional model manipulation device 200 according to embodiment 2 in the content of operations that a user performs on the three-dimensional model manipulation device 300. More specifically, with the three-dimensional model manipulation device 300 according to embodiment 3, a user can perform operations to enlarge the stereoscopic image and the tactile range, and operations to reduce the stereoscopic image and the tactile range, in addition to operations to rotate the stereoscopic image and the tactile range.

[0134] In the three-dimensional model operating device 300, when the user performs an enlargement or reduction operation, the stereoscopic image is enlarged or reduced using a predetermined enlargement or reduction ratio.

[0135] Fig. 12 is a block diagram showing the functional configuration of a three-dimensional model manipulation device 300 according to embodiment 3 of the present disclosure. As shown in Fig. 12, the three-dimensional model manipulation device 300 differs from the three-dimensional model manipulation device 100 and the three-dimensional model manipulation device 200 in the detection device 130, the determination device 140, and the storage device 150. The image display device 10, the tactile sense generation device 20, and the input device 50 are the same as those in embodiment 1, and therefore description thereof will be omitted.

[0136] The detection unit 132 of the detection device 130 detects the position and movement of the user's hand, as well as the size of the hand when viewed from the front of the three-dimensional model manipulation device 300. In order to detect the size of the hand when viewed from the front of the three-dimensional model manipulation device 300, it is desirable that the detection device 130 be installed in the same direction as the screen display unit 13 of the three-dimensional model manipulation device.

[0137] The determination device 140 determines whether the user has performed a rotation operation, an enlargement operation, or a reduction operation of the stereoscopic image. More specifically, when the determination unit 142 of the determination device 140 determines that the user has not performed a rotation operation, the determination unit 142 determines whether the user has performed an enlargement operation or a reduction operation of the stereoscopic image.

[0138] With this configuration, the three-dimensional model operating device 300 allows the user to perform operations other than rotation operations, thereby improving the operability of the device.

[0139] The determination unit 142 determines whether the user has performed an enlargement operation or a reduction operation of the stereoscopic image based on the size of the user's hand detected by the detection unit 132. The determination unit 142 determines that an enlargement operation has been performed when the user opens their hand. The determination unit 142 also determines that a reduction operation has been performed when the user closes their hand. Specifically, if the difference between the size of the hand before and after the operation is equal to or greater than the threshold area stored in the storage unit 150 and the size of the hand after the operation is larger than the size of the hand before the operation, the determination unit 142 determines that the user has performed an enlargement operation. If the difference between the size of the hand before and after the operation is equal to or greater than the threshold area stored in the storage unit 150 and the size of the hand after the operation is smaller than the size of the hand before the operation, the determination unit 142 determines that the user has performed a reduction operation. If the difference between the size of the hand before and after the operation is less than the threshold area stored in the storage unit 152, the determination unit 142 determines that the user has not performed an operation.

[0140] The storage device 150 differs from the first and second embodiments in the information stored in the storage unit 152. More specifically, the storage unit 111 stores a threshold area and an enlargement / reduction ratio in addition to the information stored in the storage unit 62.

[0141] 13 is a flowchart showing the processing performed when the three-dimensional model operating device 300 according to the third embodiment of the present disclosure performs an operation on a stereoscopic image and a tactile area. The processing performed when the three-dimensional model operating device 300 performs an operation on a stereoscopic image and a tactile area will be described with reference to FIG.

[0142] The processing of steps S301 to S304 is the same as the processing of steps S121 to S124, and therefore description thereof will be omitted. Step S305 is performed when it is determined in step S301 that the user has not moved their hand by more than the first threshold distance (step S301, No). In step S305, the determination unit 142 determines whether the size of the user's hand detected by the detection unit 132 has changed by more than the threshold area before and after the operation. In step S305, the processing is terminated when the determination unit 142 determines whether the condition is satisfied.

[0143] Step S306 is performed when it is determined in step S305 that the size of the user's hand has changed by more than the threshold area before and after the operation (step S305, Yes). In step S306, the determination unit 142 determines whether the area of ​​the hand at the end of the operation is larger than the area of ​​the hand at the start of the operation. In step S306, the determination unit 142 determines whether the condition is satisfied, and the process ends.

[0144] Furthermore, in addition to the processing of step S306, in order to enable the entire stereoscopic image to be displayed on the image display unit 13, the determination unit 142 may adjust the enlargement / reduction ratio so that the length, width, and height of the stereoscopic image after the enlargement operation are equal to or less than the length, width, and height of the image display unit 13, respectively. The position of the stereoscopic image may also be adjusted.

[0145] In step S305, if it is determined that the size of the user's hand has not changed by more than the threshold area before and after the operation (step S305, No), the determination unit 142 executes the process of step S301. In other words, the three-dimensional model operating device 300 is configured not to transmit a signal from the determination device 140 to other devices unless the user moves their hand within the tactile range by more than the threshold.

[0146] Step S307 is performed when it is determined in step S306 that the area of ​​the hand at the end of the operation is larger than the area of ​​the hand at the start of the operation (step S306, Yes). In step S307, the image control unit 12 enlarges the stereoscopic image using the enlargement / reduction ratio stored in the storage unit 152. In step S307, the image control unit 12 enlarges the stereoscopic image, and the process ends.

[0147] Step S308 is performed after step S307. In step S308, the tactile control unit 22 expands the tactile range using the enlargement / reduction ratio stored in the storage unit 152. In step S308, the tactile control unit 22 expands the tactile range, and the process ends.

[0148] Step S309 is performed when it is determined in step S306 that the area of ​​the hand at the end of the operation is not larger than the area of ​​the hand at the start of the operation (step S306, No). In step S309, the image control unit 12 reduces the stereoscopic image using the enlargement / reduction ratio stored in the storage unit 152. In step S309, the image control unit 12 reduces the stereoscopic image, and the process ends.

[0149] Step S310 is performed after step S309. In step S309, the tactile control unit 22 reduces the tactile range using the enlargement / reduction ratio stored in the storage unit 152. In step S310, the tactile control unit 22 reduces the tactile range, and the process ends.

[0150] When the processing of step S304, step S308, or step S310 is completed, the three-dimensional model operating device 300 ends the operation of the stereoscopic image and the tactile range.

[0151] As described above, the information processing device according to embodiment 3 (corresponding to the three-dimensional model operating device 300) has, as in embodiment 1, an image display unit 13 that displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a three-dimensional figure, by having the user's right eye and left eye respectively view different images, a tactile sense generation unit 23 that generates a tactile sense in a tactile range that is similar in shape to the three-dimensional virtual figure, and a control unit (corresponding to the image control unit 12) that executes an operation on the stereoscopic image when the user performs an operation within the tactile sense range while the tactile sense generation unit is generating a tactile sense in the tactile sense range. With this configuration, the information processing device according to embodiment 3 achieves the same effects as those described in embodiment 1.

[0152] Furthermore, the information processing method and program according to embodiment 3, like embodiment 1, includes a first step in which the image display unit 13 displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a three-dimensional figure, by making the user's right eye and left eye respectively see different images, a second step in which the tactile sensation generation unit generates a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure, and a third step in which, after the processing of the second step is completed, a control unit (corresponding to image control unit 12) executes an operation on the stereoscopic image when the user performs an operation within the tactile range. With this configuration, the information processing method and program according to embodiment 3 achieve the same effects as those described in embodiment 1.

[0153] Furthermore, the information processing device according to the third embodiment (corresponding to the three-dimensional model device 300) further includes, as an additional configuration, a determination unit (corresponding to the determination device 140) that acquires the detection results of a detection unit that detects the movement of the user's hand and determines whether the user is performing an enlargement operation to enlarge a three-dimensional virtual figure or a reduction operation to reduce the three-dimensional virtual figure, and if the determination unit determines that the operation is an enlargement operation, the control unit enlarges the stereoscopic image, and if the determination unit determines that the operation is a reduction operation, the control unit reduces the stereoscopic image. With this additional configuration, the information processing device according to the third embodiment allows the user to perform operations other than a rotation operation, thereby achieving the effect of improving the operability of the device.

[0154] The three-dimensional model operating device according to the third embodiment is configured to be able to perform zoom-in and zoom-out operations in addition to rotation operations on a stereoscopic image, but is not limited to this. It may be configured to be able to perform only zoom-in or zoom-out operations on a stereoscopic image. The types of operations are not limited to these three. For example, it may be configured to be able to perform movement operations on a stereoscopic image.

[0155] Furthermore, although the three-dimensional model operating device according to the third embodiment is configured to use microwaves to detect the size of the hand as viewed from the front of the image display device, this is not limiting. It may also be configured to detect the size of the hand using a sensor other than microwaves, or to detect the size of the hand through image recognition using a camera or the like.

[0156] Furthermore, the three-dimensional model operating device according to the third embodiment is configured to determine whether the user is enlarging or reducing the stereoscopic image based on the size of the hand when viewed from the front of the image display device, but this is not limiting. For example, it may be configured to determine that the user is enlarging or reducing the stereoscopic image when it is determined that the shape of the user's hand has taken on a predetermined shape.

[0157] Furthermore, the determination device of the three-dimensional model operating device according to the third embodiment is configured to determine whether the user has performed an enlargement or reduction operation on the stereoscopic image when it is determined that the user has not performed a rotation operation on the stereoscopic image, but this is not limiting. In other words, the determination device may be configured to determine whether the user has performed a rotation operation on the stereoscopic image when it is determined that the user has not performed an enlargement or reduction operation on the stereoscopic image.

[0158] Furthermore, the determination device of the three-dimensional model operating device according to the third embodiment is configured to determine the type of operation performed by the user, but is not limited to this. The type of operation performed by the user may be determined by a device other than the determination device. Furthermore, instead of being configured to determine the type of operation, the user may input the type of operation to be performed into an input device. In this case, the three-dimensional model operating device can recognize the type of operation to be performed in advance.

[0159] Furthermore, the storage device of the three-dimensional model manipulation device according to the third embodiment is configured to store the enlargement / reduction ratio, but is not limited to this. It may also be configured to store the stereoscopic image and tactile range after enlargement and reduction.

[0160] Furthermore, although the three-dimensional model manipulation devices according to the first to third embodiments are configured to generate haptic feedback using ultrasonic waves, this is not limiting. The device may be configured to generate haptic feedback to the user using means other than ultrasonic waves. For example, the device may be configured to generate haptic feedback using laser, air pressure, or electromagnetic waves.

[0161] Furthermore, in the three-dimensional model operating devices according to the first to third embodiments, the image display unit is a naked-eye 3D display, but is not limited to this. The screen display unit may be configured to display a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure. In other words, the screen display unit may be configured with a display and a device for displaying different images to the left and right eyes, such as 3D glasses. In this case, the left and right images are switched at high speed, and the glasses open and close the left and right lenses in synchronization, allowing the left and right eyes to see different images, creating the illusion of visual depth.

[0162] Furthermore, in the three-dimensional model operating devices according to the first to third embodiments, a tactile sensation is generated in the tactile range after a stereoscopic image is displayed on the image display unit, but this is not limiting. A tactile sensation may be generated in the tactile range when the detection device detects that the user's hand is within the tactile range.

[0163] In the three-dimensional model operating devices according to the first to third embodiments, the determination device is configured with hardware different from that of the image display device and the tactile sense generating device, but this is not limiting. The determination device may be configured with the same hardware as that of the image display device or the tactile sense generating device.

[0164] Furthermore, the three-dimensional model manipulation devices according to the first to third embodiments have a configuration including a detection device, a determination device, and a storage device, but are not limited to this. As long as the three-dimensional model manipulation device is configured so that the image control unit and the tactile control unit can receive various signals, these units may be external to the three-dimensional model manipulation device.

[0165] In the three-dimensional model operating devices according to the first to third embodiments, the image control unit and the haptic control unit are configured by different hardware, but this is not limiting. The image control unit and the haptic control unit may be configured by the same hardware as the control device.

[0166] In addition, in the three-dimensional model operating devices according to the first to third embodiments, the input device is configured by hardware different from that of the image display device, but this is not limited to this. The input device and the image display device may be configured by the same hardware.

[0167] Furthermore, in the three-dimensional model operating devices according to the first to third embodiments, the process of rotating the tactile area is performed after the process of rotating the stereoscopic image is performed, but this is not limiting. The process of rotating the stereoscopic image and the process of rotating the tactile area may be performed in parallel.

[0168] Furthermore, although the three-dimensional model operating devices according to the first to third embodiments are configured to transmit signals from the input device to the image display device and the haptic sense generating device, the present invention is not limited to this. A signal may be transmitted from the input device to the image display device, and then the signal may be transmitted from the image display device to the haptic sense generating device.

[0169] Furthermore, a configuration may be adopted in which there are multiple stereoscopic images.

[0170] The configurations described in the above embodiments are merely examples of the contents of the present disclosure, and may be combined with other known technologies. Furthermore, parts of the configurations may be omitted or modified without departing from the scope of the present disclosure.

[0171] Various aspects of the present disclosure are described below as appendices. (Appendix 1) An information processing device comprising: an image display unit that displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure, which is a stereoscopic figure, by visualizing different images in each of the user's right and left eyes; a tactile sense generation unit that generates a tactile sense in a tactile range that is similar in shape to the three-dimensional virtual figure; and a control unit that, when the user performs an operation within the tactile sense range while the tactile sense generation unit is generating the tactile sense in the tactile sense range, executes the operation on the stereoscopic image. (Appendix 2) The information processing device according to claim 1, wherein the control unit executes the operation on the tactile sense range when the user performs an operation within the tactile sense range while the tactile sense generation unit is generating the tactile sense in the tactile sense range. (Appendix 3) The information processing device according to Appendices 1 or 2, wherein the tactile sense generation unit generates the tactile sense when the stereoscopic image is displayed on the image display unit. (Supplementary Note 4) The information processing device according to any one of Supplementary Notes 1 to 3, wherein the position of the tactile range is the same as the position where the three-dimensional virtual figure is perceived as an illusion, and the size of the tactile range is the same as the size of the three-dimensional virtual figure. (Supplementary Note 5) The information processing device according to any one of Supplementary Notes 1 to 4, further comprising: a determination unit that acquires a detection result of a detection unit that detects movement of the user's hand and determines whether the user is performing a rotation operation to rotate the three-dimensional virtual figure, and when the determination unit determines that the user is performing the rotation operation, the control unit rotates the stereoscopic image by a predetermined reference rotation angle. (Supplementary Note 6) The information processing device according to Supplementary Note 5, wherein the determination unit determines a direction of the rotation operation, and when the determination unit determines that the user is performing the rotation operation, the control unit rotates the stereoscopic image in the same direction as the direction of the rotation operation. (Supplementary Note 7) In the information processing device described in Supplementary Note 5 or Supplementary Note 6, when the determination unit determines that the user is performing a rotation operation, if the distance the user has moved their hand in the rotation operation is equal to or greater than a predetermined threshold distance, the control unit increases the operation rotation angle compared to when the distance the user has moved their hand in the rotation operation is less than the threshold distance.(Supplementary Note 8) The information processing device according to any one of Supplementary Notes 1 to 7, further comprising: a determination unit that acquires a detection result from a detection unit that detects movement of the user's hand, and determines whether the user is performing an enlargement operation to enlarge the three-dimensional virtual figure or a reduction operation to reduce the three-dimensional virtual figure, wherein if the determination unit determines that the operation is the enlargement operation, the control unit enlarges the stereoscopic image, and if the determination unit determines that the operation is the reduction operation, the control unit reduces the stereoscopic image. (Supplementary Note 9) The information processing device according to any one of Supplementary Notes 1 to 8, wherein the haptic generation unit generates the haptic sensation by outputting ultrasonic waves to the haptic range. (Supplementary Note 10) The information processing device according to Supplementary Note 9, wherein the haptic generation unit generates ultrasonic waves of a first amplitude in a predetermined first portion of the haptic range, and generates ultrasonic waves of a second amplitude different from the first amplitude in a second portion of the haptic range different from the first portion. (Supplementary Note 11) The information processing device according to any one of Supplementary Notes 1 to 10, wherein the image display unit is a naked-eye 3D display. (Supplementary Note 12) An information processing method comprising: a first step in which the image display unit displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure that is a stereoscopic figure by having the user's right eye and left eye respectively visualize different images; a second step in which a tactile sense generation unit generates a tactile sense in a tactile range that is similar in shape to the three-dimensional virtual figure; and a third step in which, after completion of the processing of the second step, the control unit executes the operation on the stereoscopic image when the user performs an operation within the tactile range. (Supplementary Note 13) The information processing method according to Supplementary Note 12, further comprising: a fourth step in which, after completion of the processing of the second step, the control unit executes the operation on the tactile range when the user performs an operation within the tactile range. (Supplementary Note 14) The information processing method according to Supplementary Note 12 or 13, wherein the second step is performed after the processing of the first step is completed.(Supplementary Note 15) The information processing method according to any one of Supplements 12 to 14, wherein the position of the tactile area where the tactile sensation is generated in the second step is the same as the position where the three-dimensional virtual figure is perceived as an illusion, and the size of the tactile area is the same as the size of the three-dimensional virtual figure. (Supplementary Note 16) The third step is executed when a determination unit determines, based on a result of a detection unit that detects the movement of the user's hand, that the user is performing a rotation operation to rotate the three-dimensional virtual figure, and in the third step, the control unit rotates the stereoscopic image by a predetermined reference rotation angle. The information processing method according to any one of Supplements 12 to 15. (Supplementary Note 17) The information processing method according to any one of Supplements 12 to 16, wherein, in the second step, the tactile sensation is generated by outputting ultrasound to the tactile area. (Supplementary Note 18) The information processing method according to Supplementary Note 17, wherein in the second step, the tactile sense generation unit generates ultrasonic waves of a first amplitude in a predetermined first portion of the tactile range, and generates ultrasonic waves of a second amplitude different from the first amplitude in a second portion of the tactile range that is different from the first portion. (Supplementary Note 19) A program causing a computer to execute the following steps: a first step in which an image display unit displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure that is a stereoscopic figure by having the user's right eye and left eye respectively visualize different images; a second step in which a tactile sense generation unit generates a tactile sense in a tactile range that is similar in shape to the three-dimensional virtual figure; and a third step in which a control unit executes an operation on the stereoscopic image when the user performs the operation within the tactile range after the processing of the second step is completed. (Supplementary Note 20) The program according to Supplementary Note 19, further comprising: a fourth step of, when the user performs an operation within the tactile range after the processing of the second step is completed, the control unit executing the operation on the tactile range. (Supplementary Note 21) The program according to Supplementary Note 19 or 20, wherein the second step is performed after the processing of the first step is completed.(Supplementary Note 22) The program according to any one of Supplements 19 to 21, wherein the position of the tactile area where the tactile sensation is generated in the second step is the same as the position where the three-dimensional virtual figure is perceived as an illusion, and the size of the tactile area is the same as the size of the three-dimensional virtual figure. (Supplementary Note 23) The third step is executed when a determination unit determines, based on a result of a detection unit that detects the movement of the user's hand, that the user is performing a rotation operation to rotate the three-dimensional virtual figure, and in the third step, the control unit rotates the stereoscopic image by a predetermined reference rotation angle. The program according to any one of Supplements 19 to 22. (Supplementary Note 24) The program according to any one of Supplements 19 to 23, wherein, in the second step, the tactile sensation is generated by outputting ultrasound to the tactile area. (Appendix 25) The program described in Appendix 24, wherein in the second step, the tactile generating unit generates ultrasonic waves of a first amplitude in a predetermined first portion of the tactile range, and generates ultrasonic waves of a second amplitude different from the first amplitude in a second portion of the tactile range different from the first portion.

[0172] 10 Image display device, 11 Receiving unit, 12 Image control unit, 13 Image display unit, 20 Tactile generation device, 21 Receiving unit, 22 Tactile control unit, 23 Tactile generation unit, 30 Detection device, 31 Transmission unit, 32 Detection unit, 40 Determination device, 41 Transmitting / receiving unit, 42 Determination unit, 50 Input device, 51 Input unit, 52 Transmission unit, 60 Storage device, 61 Storage unit, 62 Transmission unit, 90 Determination device, 91 Rotation angle detection unit, 100 Three-dimensional model operation device, 101 Communication interface, 102 Processor, 103 Memory, 104 Naked-eye display, 105 Bus, 110 Storage device, 112 Storage unit, 120 Three-dimensional virtual image, 121 Three-dimensional model, 122 Sphere, 130 Detection device, 140 Determination device, 150 Storage device, 201 Communication interface, 202 Processor, 203 Memory, 204 Ultrasonic transducer array, 205 Bus, 301 Communication interface, 302 Microwave sensor, 303 Bus, 401 Communication interface, 402 Processor, 403 Memory, 404 Bus, 501 Communication interface, 502 Display, 503 Bus, 601 Communication interface, 602 Storage, 603 Bus, 800 Network.

Claims

1. An information processing device having: an image display unit that displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure by having the user's right eye and left eye see different images; a tactile generation unit that generates a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure; and a control unit that executes an operation on the stereoscopic image when the user performs an operation within the tactile range while the tactile generation unit is generating the tactile sensation in the tactile range.

2. The information processing device according to claim 1, wherein when the user performs an operation within the tactile range while the tactile generation unit is generating the tactile sensation in the tactile range, the control unit executes the operation within the tactile range.

3. The information processing device according to claim 1 or 2, wherein the tactile sensation generating unit generates the tactile sensation when the stereoscopic image is displayed on the image display unit.

4. An information processing device according to any one of claims 1 to 3, wherein the position of the tactile range is the same as the position where the three-dimensional virtual figure is perceived as an illusion, and the size of the tactile range is the same as the size of the three-dimensional virtual figure.

5. An information processing device according to any one of claims 1 to 4, further comprising a determination unit that acquires the detection result of a detection unit that detects the movement of the user's hand and determines whether the user is performing a rotation operation to rotate the three-dimensional virtual figure, and when the determination unit determines that the user is performing the rotation operation, the control unit rotates the stereoscopic image by a predetermined operation rotation angle.

6. The information processing device according to claim 5, wherein the determination unit determines the direction of the rotation operation, and when the determination unit determines that the user is performing the rotation operation, the control unit rotates the stereoscopic image in the same direction as the direction of the rotation operation.

7. An information processing device as described in claim 5 or 6, wherein when the determination unit determines that the user is performing a rotation operation, if the distance the user has moved their hand in the rotation operation is equal to or greater than a predetermined threshold distance, the control unit increases the operation rotation angle compared to when the distance the user has moved their hand in the rotation operation is less than the threshold distance.

8. An information processing device according to any one of claims 1 to 7, further comprising a determination unit that acquires detection results from a detection unit that detects the movement of the user's hands and determines whether the user is performing an enlargement operation to enlarge the three-dimensional virtual figure or a reduction operation to reduce the three-dimensional virtual figure, wherein if the determination unit determines that the operation is the enlargement operation, the control unit enlarges the stereoscopic image, and if the determination unit determines that the operation is the reduction operation, the control unit reduces the stereoscopic image.

9. An information processing device according to any one of claims 1 to 8, wherein the tactile sensation generating unit generates the tactile sensation by outputting ultrasonic waves to the tactile range.

10. An information processing device as described in claim 9, wherein the tactile generating unit generates ultrasonic waves of a first amplitude in a predetermined first portion of the tactile range, and generates ultrasonic waves of a second amplitude different from the first amplitude in a second portion of the tactile range different from the first portion.

11. The information processing device according to any one of claims 1 to 10, wherein the image display unit is a naked-eye 3D display.

12. An information processing method comprising: a first step in which an image display unit displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure by making the user's right eye and left eye see different images; a second step in which a tactile sensation generation unit generates a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure; and a third step in which a control unit executes an operation on the stereoscopic image when the user performs an operation within the tactile range after the processing of the second step is completed.

13. A program that causes a computer to execute the following steps: a first step in which an image display unit displays a stereoscopic image that gives the user the illusion of a three-dimensional virtual figure by making the user's right eye and left eye see different images; a second step in which a tactile sensation generation unit generates a tactile sensation in a tactile range that is similar in shape to the three-dimensional virtual figure; and a third step in which, when the user performs an operation within the tactile range after the processing of the second step is completed, a control unit executes the operation on the stereoscopic image.

Images