Electronic device for displaying three-dimensional screen, and operating method thereof

Abstract

An electronic device according to one embodiment of the present disclosure may comprise: a three-dimensional display capable of displaying a three-dimensional screen; at least one sensor configured to measure at least one value related to the position of a user's eyeballs (L, R); and at least one processor. The at least one processor may be configured to: obtain the at least one value from the at least one sensor; set a three-dimensional rendering space including a screen display area of the three-dimensional display on the basis of the at least one value; obtain an input related to at least one of rotation, movement, or enlargement of an object included in the three-dimensional screen; and on the basis of a relative position of a preview corresponding to the input of the object with respect to the three-dimensional rendering space, provide feedback related to at least one of rotation, movement, or reduction of the object or the preview in response to the input.

Description

Electronic device for displaying a three-dimensional screen and its operating method

[0001] The present disclosure relates to an electronic device including a three-dimensional display for displaying a three-dimensional screen according to one embodiment, and a method of operating the same.

[0002] With the advancement of electronic device technology, various types of electronic devices such as mobile terminals, personal digital assistants (PDAs), electronic notebooks, smart phones, tablet PCs (personal computers), and wearable devices are being widely used.

[0003] For example, electronic devices overcome the limitations of conventional 3D stereoscopic imaging methods that require glasses, focusing on allowing viewers to view images with a three-dimensional effect from any position, without the hassle of glasses. Three-dimensional displays that display glasses-free 3D images fuse multiple perspectives into a single image, exploiting the viewer's visual parallax to present distinct images to the left and right eyes, creating a three-dimensional effect.

[0004] For example, electronic devices can provide virtual reality (VR), which allows users to experience a realistic world created by a computer; augmented reality (AR), which adds virtual information (or objects) to the real world; and mixed reality (MR), which combines virtual reality and augmented reality. Electronic devices provide users with 3D stereoscopic images corresponding to virtual reality, augmented reality, or mixed reality.

[0005] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0006] An electronic device according to one embodiment of the present disclosure may include a three-dimensional display capable of displaying a three-dimensional screen, at least one sensor configured to measure at least one value related to a position of a user's eyes, and at least one processor. The at least one processor may be configured to obtain the at least one value from the at least one sensor. The at least one processor may be configured to set a three-dimensional rendering space including a screen display area of ​​the three-dimensional display based on the at least one value. The at least one processor may be configured to obtain an input related to movement of an object included in the three-dimensional rendering space displayed through the three-dimensional display. The at least one processor may be configured to display a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on the input. The at least one processor may be configured to provide feedback for adjusting display of the object in the three-dimensional rendering space based on a position of the object in the three-dimensional rendering space corresponding to the input after the preview is displayed.

[0007] According to one embodiment of the present disclosure, a method for operating an electronic device may include an operation of obtaining at least one value related to a position of a user's eye from at least one sensor. According to one embodiment, the method for operating an electronic device may include an operation of setting a three-dimensional rendering space including a screen display area of ​​a three-dimensional display capable of displaying a three-dimensional screen based on the at least one value. According to one embodiment, the method for operating an electronic device may include an operation of obtaining an input related to movement of an object included in the three-dimensional rendering space displayed through the three-dimensional display. According to one embodiment, the method for operating an electronic device may include an operation of displaying a preview of the object moved in the three-dimensional rendering space through the three-dimensional display based on the input. According to one embodiment, the method for operating an electronic device may include an operation of providing feedback for adjusting a display of the object in the three-dimensional rendering space based on a position of the object in the three-dimensional rendering space corresponding to the input after the preview is displayed.

[0008] A non-transitory computer-readable storage medium storing one or more programs according to one embodiment of the present disclosure may include an operation of obtaining at least one value related to the position of a user's eyes from at least one sensor based on the execution of an application. According to one embodiment, the storage medium may include an operation of setting a three-dimensional rendering space including a screen display area of ​​a three-dimensional display capable of displaying a stereoscopic screen based on the at least one value. According to one embodiment, the storage medium may include an operation of obtaining an input related to movement of an object included in the three-dimensional rendering space displayed through the three-dimensional display. According to one embodiment, the storage medium may include an operation of displaying a preview of the object moved in the three-dimensional rendering space through the three-dimensional display based on the input. According to one embodiment, the storage medium may include an operation of providing feedback for adjusting the display of the object in the three-dimensional rendering space based on the position of the object in the three-dimensional rendering space corresponding to the input after the preview is displayed.

[0009] An electronic device according to one embodiment of the present disclosure may include a three-dimensional display capable of displaying a three-dimensional screen, at least one sensor configured to measure at least one value related to the position of a user's eyes, and at least one processor. The at least one processor may be configured to obtain the at least one value from the at least one sensor. The at least one processor may be configured to set a three-dimensional rendering space including a screen display area of ​​the three-dimensional display based on the at least one value. The at least one processor may be configured to obtain an input related to movement of an object included in the three-dimensional rendering space displayed through the three-dimensional display. The at least one processor may be configured to display a preview of the object moved in the three-dimensional rendering space through the three-dimensional display based on the input. After the preview is displayed, the at least one processor may identify a position of the object in the three-dimensional rendering space corresponding to the input. The at least one processor may be configured to rotate, enlarge, or reduce the object and display it on the three-dimensional display based on identifying the location of the object.

[0010] According to one embodiment of the present disclosure, a method of operating an electronic device may include an operation of obtaining at least one value related to a position of a user's eye from at least one sensor. According to one embodiment, the method of operating an electronic device may include an operation of setting a three-dimensional rendering space including a screen display area of ​​a three-dimensional display based on the at least one value. According to one embodiment, the method of operating an electronic device may include an operation of obtaining an input related to movement of an object included in the three-dimensional rendering space displayed through the three-dimensional display. According to one embodiment, the method of operating an electronic device may include an operation of displaying a preview of the object moved in the three-dimensional rendering space through the three-dimensional display based on the input. According to one embodiment, the method of operating an electronic device may include an operation of identifying a position of the object in the three-dimensional rendering space corresponding to the input after the preview is displayed. According to one embodiment, the method of operating an electronic device may include an operation of displaying the object on the three-dimensional display by rotating, enlarging, or reducing the object based on the identification of the position of the object.

[0011] A non-transitory computer-readable storage medium storing one or more programs according to one embodiment of the present disclosure may include an operation of obtaining at least one value related to the position of a user's eyes from at least one sensor based on the execution of an application. According to one embodiment, the storage medium may include an operation of setting a three-dimensional rendering space including a screen display area of ​​a three-dimensional display based on the at least one value. According to one embodiment, the storage medium may include an operation of obtaining an input related to movement of an object included in the three-dimensional rendering space displayed through the three-dimensional display. According to one embodiment, the storage medium may include an operation of displaying a preview of the object moved in the three-dimensional rendering space through the three-dimensional display based on the input. According to one embodiment, the storage medium may include an operation of identifying a position of the object in the three-dimensional rendering space corresponding to the input after the preview is displayed. According to one embodiment, the storage medium may include an operation of displaying the object on the three-dimensional display by rotating, enlarging, or reducing the object based on the identification of the position of the object.

[0012] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment of the present disclosure.

[0013] FIG. 2 is a block diagram of an electronic device according to one embodiment of the present disclosure.

[0014] FIG. 3a is a screen provided to each of the user's left and right eyes according to a left-right division method according to one embodiment of the present disclosure.

[0015] Figure 3b is a screen recognized by the user according to the display screen of Figure 3a.

[0016] FIG. 4 illustrates objects included in a three-dimensional screen corresponding to various depths according to one embodiment of the present disclosure.

[0017] FIG. 5a illustrates a method of using a lenticular lens according to one embodiment of the present disclosure.

[0018] FIG. 5b illustrates a method of utilizing a parallax barrier according to one embodiment of the present disclosure.

[0019] FIG. 6 illustrates a three-dimensional rendering space (view frustum) according to one embodiment of the present disclosure.

[0020] FIG. 7a illustrates a free display space and a restriction zone within a three-dimensional rendering space according to one embodiment of the present disclosure.

[0021] Figure 7b illustrates a plan view of Figure 7a.

[0022] FIG. 8A illustrates a state in which an object is rotated, translated, or scaled to leave a three-dimensional rendering space, according to one embodiment of the present disclosure.

[0023] FIG. 8b illustrates a state in which an object deviates from a three-dimensional rendering space depending on the position of the user's eyes, according to one embodiment of the present disclosure.

[0024] FIG. 8c illustrates a state in which the image of an object appears to overlap as it is enlarged or moved beyond a threshold point, according to one embodiment of the present disclosure.

[0025] FIG. 9 illustrates an embodiment of adjusting a movement speed of an object of an electronic device according to a movement input according to one embodiment of the present disclosure.

[0026] FIG. 10 illustrates an example of moving an object in the reverse direction of a movement input for an object of an electronic device according to one embodiment of the present disclosure.

[0027] FIG. 11 illustrates an example of moving to a designated location according to a movement input for an object of an electronic device according to one embodiment of the present disclosure.

[0028] FIG. 12a illustrates an embodiment of displaying a boundary plane according to movement of an object according to one embodiment of the present disclosure.

[0029] FIG. 12b illustrates an embodiment of providing haptic feedback according to movement of an object, according to one embodiment of the present disclosure.

[0030] FIG. 13 illustrates an example of enlarging an object according to an enlargement input for an object of an electronic device according to an embodiment of the present disclosure.

[0031] FIG. 14 illustrates an embodiment of rotating an object according to a rotation input for an object of an electronic device according to one embodiment of the present disclosure.

[0032] FIG. 15 illustrates an embodiment of moving the rotation axis of an object according to a rotation input for an object of an electronic device according to one embodiment of the present disclosure.

[0033] FIG. 16A illustrates a user input that falls outside a specified range according to one embodiment of the present disclosure.

[0034] FIG. 16b illustrates an embodiment of applying a depth of field effect to at least a portion of an edge of an object according to one embodiment of the present disclosure.

[0035] FIG. 17a illustrates a user input that falls outside a specified range according to one embodiment of the present disclosure.

[0036] FIG. 17b illustrates an embodiment of limiting movement of an object in the depth direction of a three-dimensional display according to one embodiment of the present disclosure.

[0037] FIG. 18 is a flowchart of a method of operating an electronic device according to one embodiment of the present disclosure.

[0038] FIG. 19 is a flowchart of a method of operating an electronic device according to one embodiment of the present disclosure.

[0039] FIG. 20 is a flowchart of a method of operating an electronic device according to one embodiment of the present disclosure.

[0040] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to one embodiment of the present disclosure.

[0041] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0042] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0043] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0044] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0045] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0046] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0047] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0048] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0049] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0050] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0051] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0052] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0053] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0054] The camera module (180) can capture still images and moving images. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0055] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0056] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0057] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0058] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0059] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0060] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0061] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0062] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0063] FIG. 2 is a block diagram of an electronic device (101) according to one embodiment of the present disclosure.

[0064] Referring to FIG. 2, an electronic device (101) according to one embodiment may include a processor (210, e.g., the processor (120) of FIG. 1), a memory (220, e.g., the memory (120) of FIG. 1), a three-dimensional display (230, e.g., the display module (160) of FIG. 1), an eye tracking sensor (240, e.g., the sensor module (176) of FIG. 1), an input interface (250, e.g., the interface (166) of FIG. 1), a touch panel (260), and / or a camera sensor (270, e.g., the camera module (180) of FIG. 1). In addition, the electronic device (101) according to one embodiment may further include some components included in the electronic device (101) of FIG. 1, or may be configured by excluding some components.

[0065] According to one embodiment, the processor (210) may control at least one other component (e.g., a hardware or software component) of the electronic device (101). In one embodiment, the processor (210) may perform various data processing or operations, and as at least part of the data processing or operations, may store commands or data received from other components in the memory (220), process the commands or data stored in the memory (220), and store result data in the memory (220).

[0066] In one embodiment, the processor (210) may be equipped with a rendering engine that renders a three-dimensional screen to be displayed on a three-dimensional display (230) in real time. For example, the rendering engine may be included in a graphic processing unit (GPU) or a central processing unit (CPU).

[0067] According to one embodiment, the memory (220) can store instructions executable by the processor (210).

[0068] A three-dimensional display (230) according to one embodiment can display a visual screen that implements stereoscopic vision to a user. In one embodiment, the three-dimensional display (230) can display a three-dimensional screen including an object. For example, the object can be a 3D object in which at least a portion is displayed three-dimensionally. For example, the object can be displayed three-dimensionally on the three-dimensional display (230) by having at least a portion positioned in the front (positive) or back (negative) of the three-dimensional display in the depth direction.

[0069] In one embodiment, the 3D display (230) can display a three-dimensional screen by displaying different images to the user's left and right eyes according to a left-right division method as described below.

[0070] An eye tracking sensor (240) according to one embodiment can measure at least one value related to the position of a user's eyes. In one embodiment, the eye tracking sensor (240) can use an RGB sensor or a depth sensor.

[0071] In one embodiment, the eye tracking sensor (240) may be an image sensor that directly detects the position of the user's eyes (e.g., left and right eyes). In one embodiment, the eye tracking sensor (240) may be a position detection sensor that detects the position of the user's face, and the processor (210) may estimate the position of the user's eyes based on the detected position of the user's face.

[0072] An input interface (250) according to one embodiment can receive a user's input corresponding to a three-dimensional screen displayed on a three-dimensional display (230). In one embodiment, the input interface (230) can receive an indirect manipulation input through an input device (105) connected to an electronic device (101) or a direct manipulation input of the user obtained through a sensor (e.g., a touch panel (260) or a camera sensor (270)). In one embodiment, the input interface (250) can support a designated protocol for connection with an external input device (105).

[0073] A touch panel (260) according to one embodiment is integrally coupled to a three-dimensional display (230) and can detect a user's touch input corresponding to a three-dimensional screen displayed on the three-dimensional display (230). In one embodiment, the electronic device (101) can display a controller (e.g., a slider or coordinates) on at least a portion of the three-dimensional display (230) and detect a user's touch input corresponding to the displayed controller. For example, the touch panel (260) can detect a touch using a principle such as an electrostatic method, an electromagnetic method, a pressure method, or an infrared method, but is not limited thereto.

[0074] In one embodiment, the touch panel (260) may detect a proximity touch that is located near or approaches the touch panel without additional mechanical contact, in addition to a physical contact touch of the user. For example, the touch panel (260) may detect a proximity touch using principles such as, but not limited to, a transmission-type photoelectric sensor, a direct reflection-type photoelectric sensor, a mirror reflection-type photoelectric sensor, a high-frequency oscillation-type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

[0075] In one embodiment, when the touch panel (260) uses an electrostatic method, it can detect not only the user's contact touch but also the proximity touch of the pointer by a change in the electric field according to the proximity of the pointer.

[0076] According to one embodiment, a camera sensor (270) can detect a user's gesture occurring in front of a three-dimensional display (230). In one embodiment, a gesture by a user's hand can occur in front of the three-dimensional display (230) to correspond to a stereoscopic screen of the three-dimensional display (230), and the camera sensor (270) can detect the location and type of the user's gesture.

[0077] An input device (105) according to one embodiment may be connected to an electronic device (101) via a wire or wirelessly. For example, the input device (105) may be a stylus pen, a slider, or a controller that transmits manipulation input according to coordinates. For example, the input device (105) may obtain inputs such as movement, touch, button click, and hovering, and provide a signal corresponding to the obtained input to the electronic device (101, e.g., input interface (250)).

[0078] An electronic device (101) according to one embodiment can move, enlarge, or change a three-dimensional screen (or object) displayed on a three-dimensional display (230) based on a user input obtained from an input interface (250) connected to a touch panel (260) or a camera sensor (270), or an external input device (101).

[0079] FIG. 3a illustrates a screen provided to the user's left eye (L) and right eye (R) respectively according to a left-right division method according to one embodiment of the present disclosure. FIG. 3b illustrates a screen perceived by the user according to the display screen of FIG. 3a. FIG. 4 illustrates objects (2D images, 3D images) included in a three-dimensional screen corresponding to various depth perceptions according to one embodiment of the present disclosure.

[0080] Referring to FIGS. 3A, 3B, and 4, an electronic device (101) according to one embodiment can provide a stereoscopic screen by a left-right division method using binocular parallax through a three-dimensional display (e.g., a three-dimensional display (230) of FIG. 2). In one embodiment, the electronic device (101) can use a left-right division method that differently displays an image (L image) provided to a user's left eye (L) and an image (R image) provided to the user's right eye (R) so that the user can recognize a stereoscopic screen. The human brain can observe a stereoscopic image by synthesizing an image (L image) viewed through the left eye (L) and an image (R image) viewed through the right eye (R).

[0081] In one embodiment, the electronic device (101) can control the depth of an object (2D image, 3D image) included in a stereoscopic screen perceived by the user by adjusting the relative position and distance between an image (L image) corresponding to the left eye (L) and an image (R image) corresponding to the right eye (R).

[0082] In one embodiment, the electronic device (101) can display a three-dimensional screen so that the user perceives a two-dimensional object (2D image) as having a sense of depth corresponding to the screen display area (235) of the three-dimensional display (230) by displaying an image (L image) corresponding to the left eye (L) and an image (R image) corresponding to the right eye (R) in the same manner.

[0083] In one embodiment, the electronic device (101) can display a stereoscopic screen so that the user perceives a three-dimensional object (3D image) as having a sense of depth located behind (negative) the screen display area (235) of the three-dimensional display (230) by moving the image (L image) corresponding to the left eye (L) to the left and moving the image (R image) corresponding to the right eye (R) to the right.

[0084] In one embodiment, the electronic device (101) can display a stereoscopic screen so that the user perceives a three-dimensional object (3D image) as having a sense of depth located in front (positive) of the screen display area (235) of the three-dimensional display (230) by moving the image (L image) corresponding to the left eye (L) to the right and moving the image (R image) corresponding to the right eye (R) to the left.

[0085] FIG. 5A illustrates a method of using a lenticular lens (510) according to one embodiment of the present disclosure. FIG. 5B illustrates a method of using a parallax barrier (520) according to one embodiment of the present disclosure.

[0086] Referring to FIGS. 5a and 5b, the electronic device (101) can generate binocular parallax between the user's left eye (L) and right eye (R) to provide a three-dimensional screen according to a left-right division method.

[0087] Here, binocular parallax (stereo disparity) refers to the difference in perspective between a person's left (L) and right (R) eyes. When the brain synthesizes the images seen through the left (L) eye and the images seen through the right (R) eye, the resulting image creates a three-dimensional effect. Stereoscopic image display methods based on binocular parallax can be divided into glasses-based, requiring special glasses, and glasses-free, which do not require glasses.

[0088] Spectacle-based methods include, for example, the use of wavelength-selective colored glasses, the use of polarized glasses that utilize the shading effect of polarization differences, and the use of time-division glasses that alternately present left and right images within the eye's residual image time. Furthermore, a method is available in which filters with different transmittances are installed on the left and right eyes (L and R), and the resulting difference in transmittance creates a three-dimensional effect for left-right movement based on the temporal lag in the visual system.

[0089] There are various types of autostereoscopic methods that produce a three-dimensional effect on the display surface side rather than the observer side, such as the parallax barrier method, the lenticular lens method, or the microlens array method.

[0090] In one embodiment, the electronic device (101) may have a lenticular lens (510) positioned in front of the 3D display (230), as illustrated in FIG. 5A. For example, the lenticular lens (510) may be formed of a film or glass and attached to the front of the 3D display (230). For example, in the 3D display (230), pixels (PL) to be input to the left eye and pixels (PR) to be input to the right eye may be arranged alternately in a horizontal direction, and the lenticular lens (510) may provide optical discrimination directivity for the pixels (PL) to be input to the left eye and the pixels (PR) to be input to the right eye. In one embodiment, the 3D display (230) may separate different screens by the lenticular lens (510) positioned in front and provide them to the user's left eye (L) and right eye (R).

[0091] In one embodiment, the electronic device (101) may have a parallax barrier (520) positioned in front of the 3D display (230), as illustrated in FIG. 5B. For example, the parallax barrier (520) may be attached to the front of the 3D display (230), and the electronic device (101) may control the movement of the parallax barrier (520). For example, the 3D display (230) may have pixels (PL) to be input to the left eye and pixels (PR) to be input to the right eye arranged alternately in the horizontal direction, and may provide a screen by separating the screen into the left eye (L) and the right eye (R) through a vertical grid-shaped aperture. In one embodiment, the 3D display (230) may provide different screens to the user's left eye (L) and right eye (R) by separating the screens from each other by the parallax barrier (520) positioned in front.

[0092] Additionally, in one embodiment, the electronic device (101) may be formed in the form of glasses and may be provided with separate displays (not shown) corresponding to the user's left and right eyes, respectively, thereby inducing binocular parallax. In one embodiment, the electronic device (101) may provide the user with a three-dimensional screen by displaying different images or videos to the user's left and right eyes.

[0093] In one embodiment, the electronic device (101) can display a three-dimensional screen through a screen display area of ​​a display (not shown) that displays screens corresponding to each of the user's two eyes, and can move an object included in the three-dimensional screen in the depth direction of the display (not shown).

[0094] An electronic device (101) according to one embodiment can implement virtual reality (VR) through a screen display area of ​​a display (not shown). In one embodiment, the electronic device (101) can display a virtual object on the screen display area of ​​the display (not shown).

[0095] An electronic device (101) according to one embodiment can implement augmented reality (AR) through a screen display area of ​​a display (not shown). In one embodiment, the electronic device (101) can expose an external environment to a user through a screen display area of ​​a transparent or translucent display (not shown). In one embodiment, the electronic device (101) can display a virtual object on the screen display area of ​​a transparent or translucent display (not shown) so as to overlap with the external environment.

[0096] In one embodiment, the electronic device (101) may use an input interface (e.g., input interface (250) of FIG. 2) to detect the movement of the user's hand through a camera sensor (e.g., camera sensor (270) of FIG. 2) or obtain the user's input by using the movement of a cursor through an input device (e.g., input device (105) of FIG. 2).

[0097] In one embodiment, the electronic device (101) may, in response to a user's input for a virtual object, output a specified response based on properties of the object or properties of the user's input.

[0098] In one embodiment, if the electronic device (101) cannot output a specified response based on a property of the object or a property of the user's input in response to a user's input for a virtual object, the electronic device may move the virtual object in the depth direction of the display (not shown).

[0099] FIG. 6 illustrates a three-dimensional rendering space (view frustum) according to one embodiment of the present disclosure.

[0100] Referring to FIG. 6, a 3D rendering space (view frustum) according to one embodiment may be a space defined by a camera (not shown) positioned for 3D rendering or a user's eyes (L, R). In one embodiment, the electronic device (101) may render an object (not shown) located inside the 3D rendering space (view frustum) and display it on a 3D display (e.g., the 3D display (230) of FIG. 2). In one embodiment, the electronic device (101) may not render an object (O) that is outside the 3D rendering space (view frustum).

[0101] In one embodiment, when an electronic device (101) changes the position, angle, or size of an object (O) included in a stereoscopic screen displayed on a three-dimensional display (view frustum) and moves outside the three-dimensional rendering space (view frustum), a phenomenon in which the stereoscopic screen and / or object is cut off may occur.

[0102] In one embodiment, the electronic device (101) can set a three-dimensional rendering space (view frustum) in the shape of a pyramid cut by two clipping planes. In one embodiment, the electronic device (101) can set the three-dimensional rendering space (view frustum) based on the position of the eyes (L, R) (or the position of the camera), the field of view (FOV), the aspect ratio (w / h), and / or the two clipping planes (zNear, zFar).

[0103] In one embodiment, the three-dimensional rendering space (view frustum) may include a screen display area (235) of a three-dimensional display (230). In one embodiment, the electronic device (101) may set a three-dimensional rendering space (view frustum) extending forward and backward of the screen display area (235) of the three-dimensional display (230).

[0104] For example, the two clipping planes (zNear, zFar) of the 3D rendering space (view frustum) may include a first clipping plane (zNear) positioned in front of the screen display area (235) of the 3D display (230) and a second clipping plane (zFar) positioned behind the screen display area (235) of the 3D display (230). For example, the aspect ratio (w / h) of the 3D rendering space (view frustum) may match the aspect ratio (w / h) of the screen display area (235) of the 3D display (230). In one embodiment, the electronic device (101) may set the 3D rendering space (view frustum) by the position (x, y, z) of the user's position or eyes (L, R) with respect to the screen display area (235) of the 3D display (230).

[0105] In one embodiment, the electronic device (101) may acquire at least one value related to the position of the user's eyes (L, R) from at least one sensor (e.g., the eye tracking sensor (240) of FIG. 2) configured to measure at least one value related to the position of the user's eyes (L, R). In one embodiment, the electronic device (101) may newly acquire at least one value related to the position of the user's eyes (L, R) or newly set a three-dimensional rendering space (view frustum) based on satisfaction of a specified update condition. For example, the electronic device (101) may acquire at least one value related to the position of the user's eyes (L, R) from at least one sensor according to a specified cycle. For example, the electronic device (101) may be configured to update the position of the user or the position of the user's eyes (L, R) as the acquired at least one value related to the position of the user's eyes (L, R) changes.

[0106] In one embodiment, the electronic device (101) can identify a user's location or the location of the user's eyes (L, R) based on at least one value obtained from at least one sensor (240), and set a three-dimensional rendering space (view frustum) based on the identified user's location or the location of the user's eyes (L, R).

[0107] An electronic device (101) according to one embodiment can divide a three-dimensional rendering space (view frustum) into a positive space in front of a screen display area of ​​a three-dimensional display (230) and a negative space behind the screen display area of ​​the three-dimensional display.

[0108] An electronic device (101) according to one embodiment can identify whether an object (not shown) of a three-dimensional screen displayed on a three-dimensional display (230) is located in the forward (positive) space of a three-dimensional rendering space. In one embodiment, when an object (not shown) is located in the forward (positive) space of a three-dimensional rendering space (view frustum), the electronic device (101) can identify whether a culling phenomenon occurs in which a part of the object (not shown) is cut off as the object (not shown) leaves the three-dimensional rendering space (view frustum) or a free display space (free zone) described later.

[0109] In one embodiment, the electronic device (101) may display a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on a user's input.

[0110] In one embodiment, the electronic device (101) may provide feedback regarding movement of an object (not shown) in response to a user's input when the object is located in the forward (positive) space of a three-dimensional rendering space (view frustum).

[0111] In one embodiment, the electronic device (101) may not identify whether the object (O) leaves the 3D rendering space (view frustum) or the free display space (free zone) described below when the object (O) is located in the negative space of the 3D rendering space (view frustum). Since the culling phenomenon occurring in the negative space of the 3D rendering space (view frustum) can be accepted as a natural phenomenon by the user, the occurrence of the culling phenomenon may not be determined. Accordingly, unnecessary operations can be omitted, and the rendering speed of the object (not shown) can be improved.

[0112] FIG. 7A illustrates a free zone and a restriction zone within a three-dimensional rendering space (view frustum) according to one embodiment of the present disclosure. FIG. 7B illustrates a plan view of FIG. 7A.

[0113] Referring to FIGS. 7A and 7B , an electronic device (101) according to an embodiment may set a free zone within a three-dimensional rendering space (view frustum). An electronic device (101) according to an embodiment may set an external space of the free zone within the three-dimensional rendering space (view frustum) as a restriction zone. In an embodiment, the electronic device (101) may divide the three-dimensional rendering space (view frustum) into a free zone and a restriction zone so as to detect in advance that an object (O) is adjacent to a boundary of the three-dimensional rendering space (view frustum).

[0114] For example, the electronic device (101) can set a free display space (free zone) having a boundary spaced inward by a specified distance from the boundary of the three-dimensional rendering space (view frustum). For example, the electronic device (101) can set the free display space (free zone) by applying a margin of a specified size to the interior of the three-dimensional rendering space (view frustum).

[0115] An electronic device (101) according to one embodiment can freely apply movement, rotation and / or scaling to an object (O) located within a free zone.

[0116] An electronic device (101) according to one embodiment can identify whether at least a portion of an object (O) leaves a free zone in response to a user's input. An electronic device (101) according to one embodiment can identify whether at least a portion of an object (O) leaves a free zone as the position of the user's eyes (L, R) changes.

[0117] In one embodiment, when the electronic device (101) identifies that at least a portion of the object (O) has left the free zone due to a user input, the electronic device (101) may provide feedback related to at least one of rotation, translation, or reduction of the object (O) in response to the user input.

[0118] In one embodiment, the electronic device (101) may restrict movement, rotation, enlargement and / or reduction of an object (O) at least partially located within a restriction zone, or provide feedback to a user's input.

[0119] An electronic device (101) according to one embodiment may determine a type of transformation and / or a transformation weight of feedback to be provided in response to a user's input when it is identified that at least a portion of an object (O) is out of a three-dimensional rendering space (view frustum) or a free display space (free zone).

[0120] FIG. 8A illustrates a state in which an object (O) is rotated, translated, or enlarged so as to leave a three-dimensional rendering space (view frustum) according to an embodiment of the present disclosure. FIG. 8B illustrates a state in which the image of the object (O) appears to overlap as it is enlarged or moved beyond a threshold point according to an embodiment of the present disclosure. FIG. 8C illustrates a state in which an object (O) leaves a three-dimensional rendering space (view frustum) depending on the position of the user's eyes (L, R) according to an embodiment of the present disclosure.

[0121] Referring to FIG. 8A, in an electronic device (101) according to one embodiment, an object (O) included in a stereoscopic screen displayed on a three-dimensional display (230) may be moved, rotated, enlarged, or reduced according to an input of a user (U). For example, a three-dimensional object (O) included in a stereoscopic screen may be rotated, moved, or enlarged so as to leave a three-dimensional rendering space (view frustum). In this way, if at least a part of the object (O) is cut off outside the three-dimensional rendering space (view frustum), the user's (U's) immersion in the three-dimensionally displayed object (O) may be broken.

[0122] Referring to FIG. 8B, in an electronic device (101) according to one embodiment, when an object (O) is rotated, moved, or enlarged beyond a threshold point, at least a portion of the object (O) may leave a three-dimensional rendering space (view frustum) in the forward direction of the three-dimensional display (230). When at least a portion of the object (O) is rotated, moved, or enlarged beyond a threshold point in the forward direction of the three-dimensional display (230), two images of the object (O) may appear to overlap, and thus, the immersion of the user (U) in the object (O) displayed in three dimensions may be broken.

[0123] Referring to FIG. 8C, an electronic device (101) according to one embodiment may set a three-dimensional rendering space (view frustum) differently as the position of the user's eyes (L, R) changes. For example, when a user (U) moves in an external direction of a screen display area based on a three-dimensional display (230), at least a part of an object (O) located inside the three-dimensional rendering space (view frustum) may move outside the newly set three-dimensional rendering space (view frustum). In this case, when at least a part of the object (O) is cut off outside the three-dimensional rendering space (view frustum), the user's (U's) immersion in the object (O) displayed in three dimensions may be broken.

[0124] FIG. 9 illustrates an embodiment of adjusting a movement speed of an object (O) of an electronic device (101) according to a movement input according to one embodiment of the present disclosure.

[0125] Referring to FIG. 9, an electronic device (101) according to one embodiment can receive a movement input from a user (U) for an object (O) included in a three-dimensional screen displayed on a three-dimensional display (230). For example, the movement input from the user (U) for the object (O) can include a drag input or a sliding input for the object (O).

[0126] In one embodiment, when the electronic device (101) receives a movement input from a user (U) for an object (O) within a free zone, the electronic device (101) can move the object (O) displayed on the 3D display (230) in real time. For example, the electronic device (101) can proportionally move the object (O) from a location A to a location B by an amount x in response to a touch input or a movement of the input device by an amount y within the free zone.

[0127] An electronic device (101) according to one embodiment can identify whether at least a portion of an object (O) leaves a free zone due to a movement input from a user (U). For example, the electronic device (101) can identify that an end of the object (O) leaves the free zone at a location B.

[0128] According to one embodiment, the electronic device (101) may adjust the movement speed of the object (O) according to the movement input of the user (U) when it is identified that at least a part of the object (O) is leaving the free zone. For example, the electronic device (101) may slow down the movement distance of the object (O) that is moving in response to the movement input of the user (U). For example, the electronic device (101) may move the object (O) from position B to position C by xx in response to a touch input or a relatively long movement of the input device by yy.

[0129] An electronic device (101) according to one embodiment can adjust the movement speed of an object (O) according to the movement input of the user (U), when the input of the user (U) to the object (O) is a movement input in a direction parallel to the screen display area of ​​the three-dimensional display (230).

[0130] Accordingly, when at least a part of the object (O) leaves the free zone, the user (U) can perceive that the movement of the object (O) according to the movement input is slow, and can be induced to judge that the movement of the object (O) is difficult.

[0131] FIG. 10 illustrates an example of moving an object (O) in the reverse direction of a movement input for an object (O) of an electronic device (101) according to one embodiment of the present disclosure.

[0132] Referring to FIG. 10, an electronic device (101) according to one embodiment can receive a movement input from a user (U) for an object (O) included in a stereoscopic screen displayed on a three-dimensional display (230). In one embodiment, when the electronic device (101) receives a movement input from the user (U) for the object (O) inside a free zone, the electronic device (101) can move the object (O) displayed on the three-dimensional display (230) in real time. For example, the electronic device (101) can proportionally move the object (O) from a position A to a position B by an amount x in response to a touch input or a movement of an input device from a position 1 to a position 2 by an amount y inside the free zone.

[0133] An electronic device (101) according to one embodiment can identify whether at least a portion of an object (O) leaves a free zone due to a movement input from a user (U). For example, the electronic device (101) can identify that an end of the object (O) leaves the free zone at a location B.

[0134] According to one embodiment, the electronic device (101) may move the object (O) in a direction opposite to a movement input of the electronic device (101) to the object (O) when it is identified that at least a portion of the object (O) has left a free zone. For example, the electronic device (101) may move the object (O) in a direction opposite to a direction of a movement input of a user (U).

[0135] For example, the electronic device (101) can move the object (O) from position B to position C in response to a touch input or movement of the input device from position 1 to position 2 by y, and then move the object (O) from position C to position B in the reverse direction of the movement. For example, the electronic device (101) can move the object (O) by xx, which is relatively shorter than x, which moves the object (O) from position A to position B in response to a touch input or movement of the input device from position 1 to position 2 by y within the free zone.

[0136] According to one embodiment, the electronic device (101) can apply a bounce back effect, such as bouncing an object (O) in the reverse direction, in response to a touch input or a movement of the input device by y, and move it to a designated position (e.g., position B where the end of the object (O) leaves the free zone) in the reverse direction of the movement.

[0137] An electronic device (101) according to one embodiment can identify a property of an object (O), and if the property of the identified object (O) is an elastic body, move the object (O) in the reverse direction according to a movement input of a user (U).

[0138] In one embodiment, the material properties of the object (O) can be applied in various ways, such as plastic, wood, cement, and stone. For example, the material properties of the object (O) can be an elastic material with elasticity, such as a thank you, or a magnetic material that responds to magnetism, such as a metal.

[0139] Therefore, if at least a part of the object (O) leaves the free zone, the user (U) can induce the user to judge that the movement of the object (O) is difficult by moving the object (O) in the opposite direction to the movement input.

[0140] FIG. 11 illustrates an example of moving an object (O) of an electronic device (101) to a designated location according to a movement input according to one embodiment of the present disclosure.

[0141] Referring to FIG. 11, an electronic device (101) according to one embodiment can receive a movement input from a user (U) for an object (O) included in a three-dimensional screen displayed on a three-dimensional display (230).

[0142] In one embodiment, when the electronic device (101) receives a movement input from a user (U) for an object (O) within a free zone, the electronic device (101) can move the object (O) displayed on the 3D display (230) in real time. For example, the electronic device (101) can proportionally move the object (O) from a location A to a location B by an amount x in response to a touch input or a movement of the input device by an amount y within the free zone.

[0143] An electronic device (101) according to one embodiment can identify whether at least a portion of an object (O) leaves a free zone due to a movement input from a user (U). For example, the electronic device (101) can identify that an end of the object (O) leaves the free zone at a location B.

[0144] According to one embodiment, the electronic device (101) may move the object (O) to a designated location based on a movement input from a user (U) when it is determined that at least a portion of the object (O) has left a free zone. For example, the designated location may be a location moved in a forward direction (e.g., parallel or in the same direction) of the movement input from the user (U). For example, the electronic device (101) may move the object (O) from a location B to a location C that is moved by xx in response to a touch input or a movement of the input device by yy.

[0145] An electronic device (101) according to one embodiment can set a designated location as a boundary of a three-dimensional rendering space (view frustum) spaced in the positive direction of a movement input of a user (U).

[0146] In one embodiment, when the electronic device (101) identifies that at least a portion of the object (O) has left the free zone, the electronic device (101) may apply a snap effect, such as sticking the object (O) to a magnet, to move the object (O) to a designated location in response to a movement input from the user (U).

[0147] An electronic device (101) according to one embodiment can identify a property of an object (O), and if the property of the identified object (O) is a magnetic material, move to a designated position corresponding to the positive direction of a movement input of a user (U).

[0148] Accordingly, when at least a part of the object (O) leaves the free zone, the user (U) can recognize that the object (O) moves to a designated location according to the movement input, and can be guided to recognize the limit point of the movement of the object (O).

[0149] FIG. 12a illustrates an embodiment of displaying a boundary plane according to the movement of an object (O) according to one embodiment of the present disclosure. FIG. 12b illustrates an embodiment of providing haptic feedback according to the movement of an object (O) according to one embodiment of the present disclosure.

[0150] Referring to FIG. 12a, an electronic device (101) according to one embodiment may provide additional visual effects to increase the user's (U's) awareness when, in response to a user's (U's) movement input for an object (O), the object (O) snaps to a designated position corresponding to the forward direction of the movement input or bounces in the reverse direction of the movement input.

[0151] In one embodiment, the electronic device (101) may display a virtual plane at a designated location when the object (O) is snapped to a designated location corresponding to a forward direction of a movement input or bounced in a reverse direction of the movement input. In one embodiment, the electronic device (101) may display a virtual plane (1200) at a boundary surface of a three-dimensional rendering space (view frustum) corresponding to the forward direction of the movement input. In one embodiment, the electronic device (101) may apply a grid effect that displays a grid on the virtual plane (1200).

[0152] Referring to FIG. 12b, an electronic device (101) according to one embodiment may provide a haptic effect (e.g., a tactile effect) to increase the user's (U's) awareness when, in response to a user's (U's) movement input for an object (O), the object (O) snaps to a designated position corresponding to the forward direction of the movement input or bounces in the reverse direction of the movement input.

[0153] In one embodiment, the electronic device (101) may provide a haptic effect that generates a mechanical vibration or electrical stimulation to the electronic device (101) (e.g., the haptic module (179) of FIG. 1) or the input device when the object (O) snaps to a designated position corresponding to the forward direction of the movement input or bounces in the reverse direction of the movement input.

[0154] FIG. 13 illustrates an example of enlarging an object (O) according to an enlargement input for an object (O) of an electronic device (101) according to one embodiment of the present disclosure.

[0155] Referring to FIG. 13, an electronic device (101) according to one embodiment can receive a size change input from a user (U) for an object (O) included in a three-dimensional screen displayed on a three-dimensional display (230).

[0156] In one embodiment, when the electronic device (101) receives a user's (U's) input for enlarging or reducing an object (O) within a free zone, the electronic device (101) can enlarge or reduce an object (O) displayed on a three-dimensional display (230) in real time. For example, the electronic device (101) can enlarge an object (O) within the free zone in response to an enlargement input via a touch input or an input device.

[0157] An electronic device (101) according to one embodiment can identify whether at least a portion of an object (O) leaves a free zone due to a zoom input from a user (U). For example, the electronic device (101) can identify that an end portion (B) of the object (O) leaves the free zone.

[0158] According to one embodiment, when the electronic device (101) identifies that at least a portion of the object (O) (e.g., an end portion (B) of the object (O)) is out of the free zone, the electronic device (101) may reduce the object (O) so that the object (O) is located within the free zone. For example, the electronic device (101) may enlarge or reduce the object (O) based on the center point (C) of the object (O).

[0159] Therefore, when at least a part of the object (O) leaves the free zone, the user (U) can recognize that the object (O) is reduced according to the zoom input for the object (O), and can be guided to recognize the limit of the zoom of the object (O).

[0160] FIG. 14 illustrates an embodiment of rotating an object (O) according to a rotation input to an object (O) of an electronic device (101) according to an embodiment of the present disclosure. FIG. 15 illustrates an embodiment of moving a rotation axis of an object (O) according to a rotation input to an object (O) of an electronic device (101) according to an embodiment of the present disclosure.

[0161] Referring to FIGS. 14 and 15, an electronic device (101) according to one embodiment can receive a rotation input from a user (U) for an object (O) included in a three-dimensional screen displayed on a three-dimensional display (230).

[0162] In one embodiment, when the electronic device (101) receives a rotation input for the object (O) within a three-dimensional rendering space (view frustum), the electronic device (101) can rotate the object (O) displayed on the three-dimensional display (230) in real time. For example, the electronic device (101) can rotate the object (O) within the three-dimensional rendering space (view frustum) in response to a rotation input via a touch input or an input device. For example, the electronic device (101) can rotate the object (O) with the center point of the object (O) as a rotation axis (axis 1).

[0163] In one embodiment, the electronic device (101) can identify whether at least a portion of the object (O) is out of the three-dimensional rendering space (view frustum) based on a rotation input of a user (U) with respect to the object (O). For example, the electronic device (101) can identify that at least a portion of the object (O) (e.g., an end portion of the object (O)) is out of the three-dimensional rendering space (view frustum) as the object (O) is rotated by a rotation angle (B) based on the rotation input of the user (U).

[0164] In one embodiment, the electronic device (101) can rotate the object (O) in the opposite direction of the rotation input of the user (U). In one embodiment, the electronic device (101) can rotate the object (O) at a rotation angle (C) such that at least a portion of the object (O) that is displaced outside the three-dimensional rendering space (view frustum) due to the rotation of the object (O) is brought into the three-dimensional rendering space (view frustum). For example, the electronic device (101) can rotate the object (O) with the center point (axis 1) of the object (O) as the rotation axis.

[0165] In one embodiment, the electronic device (101) can move the rotation axis (axis 2) of the object (O) toward at least a portion of the object (O) that is out of the three-dimensional rendering space (view frustum). In one embodiment, the electronic device (101) can rotate the object (O) by a rotation angle (C) such that at least a portion of the object (O) that is out of the three-dimensional rendering space (view frustum) due to the rotation of the object (O) is brought into the three-dimensional rendering space (view frustum) along with the movement of the rotation axis (axis 2) of the object (O). Accordingly, the rotation range and / or rotation radius of the object (O) can be reduced.

[0166] Therefore, when at least a part of the object (O) leaves the 3D rendering space (view frustum), the user (U) can recognize that the object (O) is rotated in the opposite direction according to the rotation input for the object (O), and can be guided to recognize the limits of the rotation of the object (O).

[0167] FIG. 16a illustrates an input from a user (U) that falls outside a specified range according to one embodiment of the present disclosure. FIG. 16b illustrates an embodiment of applying a depth-of-field effect to at least a portion of an edge of an object (O) according to one embodiment of the present disclosure.

[0168] Referring to FIGS. 16A and 16B , an electronic device (101) according to one embodiment can identify whether a size of an input of a user (U) is within a specified range. In one embodiment, when at least a part of an object (O) leaves a three-dimensional rendering space (view frustum) or a free zone, the electronic device (101) can identify whether a size of an input of a user (U) from a location where at least a part of the object (O) leaves a three-dimensional rendering space (view frustum) or a free zone is within a specified range.

[0169] As illustrated in FIG. 16A, a movement input of a user (U) according to one embodiment may be a size yy from a position 2 to a position 3, from a position where at least a part of an object (O) leaves the three-dimensional rendering space (view frustum). For example, the electronic device (101) may identify the size (yy) of the movement input of the user (U) as being outside a specified range or not.

[0170] For example, the specified range can be set to a specified distance from where at least part of the object (O) leaves the three-dimensional rendering space (view frustum).

[0171] In one embodiment, the electronic device (101) may apply a specified visual effect to an object (O) through a three-dimensional display (230) based on identifying that the size of the user's (U's) input is outside a specified range.

[0172] In one embodiment, the visual effect may be an effect applied so as not to impair the 3D immersion for an object (O) that is at least partially outside the 3D rendering space (view frustum), or is likely to be outside the 3D rendering space (view frustum).

[0173] As illustrated in FIG. 16b, in one embodiment, the electronic device (101) may apply a depth-of-field effect to at least a portion of the edge of an object (O) based on identifying that the size of the user's (U's) input falls outside a specified range. Accordingly, the edge of the object (O) that deviates from the three-dimensional rendering space (view frustum) may be displayed faintly, thereby providing a buffering effect against visual awkwardness.

[0174] FIG. 17a illustrates an input of a user (U) that falls outside a specified range according to one embodiment of the present disclosure. FIG. 17b illustrates an embodiment of limiting the movement of an object (O) in the depth direction of a three-dimensional display (230) according to one embodiment of the present disclosure.

[0175] Referring to FIGS. 17A and 17B , an electronic device (101) according to one embodiment can identify whether a size of a user's (U's) input is within a specified range. In one embodiment, when at least a part of an object (O) leaves a three-dimensional rendering space (view frustum) or a free zone, the electronic device (101) can identify whether a size of a user's (U's) input from a location where at least a part of the object (O) leaves the three-dimensional rendering space (view frustum) or the free zone is within a specified range.

[0176] As illustrated in FIG. 17a, a movement input of a user (U) according to one embodiment may be a size yy from a position 2 to a position 3, from a position where at least a part of an object (O) leaves the three-dimensional rendering space (view frustum). For example, the electronic device (101) may identify the size (yy) of the movement input of the user (U) as being outside a specified range or not.

[0177] In one embodiment, the electronic device (101) may apply a specified visual effect to an object (O) through a three-dimensional display (230) based on identifying that the size of the user's (U's) input is outside a specified range.

[0178] As illustrated in FIG. 17b, in one embodiment, the electronic device (101) may limit the movement of the object (O) in the depth direction of the 3D display (230) based on identifying that the size of the movement input in the depth direction of the 3D display (230) of the user (U) is outside a specified range.

[0179] Accordingly, the electronic device (101) can prevent the object (O) from being recognized as two images by restricting the movement of the object (O) moving forward of the 3D display (230) while leaving the 3D rendering space (view frustum).

[0180] In one embodiment, the electronic device (101) may, based on identifying that the size of the movement input in the depth direction of the three-dimensional display (230) of the user (U) is outside a specified range, display the object (O) by enlarging it while limiting the movement of the object (O) in response to the movement input in the depth direction of the three-dimensional display (230) of the user (U).

[0181] FIG. 18 is a flowchart (1800) regarding a method of operating an electronic device (101) according to one embodiment of the present disclosure.

[0182] Referring to FIG. 18, an electronic device (101) according to an embodiment may, in operation 1810, obtain at least one value related to the position of the eyes (L, R) of a user (U) from at least one sensor (240; 176). For example, the electronic device (101) may obtain at least one value related to the position of the user (U), the position of the face of the user (U), and / or the position of the eyes (L, R) of the user from the eye tracking sensor (240).

[0183] According to one embodiment, an electronic device (101) may, in operation 1820, set a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230; 160) capable of displaying a three-dimensional screen. In one embodiment, the electronic device (101) may identify the positions of the user's eyes (L, R) based on at least one value related to the positions of the user's eyes (L, R), and set a three-dimensional rendering space (view frustum) based on the positions of the identified eyes (L, R).

[0184] An electronic device (101) according to one embodiment can identify, at operation 1830, whether at least a portion of an object (O) is located in front of a screen display area (235) of a three-dimensional display (230; 160) in a three-dimensional rendering space (view frustum).

[0185] According to one embodiment, when an electronic device (101) identifies that at least a part of an object (O) is located in front of a screen display area (235) of a three-dimensional display (230; 160) in a three-dimensional rendering space (view frustum) (operation 1830 - Yes), in operation 1840, the electronic device (101) can obtain an input from a user (U) related to movement of the object (O) included in the three-dimensional screen.

[0186] In one embodiment, user (U) input related to movement of an object (O) may relate to rotation, enlargement and / or reduction of the object (O).

[0187] According to one embodiment, the electronic device (101) can display a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on an input of a user (U) related to movement of the object (O).

[0188] According to one embodiment, when the electronic device (101) identifies that at least a portion of the object (O) is not located in front of the screen display area (235) of the three-dimensional display (230; 160) in the three-dimensional rendering space (view frustum) (operation 1830-No), even if an input from the user (U) occurs, the electronic device (101) may not identify the relative position of the object in the three-dimensional rendering space (view frustum) or may not provide feedback for adjusting the display of the object in the three-dimensional rendering space (view frustum).

[0189] In one embodiment, the electronic device (101) may, in operation 1850, identify whether at least a portion of an object (O) is out of a three-dimensional rendering space (view frustum) based on an input of a user (U). In one embodiment, the electronic device (101) may monitor whether at least a portion of an object (O) that is rotated, reduced, or enlarged by an input of a user (U) is out of a three-dimensional rendering space (view frustum).

[0190] According to one embodiment, when the electronic device (101) identifies, based on the input of the user (U), that at least a part of the object (O) is out of the three-dimensional rendering space (view frustum) (operation 1850 - Yes), in operation 1860, it can identify whether the size of the input of the user (U) is within a specified range.

[0191] According to one embodiment, the electronic device (101) may obtain the user's (U's) input again in operation 1840 if it is determined that at least a portion of the object (O) does not leave the three-dimensional rendering space (view frustum) based on the user's (U's) input (operation 1850-No).

[0192] According to one embodiment, if the size of the input of the user (U) is within a specified range (operation 1860 - Yes), in operation 1870, the electronic device (101) may provide feedback related to at least one of rotation, translation, or reduction of the object (O) or the preview in response to the input of the user (U).

[0193] An electronic device (101) according to one embodiment may display a preview corresponding to a user's (U's) input while the user's (U's) input is maintained. An electronic device (101) according to one embodiment may display the feedback at a point in time when it is determined that the user's input has been released.

[0194] An electronic device (101) according to one embodiment may display a preview corresponding to a user's (U) input at the time when the user's input is confirmed. An electronic device (101) according to one embodiment may display the feedback after a preset time from the time when the user's input is confirmed.

[0195] In another embodiment, the user input may include input by a stylus pen or a controller that transmits manipulation input according to a slider or coordinates. In another embodiment, the user input may include touch input.

[0196] For example, feedback may be provided in response to a user's (U) input to an object (O) that is likely to deviate or depart from the three-dimensional rendering space (view frustum), thereby causing the user (U) to perceive that a culling phenomenon is occurring in the three-dimensionally displayed object (O).

[0197] According to one embodiment, the electronic device (101) may apply a specified visual effect to the object (O) in operation 1880 when the size of the input of the user (U) is outside the specified range (operation 1860-No). In one embodiment, when the input of the user (U) with a size outside the specified range occurs to the object (O), the electronic device (101) may rotate, reduce, or enlarge the object (O) to correspond to the input of the user (U). In one embodiment, when the input of the user (U) with a size outside the specified range occurs to the object (O), the electronic device (101) may apply a visual effect to the object (O) that reduces the sense of incongruity due to the curling phenomenon occurring in the object (O).

[0198] FIG. 19 is a flowchart (1900) regarding a method of operating an electronic device (101) according to one embodiment of the present disclosure.

[0199] Referring to FIG. 19, an electronic device (101) according to one embodiment may, in operation 1910, obtain at least one value related to the position of an eye (L, R) of a user (U) from at least one sensor (240; 176).

[0200] An electronic device (101) according to one embodiment can set a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230; 160) capable of displaying a three-dimensional screen, in operation 1920.

[0201] An electronic device (101) according to one embodiment can identify, at operation 1930, whether at least a portion of an object (O) is located in front of a screen display area (235) of a three-dimensional display (230; 160) in a three-dimensional rendering space (view frustum).

[0202] According to one embodiment, when the electronic device (101) identifies that at least a part of the object (O) is located in front of the screen display area (235) of the 3D display (230; 160) in the 3D rendering space (view frustum) (operation 1930 - Yes), in operation 1940, the electronic device (101) may set a free zone inside the 3D rendering space (view frustum). In one embodiment, the electronic device (101) may set the free zone by applying a margin of a predetermined distance from the boundary of the 3D rendering space (view frustum). In one embodiment, the electronic device (101) may set a space inside the 3D rendering space (view frustum) but outside the free zone as a restriction zone.

[0203] An electronic device (101) according to one embodiment may, in operation 1950, obtain a user's (U's) input related to rotation, movement, or enlargement of an object (O) included in a three-dimensional screen.

[0204] An electronic device (101) according to one embodiment can display a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on an input of a user (U) related to movement of the object (O).

[0205] According to one embodiment, when the electronic device (101) identifies that at least a part of the object (O) is not located in front of the screen display area (235) of the three-dimensional display (230; 160) in the three-dimensional rendering space (view frustum) (operation 1930-No), even if an input from the user (U) occurs, the electronic device (101) may not identify the relative position of the preview corresponding to the input from the user (U) of the object (O) in the three-dimensional rendering space (view frustum), or may not provide feedback according to the relative position with respect to the three-dimensional rendering space (view frustum).

[0206] In one embodiment, the electronic device (101) may, in operation 1960, identify whether at least a portion of an object (O) leaves a free zone based on an input of a user (U). In one embodiment, the electronic device (101) may monitor whether at least a portion of an object (O) that is rotated, moved, or enlarged by an input of a user (U) leaves a free zone and enters a restriction zone.

[0207] According to one embodiment, when the electronic device (101) identifies, based on the input of the user (U), that at least a part of the object (O) is out of the free zone (operation 1960 - Yes), in operation 1970, it can identify whether the size of the input of the user (U) is within a specified range.

[0208] According to one embodiment, the electronic device (101) may obtain the user's (U's) input again in operation 1950 if it is determined that at least a portion of the object (O) does not leave the free zone based on the user's (U's) input (operation 1960-No).

[0209] According to one embodiment, the electronic device (101) may provide feedback related to movement of the object (O) in response to the input of the user (U) in operation 1980, if the size of the input of the user (U) is within a specified range (operation 1970 - Yes).

[0210] According to one embodiment, the electronic device (101) may apply a specified visual effect to the object (O) in operation 1990 when the size of the input of the user (U) is outside the specified range (operation 1970-No).

[0211] FIG. 20 is a flowchart (2000) regarding a method of operating an electronic device (101) according to one embodiment of the present disclosure.

[0212] Referring to FIG. 20, an electronic device (101) according to one embodiment may determine, in operation 2010, whether a specified update condition is satisfied. In one embodiment, the specified update condition may be a condition for updating at least one value related to the position of the eye (L, R) of the user (U) from at least one sensor (240; 176), or updating a three-dimensional rendering space (view frustum) of a three-dimensional display (230; 160).

[0213] For example, the electronic device (101) may obtain at least one value related to the position of the eye (L, R) of the user (U) from at least one sensor (240; 176) according to a specified period. For example, the electronic device (101) may be configured to update the position of the user (U) or the position of the eye (L, R) of the user as the obtained at least one value related to the position of the eye (L, R) of the user (U) changes.

[0214] According to one embodiment, if the electronic device (101) determines that a specified update condition is satisfied (operation 2010 - Yes), in operation 2020, the electronic device (101) may obtain at least one value related to the position of the eye (L, R) of the user (U) from at least one sensor (240; 176).

[0215] According to one embodiment, if the electronic device (101) determines that the specified update condition is not satisfied (operation 2010-No), it can determine again in operation 2010 whether the specified update condition is satisfied.

[0216] An electronic device (101) according to one embodiment can set a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230; 160) capable of displaying a three-dimensional screen in operation 2030.

[0217] Additionally, the electronic device (101) may optionally include an operation for identifying whether at least a portion of the object (O) is positioned in front of a screen display area (235) of a three-dimensional display (230; 160) in a three-dimensional rendering space (view frustum).

[0218] An electronic device (101) according to one embodiment may, in operation 2040, set a free zone within a three-dimensional rendering space (view frustum). In one embodiment, the electronic device (101) may set the free zone by applying a margin of a certain distance from the boundary of the three-dimensional rendering space (view frustum).

[0219] In one embodiment, the electronic device (101) may, in operation 2050, identify whether at least a portion of the object (O) leaves the free zone. In one embodiment, the 3D rendering space (view frustum) and / or the free zone may change as the position of the user (U), the position of the face or the eyes (L, R) of the user (U) change.

[0220] In one embodiment, the electronic device (101) can identify whether an object (O) of a stereoscopic screen displayed on a three-dimensional display (230; 160) has left a newly updated three-dimensional rendering space (view frustum) and / or a free zone. For example, the electronic device (101) can identify whether the object has left the three-dimensional rendering space (view frustum) and / or a free zone so as to match a situation in which the object is rotated, moved, or enlarged by an input from a user (U).

[0221] An electronic device (101) according to one embodiment may omit the operation of setting a free zone and identify whether at least a part of an object (O) leaves a three-dimensional rendering space (view frustum).

[0222] An electronic device (101) according to one embodiment can, in operation 2060, rotate, move, or scale an object (O) and display it on a three-dimensional display (230; 160).

[0223] In one embodiment, the electronic device (101) can move the object (O) into the 3D rendering space (view frustum) and / or the free zone when at least a portion thereof leaves the 3D rendering space (view frustum) and / or the free zone. In one embodiment, the electronic device (101) can reduce the object (O) so that at least a portion of the object (O) that has moved out is located inside the 3D rendering space (view frustum) and / or the free zone when at least a portion thereof leaves the 3D rendering space (view frustum) and / or the free zone.

[0224] The method of operating the electronic device (101) described with reference to FIG. 20 may further include the operations described with reference to FIG. 18 and / or FIG. 19. A detailed description thereof is omitted as it is redundant.

[0225] The technical tasks to be achieved in this document are not limited to the technical tasks mentioned above, and other technical tasks not mentioned will be clearly understood by those with ordinary skill in the technical field to which this document pertains from the description below.

[0226] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

[0227] An electronic device (101) according to one embodiment of the present disclosure may include a three-dimensional display (230, 160) capable of displaying a three-dimensional screen, at least one sensor (240, 176) configured to measure at least one value related to the position of an eye (L, R) of a user (U), and at least one processor (210, 120). The at least one processor (210, 120) may be configured to obtain the at least one value from the at least one sensor (240, 176).

[0228] The at least one processor (210, 120) may be configured to set a three-dimensional rendering space (view frustum) including a screen display area (235) of the three-dimensional display (230, 160) based on the at least one value.

[0229] The at least one processor (210, 120) may be set to obtain input related to movement of an object (O) included in the three-dimensional rendering space displayed through the three-dimensional display.

[0230] The at least one processor (210, 120) may, based on the input, display a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160).

[0231] The at least one processor (210, 120) may provide feedback for adjusting the display of the object (O) in the 3D rendering space based on the position of the object (O) in the 3D rendering space corresponding to the input after the preview is displayed.

[0232] In an electronic device (101) according to one embodiment, the three-dimensional display (230, 160) may include a lenticular lens (510) or a paralax barrier (520) positioned in front of the screen display area (235) to induce binocular parallax.

[0233] In an electronic device (101) according to one embodiment, the input may include an input for rotating the object (O) displayed in the three-dimensional rendering space.

[0234] In an electronic device (101) according to one embodiment, the input may include an input for reducing or enlarging the object (O) displayed in the three-dimensional rendering space.

[0235] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to set a three-dimensional rendering space (view frustum) extending forward and backward of the screen display area (235) of the three-dimensional display (230, 160), at least as a part of an operation of setting the three-dimensional rendering space (view frustum). The at least one processor (210, 120) may be configured to identify whether at least a portion of the object (O) is positioned in front of the screen display area (235) in the three-dimensional rendering space (view frustum). The at least one processor (210, 120) may be configured to provide feedback related to at least one of rotation, reduction, or enlargement of the object (O) based on the at least a portion of the object (O) being identified as being positioned in front of the screen display area (235).

[0236] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to set a free zone within the three-dimensional rendering space (view frustum).

[0237] The at least one processor (210, 120) may be configured to identify whether the at least a portion of the object (O) leaves the free zone in response to the input. The at least one processor (210, 120) may be configured to provide feedback related to at least one of rotation, reduction, or enlargement of the object (O) based on identifying that the at least a portion of the object (O) leaves the free zone.

[0238] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to adjust a moving speed of the object (O) according to the input based on identifying that at least a portion of the object (O) has left the free zone, at least as part of the operation of providing the feedback.

[0239] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to move the object (O) to a designated position corresponding to the direction of the input or in the opposite direction of the input, based on identifying that at least a portion of the object (O) has left the free zone, as at least part of the operation of providing the feedback.

[0240] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to identify a physical property of the object (O). The at least one processor (210, 120) may be configured to, at least as part of an operation of providing the feedback, snap the object (O) to a designated position corresponding to a direction of the input or bounce the object (O) in the opposite direction of the input, based on the physical property of the object (O).

[0241] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to, at least as a part of an operation of identifying whether the at least part of the object (O) leaves the free zone, identify whether the at least part of the object (O) leaves the free zone based on an input of enlarging the object (O). The at least one processor (210, 120) may be configured, at least as a part of an operation of providing the feedback, to, based on identifying that the at least part of the object (O) leaves the free zone, reduce the object (O) so that the object (O) is located within the free zone.

[0242] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to identify, based on a rotational input for the object (O), whether at least a portion of the object (O) is leaving the three-dimensional rendering space (view frustum). The at least one processor (210, 120), as at least a part of an operation of providing the feedback, may be configured to rotate the object (O) in a direction opposite to the rotational input based on identifying that the at least a portion of the object (O) is leaving the three-dimensional rendering space (view frustum).

[0243] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to move the rotational axis of the object (O) toward at least a portion of the object (O) that leaves the three-dimensional rendering space (view frustum), at least as part of the operation of providing the feedback.

[0244] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to identify whether the size of the input is within a specified range. The at least one processor (210, 120) may be configured to apply a specified visual effect to the object (O) through the three-dimensional display (230, 160) based on identifying that the size of the input is outside the specified range.

[0245] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to apply a depth of field effect to at least a portion of an edge of the object (O), or to restrict movement of the object (O) in a depth direction of the three-dimensional display (230, 160), as at least part of an operation of applying the specified visual effect to the object (O).

[0246] A method of operating an electronic device according to one embodiment of the present disclosure may include an operation (1810, 1910) of obtaining at least one value related to the position of an eye (L, R) of a user (U) from at least one sensor (240, 176).

[0247] A method of operating an electronic device according to one embodiment may include an operation (1820, 1920) of setting a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230, 160) capable of displaying a three-dimensional screen based on at least one value.

[0248] A method of operating an electronic device according to one embodiment may include an operation (1840, 1950) of obtaining an input related to movement of an object (O) included in the three-dimensional rendering space displayed through the three-dimensional display.

[0249] A method of operating an electronic device according to one embodiment may include an operation of displaying a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on the input.

[0250] A method of operating an electronic device according to one embodiment may include, after the preview is displayed, providing feedback for adjusting the display of the object (O) in the three-dimensional rendering space based on a position of the object (O) in the three-dimensional rendering space corresponding to the input (1870; 1980).

[0251] In an operating method of an electronic device according to one embodiment, the operation (1820, 1920) of setting the three-dimensional rendering space (view frustum) may include an operation of setting the three-dimensional rendering space (view frustum) extending forward and backward of the screen display area (235) of the three-dimensional display (230, 160).

[0252] A method of operating an electronic device according to one embodiment may further include an operation (1830, 1930) of identifying whether at least a part of the object (O) is located in front of the screen display area (235) in the three-dimensional rendering space (view frustum).

[0253] In an operating method of an electronic device according to one embodiment, an operation (1870; 1980) of providing feedback related to at least one of rotation, reduction, or enlargement of the object (O) may be included based on identification that at least a portion of the object (O) is positioned in front of the screen display area (235).

[0254] A method of operating an electronic device according to one embodiment may further include an operation (1940) of setting a free zone within the three-dimensional rendering space (view frustum).

[0255] A method of operating an electronic device according to one embodiment may further include an operation (1960) of identifying whether at least a portion of the object (O) leaves the free zone in response to the input.

[0256] In a method of operating an electronic device according to one embodiment, the operation (1980) of providing the feedback may include an operation of providing feedback related to at least one of rotation, enlargement, or reduction of the object (O) based on identifying that at least a portion of the object (O) has left the free zone.

[0257] The method of operating an electronic device according to one embodiment may further include an operation (1860, 1970) of identifying whether the size of the input is within a specified range.

[0258] An operating method of an electronic device according to one embodiment may further include an operation (1880, 1990) of applying a specified visual effect to the object (O) through the three-dimensional display (230, 160) based on identifying that the size of the input is outside the specified range.

[0259] A method of operating an electronic device according to one embodiment may further include an operation (1850) of identifying whether at least a portion of the object (O) is out of the three-dimensional rendering space (view frustum) based on an input for rotating the object (O).

[0260] In a method of operating an electronic device according to one embodiment, the operation (1870) of providing the feedback may include an operation of rotating the object (O) in a direction opposite to an input for rotating the object, based on identifying that at least a portion of the object (O) is leaving the three-dimensional rendering space (view frustum).

[0261] A non-transitory computer-readable storage medium storing one or more programs according to one embodiment of the present disclosure may include an operation (1810, 1910) of obtaining at least one value related to the position of an eye (L, R) of a user (U) from at least one sensor (240, 176).

[0262] A storage medium according to one embodiment may include an operation (1820, 1920) of setting a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230, 160) capable of displaying a three-dimensional screen based on at least one value.

[0263] A storage medium according to one embodiment may include an operation (1840, 1950) of obtaining an input related to movement of an object (O) included in the three-dimensional rendering space displayed through the three-dimensional display.

[0264] A storage medium according to one embodiment may include an operation of displaying a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on the input.

[0265] A storage medium according to one embodiment may include an operation (1870; 1980) of providing feedback for adjusting the display of the object (O) in the three-dimensional rendering space based on a position of the object (O) in the three-dimensional rendering space corresponding to the input after the preview is displayed.

[0266] An electronic device (101) according to one embodiment of the present disclosure may include a three-dimensional display (230, 160) capable of displaying a three-dimensional screen, at least one sensor (240, 176) configured to measure at least one value related to the position of an eye (L, R) of a user (U), and at least one processor (210, 120).

[0267] The at least one processor (210, 120) may be configured to obtain the at least one value from the at least one sensor (240, 176).

[0268] The at least one processor (210, 120) may be configured to set a three-dimensional rendering space (view frustum) including a screen display area (235) of the three-dimensional display (230, 160) based on the at least one value.

[0269] The at least one processor (210, 120) may be set to obtain input related to movement of an object (O) included in the three-dimensional rendering space displayed through the three-dimensional display.

[0270] The at least one processor (210, 120) may be configured to display a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on the input.

[0271] The at least one processor (210, 120) may be configured to identify the position of the object (O) in the three-dimensional rendering space corresponding to the input after the preview is displayed.

[0272] The at least one processor (210, 120) may be configured to rotate, enlarge, or reduce the object (O) and display it on the three-dimensional display (230, 160) based on identifying the location of the object (O).

[0273] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to change the three-dimensional rendering space (view frustum) based on a change in the position of the eye (L, R) of the user (U), at least as a part of an operation of setting the three-dimensional rendering space (view frustum).

[0274] In an electronic device (101) according to one embodiment, the at least one processor (210, 120) may be configured to set a free zone within the three-dimensional rendering space (view frustum).

[0275] The at least one processor (210, 120) may be configured to identify whether at least a portion of the object (O) leaves the free zone.

[0276] The at least one processor (210, 120) may be configured to display the object (O) by rotating, enlarging, or reducing the object (O) based on identifying that at least a portion of the object (O) has left the free zone.

[0277] A method of operating an electronic device according to one embodiment of the present disclosure may include an operation (2020) of obtaining at least one value related to the position of an eye (L, R) of a user (U) from at least one sensor (240, 176).

[0278] A method of operating an electronic device according to one embodiment may include an operation (2030) of setting a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230, 160) based on the at least one value.

[0279] A method of operating an electronic device according to one embodiment may include an operation (2050) of identifying a relative position of a preview corresponding to the input of the object (O) with respect to the three-dimensional rendering space (view frustum).

[0280] An operating method of an electronic device according to one embodiment may include an operation (2060) of rotating, moving, or reducing the object (O) and displaying it on the three-dimensional display (230, 160) based on the identification result.

[0281] A non-transitory computer-readable storage medium storing one or more programs according to one embodiment of the present disclosure may include an operation (2020) of obtaining at least one value related to the position of an eye (L, R) of a user (U) from at least one sensor (240, 176). The storage medium according to one embodiment may include an operation (2030) of setting a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230, 160) based on the at least one value. The storage medium according to one embodiment may include an operation (2050) of identifying a relative position of a preview of the object (O) with respect to the three-dimensional rendering space (view frustum). The storage medium according to one embodiment may include an operation (2060) of rotating, moving, or reducing the object (O) and displaying it on the three-dimensional display (230, 160) based on the identification result.

[0282] Electronic devices according to the embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0283] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0284] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0285] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0286] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0287] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), Memory for storing instructions (130; 220); 3D display capable of displaying a three-dimensional screen (230; 160); At least one sensor (240; 176) configured to measure at least one value related to the position of the eye (L, R) of the user (U); and Contains a processor (210; 120), The above instructions, when executed by the processor, cause the electronic device to: Obtaining at least one value from at least one sensor (240; 176), Based on at least one of the above values, a three-dimensional rendering space (view frustum) including a screen display area (235) of the three-dimensional display (230; 160) is set, Obtaining input related to movement of an object (O) included in the three-dimensional rendering space displayed through the three-dimensional display, Based on the above input, a preview of the object (O) moved in the three-dimensional rendering space is displayed through the three-dimensional display (230; 160), An electronic device (101) that causes, after the preview is displayed, to provide feedback for adjusting the display of the object (O) in the three-dimensional rendering space based on the position of the object (O) in the three-dimensional rendering space corresponding to the input.

2. In paragraph 1, The above 3D display (230; 160) includes a lenticular lens (510; lenticular lens) or paralax barrier (520; paralax barrier) positioned in front of the screen display area (235) to induce binocular parallax. Electronic device (101).

3. In any one of paragraphs 1 to 2, The above input is, An electronic device (101) including an input for rotating, reducing, or enlarging the object (O) displayed in the three-dimensional rendering space.

4. In any one of paragraphs 1 to 3, The above instructions, when executed by the processor, cause the electronic device to: As at least part of the operation of setting the three-dimensional rendering space (view frustum), setting the three-dimensional rendering space (view frustum) extending forward and backward of the screen display area (235) of the three-dimensional display (230; 160), Identifying whether at least a portion of the object (O) is located in front of the screen display area (235) in the three-dimensional rendering space (view frustum), Causing to provide feedback related to at least one of rotation, reduction, or enlargement of the object (O), based on identification that at least a portion of the object (O) is located in front of the screen display area (235). Electronic device (101).

5. In any one of paragraphs 1 to 4, The above instructions, when executed by the processor, cause the electronic device to: A free zone is set within the above 3D rendering space (view frustum), In response to the input, identify whether at least a part of the object (O) leaves the free zone, Causing to provide feedback related to at least one of rotation, reduction, or enlargement of the object (O), based on identifying that at least a portion of the object (O) has left the free zone. Electronic device (101).

6. In any one of paragraphs 1 to 5, The above instructions, when executed by the processor, cause the electronic device to: At least as part of the action of providing said feedback, causing said object (O) to adjust its movement speed based on said input, based on identifying that at least a portion of said object (O) has left said free zone. Electronic device (101).

7. In any one of paragraphs 1 to 6, The above instructions, when executed by the processor, cause the electronic device to: As at least part of the action of providing said feedback, causing said object (O) to move to a designated position corresponding to the direction of said input or moving in the opposite direction of said input, based on identifying that at least a portion of said object (O) has left said free zone. Electronic device (101).

8. In any one of paragraphs 1 to 7, The above instructions, when executed by the processor, cause the electronic device to: Identify the properties of the above object (O), At least as part of the action of providing said feedback, based on a property of said object (O), causing said object (O) to snap to a designated position corresponding to the direction of said input or to bounce in the opposite direction of said input. Electronic device (101).

9. In any one of paragraphs 1 to 8, The above instructions, when executed by the processor, cause the electronic device to: As at least part of the operation of identifying whether at least a portion of the object (O) exits the free zone, based on an input that enlarges the object (O), identifying whether at least a portion of the object (O) exits the free zone, At least as part of the action of providing said feedback, causing said object (O) to be reduced so that said object (O) is positioned within said free zone, based on identifying that at least a portion of said object (O) is leaving said free zone. Electronic device (101).

10. In any one of paragraphs 1 to 9, The above instructions, when executed by the processor, cause the electronic device to: Based on an input for rotating the object (O), identifying whether at least a part of the object (O) is out of the three-dimensional rendering space (view frustum), As at least part of the action of providing said feedback, causing said object (O) to rotate in the opposite direction of the input rotating said object (O) based on identifying that at least a portion of said object (O) is leaving the three-dimensional rendering space (view frustum). Electronic device (101).

11. In any one of paragraphs 1 to 10, The above instructions, when executed by the processor, cause the electronic device to: As at least part of the action of providing said feedback, causing the rotational axis of said object (O) to move towards at least a portion of said object (O) that leaves said three-dimensional rendering space (view frustum). Electronic device (101).

12. In any one of paragraphs 1 to 11, The above instructions, when executed by the processor, cause the electronic device to: Identify whether the size of the above input is within a specified range, Based on identifying that the size of the above input is outside the above specified range, causing a specified visual effect to be applied to the object (O) through the three-dimensional display (230; 160). Electronic device (101).

13. In any one of paragraphs 1 to 12, The above instructions, when executed by the processor, cause the electronic device to: As at least part of the operation of applying the specified visual effect to the object (O), applying a depth of field effect to at least a portion of an edge of the object (O), or causing movement of the object (O) to be restricted in the depth direction of the three-dimensional display (230; 160). Electronic device (101).

14. In the operating method of an electronic device (101), An operation (1810; 1910) of obtaining at least one value related to the position of the eye (L, R) of the user (U) from at least one sensor (240; 176) included in the electronic device (101); An operation (1820; 1920) of setting a three-dimensional rendering space (view frustum) including a screen display area (235) of a three-dimensional display (230; 160) included in the electronic device (101) capable of displaying a three-dimensional screen based on at least one of the above values; An operation (1840; 1950) of obtaining input related to movement of an object (O) included in the three-dimensional rendering space displayed through the three-dimensional display; and An operation of displaying a preview of the object (O) moved in the three-dimensional rendering space through the three-dimensional display (230; 160) based on the above input; and After the preview is displayed, an operation (1870; 1980) is included for providing feedback for adjusting the display of the object (O) in the 3D rendering space based on the position of the object (O) in the 3D rendering space corresponding to the input. How it works.

15. In an electronic device (101), Memory for storing instructions (130; 220); 3D display capable of displaying a three-dimensional screen (230; 160); At least one sensor (240; 176) configured to measure at least one value related to the position of the eye (L, R) of the user (U); and Contains a processor (210; 120), The above instructions, when executed by the processor, cause the electronic device to: Obtaining at least one value from at least one sensor (240; 176), Based on at least one of the above values, a three-dimensional rendering space (view frustum) including a screen display area (235) of the three-dimensional display (230; 160) is set, Obtaining input related to movement of an object (O) included in the three-dimensional rendering space displayed through the three-dimensional display, Based on the above input, a preview of the object (O) moved in the three-dimensional rendering space is displayed through the three-dimensional display (230; 160), After the above preview is displayed, the position of the object (O) in the 3D rendering space corresponding to the input is identified, Based on identifying the position of the object (O), the object (O) is set to be displayed on the three-dimensional display (230; 160) by rotating, enlarging, or reducing the object (O). Electronic device (101).

Images