Method and electronic device for providing user interaction for stereoscopic element

Abstract

An embodiment of the present invention may comprise: a display (160); a sensor module (176); a memory (130) for storing instructions; and a processor (120). The instructions, when executed by the processor, may cause an electronic device to: detect a user input for calling a control menu while a three-dimensional stereoscopic object is displayed on the display; display the control menu on the display in response to the detected user input; detect a touch selecting an option item of the control menu; detect movement of the electronic device by using the sensor module while the touch on the selected option item is maintained; and change a user interface displayed on the display on the basis of the detected movement of the electronic device when the touch on the selected option item is released.

Description

Method for providing user interaction with three-dimensional elements and electronic device thereof

[0001] Various embodiments of the present invention disclose a method for providing user interaction for a three-dimensional element and an electronic device thereof.

[0002] With the advancement of digital technology, various types of electronic devices, such as mobile terminals, personal digital assistants (PDAs), electronic notebooks, smartphones, tablet PCs (personal computers), and wearable devices, are becoming widely used. These electronic devices are constantly undergoing improvements in their hardware and / or software to support and enhance their functionality.

[0003] For example, an electronic device can use voice recognition technology to control the execution of an electronic device or application through a user's voice commands, or perform functions using a web-based service. Alternatively, the electronic device can connect to a wireless input / output device (e.g., earphones, headphones) via short-range wireless communication such as Bluetooth, and output the sound of music or videos through the wireless input / output device. Alternatively, an electronic device (e.g., a smartphone) can connect to a wearable display device (e.g., AR glasses) to provide extended reality (XR) content such as virtual reality (VR), augmented reality (AR), and / or mixed reality (MR).

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] In one embodiment, a method and device for providing a control menu that enables a user to operate an electronic device with one hand without obscuring a three-dimensional stereoscopic object displayed on a display of an electronic device and changing a user interface through the control menu are disclosed.

[0006] An electronic device (101) according to an embodiment of the present disclosure may include a display (160), a sensor module (176), a memory (130) storing instructions, and a processor (120). The instructions, when executed by the processor, may cause the electronic device to detect a user input for invoking a control menu while displaying a three-dimensional stereoscopic object on the display, display the control menu on the display in response to the detected user input, detect a touch for selecting an option item of the control menu, detect a movement of the electronic device using the sensor module while maintaining the touch on the selected option item, and change a user interface displayed on the display based on the detected movement of the electronic device when the touch on the selected option item is released.

[0007] A method of operating an electronic device (101) according to an embodiment of the present disclosure may include an operation of detecting a user input for calling a control menu while displaying a three-dimensional stereoscopic object on a display (160) of the electronic device, an operation of displaying a control menu on the display in response to the detected user input, an operation of detecting a touch for selecting an option item of the control menu, an operation of detecting a movement of the electronic device using a sensor module (176) of the electronic device while maintaining the touch on the selected option item, and an operation of changing a user interface displayed on the display based on the detected movement of the electronic device when the touch on the selected option item is released.

[0008] According to one embodiment, a display can be controlled without obscuring a three-dimensional stereoscopic element.

[0009] According to one embodiment, a three-dimensional stereoscopic element can be easily controlled while holding the electronic device with one hand without obscuring the three-dimensional stereoscopic element.

[0010] According to one embodiment, the usability of an electronic device can be improved by providing an environment in which user interaction can be performed without obstructing the view of a three-dimensional stereoscopic element.

[0011] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0012] FIG. 2 is a drawing illustrating an example of controlling a three-dimensional stereoscopic object in an electronic device according to one embodiment.

[0013] FIG. 3 is a flowchart illustrating an operating method of an electronic device according to one embodiment.

[0014] FIG. 4 is a diagram illustrating an example of calling a control menu in an electronic device according to one embodiment.

[0015] FIG. 5A is a diagram illustrating an example of selecting depth movement of a control menu in an electronic device according to one embodiment.

[0016] FIG. 5b is a diagram illustrating an example of selecting a camera view change in a control menu in an electronic device according to one embodiment.

[0017] FIG. 5c is a diagram illustrating an example of selecting screen movement of a control menu in an electronic device according to one embodiment.

[0018] FIG. 6 is a flowchart illustrating a method for controlling a user interface through a control menu of an electronic device according to one embodiment.

[0019] FIG. 7 is a diagram illustrating an example of changing a user interface by selecting a control menu in an electronic device according to one embodiment.

[0020] FIGS. 8A to 8C are drawings illustrating an example of selecting a three-dimensional object through a control menu in an electronic device according to one embodiment.

[0021] FIGS. 9A and 9B are drawings illustrating an example of selecting a three-dimensional stereoscopic object through a control menu in an electronic device according to one embodiment.

[0022] FIGS. 10A to 10C are drawings illustrating an example of selecting a three-dimensional object through a control menu in an electronic device according to one embodiment.

[0023] FIG. 11 is a flowchart illustrating a method for selecting a three-dimensional stereoscopic object through a control menu of an electronic device according to one embodiment.

[0024] FIG. 12A is a diagram illustrating an example of selecting depth movement in a control menu in a VR mode of an electronic device according to one embodiment.

[0025] FIG. 12b is a diagram illustrating an example of selecting a camera view change in a control menu in a VR mode of an electronic device according to one embodiment.

[0026] FIG. 12c is a diagram illustrating an example of selecting screen movement of a control menu in a VR mode of an electronic device according to one embodiment.

[0027] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0028] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display (160)).

[0029] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0030] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., the display (160), the sensor module (176), or the communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0031] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0032] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0033] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0034] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0035] The display (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display (160) may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling the device. In one embodiment, the display (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the strength of a force generated by the touch.

[0036] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0037] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0038] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0039] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0040] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0041] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0042] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0043] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0044] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0045] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0046] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0047] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0048] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0049] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0050] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0051] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0052] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0053] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0054] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0055] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0056] FIG. 2 is a drawing illustrating an example of controlling a three-dimensional stereoscopic object in an electronic device according to one embodiment.

[0057] Referring to FIG. 2, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may provide a first user interface (210) including a three-dimensional (3D) object (205). The first user interface (210) may be displayed on a display (e.g., a display (160) of FIG. 1) of the electronic device (101). The three-dimensional (3D) object (205) may be a content corresponding to an object that is displayed three-dimensionally on the display (160) like a virtual reality. The object may include an icon of an application included in a home screen or content (e.g., text, an image, a video) included in an execution screen of an application (e.g., a contact application, a gallery application). The first user interface (210) is intended to help understand the present invention, and in the following, objects (e.g., 231, 233, 235) included in the second user interface (230) are displayed as two-dimensional images, but the objects (231, 233, 235) included in the second user interface (230) may be displayed (or expressed) like three-dimensional objects (205) of the first user interface (210).

[0058] The processor (120) may display a second user interface (230) including a control menu (237) on the display (160) in response to a user input calling up a control menu. The second user interface (230) may further include a pointer (or cursor) (e.g., a cross shape) for selecting a three-dimensional stereoscopic object. The pointer may be displayed in a designated area (e.g., the center) of the display (160). The second user interface (230) may include a plurality of objects (231, 233, 235) and a control menu (237). The plurality of objects (231, 233, 235) may be displayed like the three-dimensional stereoscopic object (205) of the first user interface (210). The plurality of objects (231, 233, 235) may be displayed based on different depth information (or depth values). The above depth information may represent the z-axis coordinate value in the x, y, and z-axis coordinate system.

[0059] For example, the x-axis and y-axis coordinates represent a two-dimensional coordinate system, and the x-axis may represent horizontal information (e.g., left, right) on the screen, and the y-axis may represent vertical information (e.g., up, down) on the screen. The z-axis coordinate represents a three-dimensional coordinate system, and the z-axis may be a depth value on the screen, which may be expressed as a layer. The second object (233) may correspond to the first depth information (or the first layer), and the first object (231) and the third object (235) may correspond to the second depth information (or the second layer). That is, the first object (231) and the third object (235) may have the same depth value. The first depth information is different from the second depth information, and the first depth information may be positioned higher in depth (e.g., closer to the user) than the second depth information.

[0060] The control menu (237) may be displayed in a two-dimensional plane, or in a three-dimensional stereoscopic manner, such as with a plurality of objects (231, 233, 235). The control menu (237) may include at least one option item, for example, at least one of a depth movement item (241), a camera view change item (243), or a screen panning item (245). Screen panning may mean moving or switching the screen up, down, left, or right. In addition, the control menu (237) may further include a touch area for object selection. Although the drawing illustrates the control menu as including three option items, the control menu may include fewer (e.g., two) or more option items than three.

[0061] The depth movement item (241) may change the depth (or layer) information (e.g., z-axis coordinate value) of an object displayed on the display (160). When the user moves the electronic device (101) while maintaining a touch on the depth movement item (241), the processor (120) may adjust the depth values ​​of a plurality of objects included in the second user interface (230) based on the movement of the electronic device (101). Here, the movement of the electronic device (101) may be acquired (or detected) using a sensor module (e.g., sensor module (176) of FIG. 1) included in the electronic device (101). For example, the sensor module (176) may include a 6-axis sensor combining a gyro sensor and an acceleration sensor, or a 9-axis sensor combining a gyro sensor, an acceleration sensor, or a geomagnetic sensor.

[0062] The processor (120) can adjust (or change) the depth values ​​of a plurality of objects included in the second user interface (230) based on a pitch value (e.g., first sensing data) obtained from the sensor module (176). The pitch value (or tilt value) may mean a rotation around an axis in a horizontal plane perpendicular to the movement direction of the electronic device (101). According to one embodiment, when a user input for a depth movement item (241) is maintained (e.g., touch hold) and moves up and down (e.g., touch drag), the processor (120) can adjust (or change) the depth values ​​of a plurality of objects included in the second user interface (230) based on the touch movement moving up and down.

[0063] The camera view change item (243) may change the user interface displayed on the display (160) according to the change in the camera view. The processor (120) may change the camera view based on a pitch value or a yaw value (e.g., second sensing data) obtained from the sensor module (176), and may change the second user interface (230) based on the changed camera view. The yaw value may mean rotation around an axis in a vertical plane perpendicular to the movement direction of the electronic device (101). The user interface is displayed differently depending on the camera view, and the user interface corresponding to the camera view may be stored in the memory of the electronic device (101) (e.g., the memory (130) of FIG. 1).

[0064] The screen panning item (245) may change the user interface displayed on the display (160) up, down, left, and right according to at least one of an x-axis movement value, a y-axis movement value, a pitch value, or a yaw value (e.g., third sensing data). Screen panning may mean moving or switching the screen up, down, left, and right. The user interface may be larger than the screen size of the display (160). When the user moves the electronic device (101) up, down, left, and right, the screen panning item (245) may cause an object (or area) that was not displayed on the display (160) to be displayed, or an object that was displayed on the display (160) may disappear and not be displayed.

[0065] The processor (120) can touch an option item in the control menu, change and display the user interface based on the movement of the electronic device (101) while the touch is maintained, and when the touch is released, fix the changed value of the option item applied at the time of the touch release (e.g., a display value related to the user interface). The user can move the electronic device (101) to align at least one 3D stereoscopic object to the pointer (e.g., align in the x, y, and z-axis directions). The alignment may mean aligning (or positioning in a straight line) at least one 3D stereoscopic object in the x, y, and z-axis directions. When the touch is released and a 3D stereoscopic object corresponds to the pointer, the processor (120) can select the 3D stereoscopic object corresponding to the pointer and execute a function corresponding to the selected 3D stereoscopic object.

[0066] In one embodiment, the processor (120) may touch an option item in a control menu and change and display a user interface based on the movement of the touch (e.g., touch drag, touch swipe) while the touch is maintained.

[0067] An electronic device according to an embodiment of the present disclosure may include a display (160), a sensor module (176), a memory (130) storing instructions, and a processor (120). The instructions, when executed by the processor, may cause the electronic device to detect a user input for invoking a control menu while displaying a three-dimensional stereoscopic object on the display, display the control menu on the display in response to the detected user input, detect a touch for selecting an option item of the control menu, detect a movement of the electronic device using the sensor module while maintaining the touch on the selected option item, and change a user interface displayed on the display based on the detected movement of the electronic device when the touch on the selected option item is released.

[0068] The above instructions, when executed by the processor, may cause the electronic device to display the control menu in a corner edge area of ​​the display and display a pointer in the center of the display in response to the detected user input.

[0069] The above instructions, when executed by the processor, may cause the electronic device to obtain first sensing data in the depth direction from the sensor module when depth movement is selected as an option item of the control menu, and to change depth information of a three-dimensional stereoscopic object displayed on the display according to the release of the touch based on the obtained first sensing data.

[0070] The above instructions, when executed by the processor, may cause the electronic device to obtain second sensing data in the depth direction from the sensor module when a change camera view is selected as an option item of the control menu, change the camera view based on the obtained second sensing data, and change the user interface based on the changed camera view.

[0071] The above instructions, when executed by the processor, may cause the electronic device to obtain third sensing data from the sensor module when screen panning is selected as an option item of the control menu, and to change the user interface based on the obtained third sensing data.

[0072] The above instructions, when executed by the processor, may cause the electronic device to select a three-dimensional stereoscopic object if the three-dimensional stereoscopic object corresponds to the pointer based on the acquired sensing data.

[0073] The above instructions, when executed by the processor, may cause the electronic device to execute a function corresponding to the selected three-dimensional stereoscopic object when the touch on the selected option item is released.

[0074] The above instructions, when executed by the processor, may cause the electronic device to select a three-dimensional stereoscopic object located adjacent to a highlighted area corresponding to the pointer based on the acquired sensing data, and to execute a function corresponding to the selected three-dimensional stereoscopic object when the touch release is detected.

[0075] The above instructions, when executed by the processor, may cause the electronic device to move the three-dimensional stereoscopic object positioned adjacent to the highlighted area to the highlighted area, and select the three-dimensional stereoscopic object when the three-dimensional stereoscopic object is aligned with the highlighted area.

[0076] The above instructions, when executed by the processor, may cause the electronic device to move a first three-dimensional stereoscopic object positioned adjacent to the highlighted area to the highlighted area, and to move a position of a second three-dimensional stereoscopic object displayed on the display based on the movement of the first three-dimensional stereoscopic object.

[0077] Figure 3 is a flowchart (300) illustrating an operating method of an electronic device according to one embodiment.

[0078] Referring to FIG. 3, in operation 301, a processor (e.g., the processor 120 of FIG. 1) of an electronic device (e.g., the electronic device 101 of FIG. 1) according to an embodiment may detect a user input for calling a control menu. The processor (120) may detect the user input for calling the control menu while displaying a user interface (e.g., the second user interface (230) of FIG. 2) including at least one three-dimensional object. The user input may include a touch input of dragging (or swiping) in a bezel area of ​​the electronic device (101) toward a display (e.g., the display (160) of FIG. 1) of the electronic device (101). For example, the user input may mean a user interaction in which a touch is initiated in a bezel area corresponding to a corner of the electronic device (101) and moves inward toward the display (160). The user input for calling the above control menu may be set as a default in the electronic device (101) or may be preset by user input. The user input for calling the above control menu may correspond to a pre-specified user interaction.

[0079] In operation 303, the processor (120) may display a control menu on the display (160). The control menu may be an item for controlling a user interface including at least one three-dimensional stereoscopic object. The control menu (e.g., the control menu (237) of FIG. 2) may include at least one of a depth movement item (e.g., the depth movement item (241) of FIG. 2), a camera view change item (e.g., the camera view change item (243) of FIG. 2), or a screen panning item (e.g., the screen panning item (245) of FIG. 2). The control menu may be displayed in a two-dimensional plane or in a three-dimensional stereoscopic manner. The processor (120) may display a pointer (or cursor) (e.g., a cross shape) for selecting a three-dimensional stereoscopic object on the display (160) together with the control menu.

[0080] In operation 305, the processor (120) may detect selection and touch of an option item of the control menu. The option item of the control menu may mean at least one of a depth movement item (241), a camera view change item (243), or a screen panning item (245). Here, screen panning may mean moving or switching the screen up, down, left, or right. The processor (120) may detect a user input of selecting (e.g., touching) at least one of the depth movement item (241), the camera view change item (243), or the screen panning item (245), and detect that the detected user input is maintained (e.g., holding the touch). For example, the user may touch the depth movement item (241) and maintain the touch.

[0081] In operation 307, the processor (120) may detect the movement of the electronic device (101). The processor (120) may detect the movement of the electronic device (101) while the user input for selecting an option item of the control menu is maintained. The movement of the electronic device (101) may be acquired (or detected) using a sensor module included in the electronic device (101) (e.g., the sensor module (176) of FIG. 1). For example, the sensor module (176) may include a 6-axis sensor combining a gyro sensor and an acceleration sensor, or a 9-axis sensor combining a gyro sensor, an acceleration sensor, or a geomagnetic sensor. The user may move the electronic device (101) while maintaining the user input for selecting the option item without releasing it.

[0082] According to one embodiment, the processor (120) may differently use (or extract) sensing data acquired by the movement of the electronic device (101) based on the selected option item. For example, when the user selects the depth movement item (241) as an option item, the processor (120) may acquire a pitch value from among the sensing data. When the user selects the camera view change item (243) as an option item, the processor (120) may acquire a pitch value or a yaw value from among the sensing data. When the user selects the screen panning item (245) as an option item, the processor (120) may acquire at least one of an x-axis movement value, a y-axis movement value, a pitch value, or a yaw value from among the sensing data.

[0083] According to one embodiment, the processor (120) may obtain all (e.g., x-axis movement value, y-axis movement value, pitch value, yaw value, roll value) sensing data acquired by the movement of the electronic device (101) from the sensing module (176), and may also selectively use the sensing data based on the selected option item. For example, if the processor (120) receives a selection of a depth movement item (241) as an option item from the user, the processor (120) may use a pitch value among the sensing data. If the processor (120) receives a selection of a camera view change item (243) as an option item from the user, the processor (120) may use a pitch value or a yaw value among the sensing data. If the processor (120) receives a selection of a screen panning item (245) as an option item from the user, the processor (120) may use at least one of an x-axis movement value, a y-axis movement value, a pitch value, or a yaw value among the sensing data.

[0084] According to one embodiment, the processor (120) may touch an option item in a control menu, and change and display a user interface based on a movement of the touch (e.g., touch drag, touch swipe) while the touch is maintained. In operation 309, the processor (120) may detect a release of the touch of the option item. For example, if a user moves the electronic device (101) while maintaining the user input for selecting the option item without releasing it, the processor (120) may obtain sensing data about the movement of the electronic device (101). If the user input for selecting the option item is released, the processor (120) may stop obtaining (or using, extracting) sensing data about the movement of the electronic device (101).

[0085] In operation 311, the processor (120) may change the user interface based on the movement of the electronic device (101). For example, the processor (120) may change and display depth (or layer) information (e.g., z-axis coordinate value) of at least one three-dimensional object displayed on the display (160) based on the movement of the electronic device (101) while the depth movement item (241) is touched and the touch is maintained. The processor (120) may change the camera shooting view based on the movement of the electronic device (101) while the camera view change item (243) is touched and the touch is maintained, and change and display the user interface displayed on the display (160) according to the changed camera view. The processor (120) may change and display the user interface up, down, left, and right based on the movement of the electronic device (101) while the screen panning item (245) is touched and the touch is maintained. The processor (120) may display an object (or screen area) that was not displayed on the display (160) or may not display an object that was displayed on the display (160) as the user moves the electronic device (101) up, down, left, or right.

[0086] According to one embodiment, the processor (120) may select at least one three-dimensional (3D) object displayed on the display (160) based on the movement of the electronic device (101), and execute a function corresponding to the selected object. When a touch is released while the three-dimensional (3D) object is included in a designated area (e.g., a pointer area) of the display (160), the processor (120) may determine that the three-dimensional (3D) object is selected. For example, when the selected three-dimensional (3D) object is a contact (e.g., a phone number), the processor (120) may attempt to call the contact. Alternatively, when the selected three-dimensional (3D) object is a photo, the processor (120) may display the photo on the entire area of ​​the display (160). When the three-dimensional (3D) object is included in the designated area of ​​the display (160), the processor (120) may guide (e.g., provide a notification) that the three-dimensional (3D) object is selectable.

[0087] FIG. 4 is a diagram illustrating an example of calling a control menu in an electronic device according to one embodiment.

[0088] Referring to FIG. 4, a processor (e.g., the processor 120 of FIG. 1) of an electronic device (e.g., the electronic device 101 of FIG. 1) according to an embodiment may detect a user input (401) for calling a control menu while displaying a first user interface (410) on a display (e.g., the display 160 of FIG. 1). The first user interface (410) may include at least one three-dimensional object (e.g., an object displayed in a circular shape). The user input (401) may include a touch input of dragging (or swiping) in a bezel area of ​​the electronic device (101) toward the display of the electronic device (101) (e.g., the display 160 of FIG. 1). For example, the user input (401) may mean a user interaction in which a touch is initiated in a bezel area corresponding to a lower left corner of the electronic device (101) and moves inward toward the display (160). Alternatively, the user input (401) may mean a user interaction in which a touch is initiated in a bezel area corresponding to at least one of the lower right corner, upper left corner, and upper right corner of the electronic device (101) and the touch moves inward toward the display (160).

[0089] The processor (120) may display a second user interface (430) on the display (160) in response to a user input (401). The second user interface (430) may include at least one three-dimensional (3D) object, a pointer (e.g., a cross shape), and a control menu (435). The control menu (435) may be displayed in a two-dimensional (2D) plane or in a three-dimensional (3D) plane. The control menu (435) may include at least one of a depth movement item (e.g., a depth movement item (241) of FIG. 2), a camera view change item (e.g., a camera view change item (243) of FIG. 2), or a screen panning item (e.g., a screen panning item (245) of FIG. 2). In addition, the control menu (435) may further include a touch area for object selection. Although the drawing illustrates the control menu as including three option items, the control menu may include fewer (e.g., two) or more option items than three.

[0090] The processor (120) may detect a second user input (455) for deleting a control menu (435) while displaying the second user interface (430). Referring to the third user interface (450), the second user input (455) may refer to a user interaction in which a touch moves from an inside corner of the display (160) to a bezel area of ​​the electronic device (101), as opposed to the user input (401) that calls up the control menu (435). When the second user input (455) is detected, the processor (120) may provide a user interface (e.g., the first user interface (410)) in which the control menu (435) is deleted. According to one embodiment, the second user input (455) may include a user interaction in which a touch is initiated from at least one of a lower right corner, an upper left corner, and an upper right corner of the display (160) and moves to a bezel area that is an outer direction of the display (160).

[0091] FIG. 5A is a diagram illustrating an example of selecting depth movement of a control menu in an electronic device according to one embodiment.

[0092] Referring to FIG. 5A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may display a first user interface (510) on a display (e.g., a display (160) of FIG. 1). The first user interface (510) may include at least one three-dimensional (3D) stereoscopic object (e.g., 503, 505) and a control menu. The first three-dimensional (3D) stereoscopic object (503) and the second three-dimensional (3D) stereoscopic object (505) may correspond to first depth information (e.g., a z-axis coordinate value, depth information). The control menu may include at least one of a depth movement item (e.g., a depth movement item (241) of FIG. 2), a camera view change item (e.g., a camera view change item (243) of FIG. 2), or a screen panning item (e.g., a screen panning item (245) of FIG. 2). The processor (120) can detect a user input (501) for selecting a depth movement item while displaying the first user interface (510). The user can move the electronic device (101) while maintaining the user input (501) (e.g., touch hold, touch maintenance). That is, the user can move the electronic device (101) forward or backward from the user, or move the electronic device (101) up, down, left, or right of the user. The processor (120) can detect the movement of the electronic device (101) in response to the user input (501).

[0093] The processor (120) can display a second user interface (520) based on the movement of the electronic device (101). The processor (120) can detect the movement of the electronic device (101) using a sensor module (e.g., the sensor module (176) of FIG. 1) included in the electronic device (101). For example, when the electronic device (101) moves while a user input (525) for selecting a depth movement item is maintained, the processor (120) can display the second user interface (520) based on a pitch value of the sensing data. The second user interface (520) can include a third three-dimensional stereoscopic object (507) and a fourth three-dimensional stereoscopic object (509) corresponding to the second depth information. According to one embodiment, the processor (120) may display a second user interface (520) based on a movement (e.g., up and down drag) of the user input (501) while maintaining (e.g., touch hold) the user input (501) for selecting a depth movement item.

[0094] For example, the first user interface (510) may be composed of at least one three-dimensional stereoscopic object corresponding to three different pieces of depth information (521, 522, 523). For example, a first three-dimensional stereoscopic object (503) and a second three-dimensional stereoscopic object (505) may correspond to the first depth information (521) (or the first layer), a third three-dimensional stereoscopic object (507) and a fourth three-dimensional stereoscopic object (509) may correspond to the second depth information (522) (or the second layer), and at least one fifth three-dimensional stereoscopic object may correspond to the third depth information (523) (or the third layer). The first depth information (521) may be positioned at a higher depth (e.g., closer to the user) than the second depth information (522), and the second depth information (522) may be positioned at a higher depth (e.g., closer to the user) than the third depth information (523).

[0095] The first user interface (510) may be a three-dimensional stereoscopic object arranged based on the first depth information (521), and the second user interface (520) may be a three-dimensional stereoscopic object arranged based on the second depth information (522). When the electronic device (101) is tilted backwards from the user (e.g., the electronic device (101) moves away from the user) while the user input (525) for selecting a depth movement item is maintained while the first user interface (510) is displayed, the processor (120) may display the second user interface (520) based on the pitch value of the sensing data. Alternatively, the processor (120) may display the first user interface (510) based on the pitch value of the sensing data when the electronic device (101) is tilted toward the user (e.g., the electronic device (101) moves closer to the user) while the user input (525) of selecting a depth movement item is maintained while displaying the second user interface (520).

[0096] When the processor (120) releases (535) a user input that touches a depth movement item while displaying the second user interface (520), the processor (120) may display a third user interface (530). The third user interface (530) may be a user interface that fixes the screen to place a three-dimensional object based on the second depth information (522). For example, the third user interface (530) may be a user interface that moves a three-dimensional object (533) to a designated area (531) and selects the three-dimensional object (533).

[0097] According to one embodiment, when a user input of double-tapping a depth movement item is detected while the second user interface (520) is displayed, the processor (120) may display the first user interface (510). That is, the processor (120) may change and display the user interface based on the movement of the electronic device (101) (e.g., the second user interface (520)), and when a user input of double-tapping a depth movement item is detected, the processor (120) may initialize a change value related to the user interface change and return to the initial screen (e.g., the first user interface (510)).

[0098] FIG. 5b is a diagram illustrating an example of selecting a camera view change in a control menu in an electronic device according to one embodiment.

[0099] Referring to FIG. 5B, the processor (120) may display a fourth user interface (540) on the display (160). While displaying the fourth user interface (540), the processor (120) may release a user input for selecting a depth movement item (541) and receive a user input for selecting a camera view movement item (543). The fourth user interface (540) may include at least one three-dimensional stereoscopic object and a control menu. The fifth user interface (550) illustrates an example in which the camera view movement item (543) of the control menu is selected.

[0100] The user can move the electronic device (101) while maintaining the input of touching the camera view movement item (555). For example, when the electronic device (101) moves while the user input of selecting the camera view movement item is maintained (565), the processor (120) can display the sixth user interface (560) based on the pitch value or yaw value of the sensing data. The processor (120) can change the user interface displayed on the display (160) according to the change in the camera (562) view. The display value of the three-dimensional stereoscopic object (561) included in the user interface can be changed according to the camera (562) view. The sixth user interface (560) may be one in which the camera view is changed compared to the fifth user interface (550).

[0101] FIG. 5c is a diagram illustrating an example of selecting screen movement of a control menu in an electronic device according to one embodiment.

[0102] Referring to FIG. 5C, the processor (120) may display a seventh user interface (570) on the display (160). While displaying the seventh user interface (570), the processor (120) may release a user input for selecting a depth movement item (571) and receive a user input for selecting a screen panning item (573). The seventh user interface (570) may include at least one three-dimensional stereoscopic object and a control menu. The eighth user interface (580) illustrates an example in which the screen panning item (573) of the control menu is selected.

[0103] The user can move the electronic device (101) while maintaining an input for touching the screen panning item (583). For example, when the electronic device (101) moves while maintaining a user input for selecting the screen panning item (583), the processor (120) can display the ninth user interface (590) based on at least one of the x-axis movement value, the y-axis movement value, the pitch value, or the yaw value of the sensing data. The processor (120) can change the user interface displayed on the display (160) according to the screen movement. When the user moves the electronic device (101) up, down, left, or right, the processor (120) can display an object (or area) that was not displayed on the display (160), or not display an object that was displayed on the display (160). The ninth user interface (590) may, compared to the eighth user interface (580), remove some objects (593) displayed on the screen and display an area (591) that is not displayed on the screen.

[0104] FIG. 6 is a flowchart (600) illustrating a method for controlling a user interface through a control menu of an electronic device according to one embodiment. FIG. 6 may be an embodiment of operations 305 to 311 of FIG. 3.

[0105] Referring to FIG. 6, in operation 601, a processor (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an embodiment may select an option item of a control menu. The control menu may be an item for controlling a user interface including at least one three-dimensional object. The control menu (e.g., the control menu (237) of FIG. 2) may include at least one of a depth movement item (e.g., the depth movement item (241) of FIG. 2), a camera view change item (e.g., the camera view change item (243) of FIG. 2), or a screen panning item (e.g., the screen panning item (245) of FIG. 2). The control menu may be displayed in a two-dimensional plane or in a three-dimensional stereoscopic manner. The processor (120) may display a pointer (or cursor) at the center of a display (e.g., the display (160) of FIG. 1)) together with the control menu. The user can select at least one of the depth movement items, the camera view change item, or the screen panning item to select a desired 3D stereoscopic object.

[0106] Hereinafter, although it is described that operation 603 of selecting a depth movement item (241) is performed first, and operation 615 of selecting a camera view change item (243) or operation 625 of selecting a screen panning item (245) is performed later, operation 615 or operation 625 may be performed simultaneously (or in parallel) with operation 603. Alternatively, operation 603, operation 615, or operation 625 may be performed regardless of the order, or operation 615 may be performed before operation 603, or operation 625 may be performed before operation 603. The drawings are provided only to help understanding of the invention, and the invention is not limited by the drawings.

[0107] In operation 603, the processor (120) can determine whether the selected option item is a depth movement item. If the selected option item is a depth movement item, the processor (120) can perform operation 605, and if the selected option item is not a depth movement item, the processor (120) can perform operation 615.

[0108] If the selected option item is a depth movement item, in operation 605, the processor (120) may detect a touch hold. The processor (120) may detect that the depth movement item is touched by the user and the touch is maintained. According to one embodiment, the user may touch the depth movement item and release the touch. If the user does not hold the touch on the depth movement item and releases the touch, the processor (120) may determine whether another option item of the control menu is selected (e.g., touched). Hereinafter, a situation in which the user input for selecting the depth movement item is not released but maintained will be described as an example.

[0109] In operation 607, the processor (120) may acquire (or identify) first z-axis sensing information. The processor (120) may acquire first z-axis sensing information while maintaining a touch on the depth movement item. The first z-axis sensing information may include a pitch value.

[0110] In operation 609, the processor (120) can determine whether the touch on the depth movement item is released. The user can move the electronic device (101) so that a three-dimensional object corresponds to the pointer while maintaining the touch on the depth movement item. If the touch on the depth movement item is released, the processor (120) can perform operation 611, and if the touch on the depth movement item is not released, the processor (120) can perform operation 613.

[0111] When the touch for the depth movement item is released, in operation 611, the processor (120) may select an object (e.g., a 3D stereoscopic object) corresponding to the pointer (e.g., arranged side by side in the x, y, and z-axis directions). Alternatively, when the touch for the camera view change item is released, the processor (120) may select an object corresponding to the pointer. When the touch for the screen panning item is released, the processor (120) may select an object corresponding to the pointer in operation 631. When the pointer is located at the center of the 3D stereoscopic object, the processor (120) may determine that the 3D stereoscopic object has been selected. The processor (120) may execute a function corresponding to the selected 3D stereoscopic object.

[0112] According to one embodiment, if a three-dimensional stereoscopic object does not correspond to the pointer, the processor (120) may select at least one three-dimensional stereoscopic object adjacent to the pointer. For example, the processor (120) may control the three-dimensional stereoscopic object having the closest x, y distance to the pointer to be aligned with the pointer in the x, y, and z-axis directions.

[0113] If the touch for the depth movement item is not released, in operation 613, the processor (120) may change and display the depth value of the user interface (UI) based on the first z-axis sensing information. If the electronic device (101) moves while the selection of the depth movement item is maintained (e.g., touch hold), the processor (120) may change and display the depth value of the user interface (UI) based on the first z-axis sensing information. After performing operation 613, the processor (120) returns to operation 607 to continuously acquire the first z-axis sensing information, and may change and display the user interface based on the acquired first z-axis sensing information until the touch is released.

[0114] If the selected option item is not a depth movement item, in operation 615, the processor (120) can determine whether the selected option item is a camera view change item. If the selected option item is a camera view change item, the processor (120) can perform operation 617, and if the selected option item is not a camera view change item, the processor (120) can perform operation 625.

[0115] If the selected option item is a camera view change item, in operation 617, the processor (120) may detect a touch hold. The processor (120) may detect that the camera view change item is touched by a user and the touch is maintained. According to one embodiment, the user may touch the camera view change item and release the touch. If the user releases the touch without maintaining the touch on the camera view change item, the processor (120) may determine whether another option item of the control menu is selected (e.g., touched). Hereinafter, a situation in which a user input for selecting a camera view change item is maintained without being released will be described as an example.

[0116] In operation 619, the processor (120) may acquire (or identify) second z-axis sensing information. The processor (120) may acquire second z-axis sensing information while maintaining a touch on the camera view change item. The second z-axis sensing information may include a pitch value or a yaw value.

[0117] In operation 621, the processor (120) can determine whether the touch on the camera view change item is released. The user can move the electronic device (101) so that a three-dimensional object corresponds to the pointer while maintaining the touch on the camera view change item. If the touch on the camera view change item is released, the processor (120) can perform operation 611, and if the touch on the camera view change item is not released, the processor (120) can perform operation 623.

[0118] If the touch for the above camera view change item is not released, in operation 623, the processor (120) may change and display the camera view based on the z-axis second sensing information. If the electronic device (101) moves while the selection of the camera view change item is maintained (e.g., touch hold), the processor (120) may change the camera view based on the z-axis second sensing information, and change and display the user interface based on the changed camera view. After performing operation 623, the processor (120) may return to operation 619 to continuously acquire the z-axis second sensing information, and change and display the user interface based on the acquired z-axis second sensing information until the touch is released.

[0119] If the selected option item is not a camera view change item, in operation 625, the processor (120) can identify that the screen panning item is selected as the selected option item.

[0120] If the selected option item is a screen panning item, in operation 627, the processor (120) may detect a touch hold. The processor (120) may detect that the screen panning item is touched by the user and the touch is maintained. According to one embodiment, the user may touch the screen panning item and release the touch. If the user releases the touch without maintaining the touch on the screen panning item, the processor (120) may determine whether another option item of the control menu is selected (e.g., touched). Hereinafter, a situation in which a user input for selecting the screen panning item is maintained without being released will be described as an example.

[0121] In operation 629, the processor (120) may obtain (or identify) x-, y-, and z-axis sensing information. The processor (120) may obtain the x-, y-, and z-axis sensing information while maintaining a touch on the screen panning item. The x-, y-, and z-axis sensing information may include at least one of an x-axis movement value, a y-axis movement value, a pitch value, or a yaw value.

[0122] In operation 631, the processor (120) can determine whether the touch on the screen panning item is released. The user can move the electronic device (101) so that a three-dimensional object corresponds to the pointer while maintaining the touch on the screen panning item. If the touch on the screen panning item is released, the processor (120) can perform operation 611, and if the touch on the screen panning item is not released, the processor (120) can perform operation 633.

[0123] If the touch for the above screen panning item is not released, in operation 633, the processor (120) can change and display the screen view based on the x, y, and z-axis sensing information. If the electronic device (101) moves while the selection of the above screen panning item is maintained (e.g., touch hold), the processor (120) can change and display the user interface based on the x, y, and z-axis sensing information. After performing operation 633, the processor (120) returns to operation 629 to continuously acquire the x, y, and z-axis sensing information, and can change and display the user interface based on the acquired x, y, and z-axis sensing information until the touch is released.

[0124] FIG. 7 is a diagram illustrating an example of changing a user interface by selecting a control menu in an electronic device according to one embodiment.

[0125] Referring to FIG. 7, a processor (e.g., the processor 120 of FIG. 1) of an electronic device (e.g., the electronic device 101 of FIG. 1) according to an embodiment may display a first user interface (710) on a display (e.g., the display 160 of FIG. 1). The first user interface (710) may include a plurality of three-dimensional stereoscopic objects, a pointer, and a control menu. While displaying the first user interface (710), the processor (120) may select at least one option item of the control menu based on a user input. The user may select at least one of a depth movement item, a camera view change item, or a screen panning item to select a desired three-dimensional stereoscopic object.

[0126] For example, when a camera view change item (e.g., a camera view change item (243) of FIG. 2) is selected in the first user interface (710), the processor (120) may display a second user interface (730). When the processor (120) touches the camera view change item (243) and detects movement of the electronic device (101) while maintaining the touch, the processor (120) may change the camera view based on a pitch value or a yaw value acquired from a sensor module (e.g., a sensor module (176) of FIG. 1) of the electronic device (101), and may change the user interface in response to the changed camera view.

[0127] When a depth movement item (e.g., depth movement item (241) of FIG. 2) is selected in the second user interface (730), the processor (120) may display a third user interface (750). The user interface corresponding to the changed camera view may be the third user interface (750). The user may select the camera view change item (243) and the depth movement item (241) in combination to select a desired three-dimensional object. When the processor (120) touches the depth movement item (241) and maintains the touch, and movement of the electronic device (101) is detected, the processor (120) may change the user interface based on the pitch value acquired from the sensor module (176).

[0128] When a screen panning item (e.g., a screen panning item (245) of FIG. 2) is selected in the third user interface (750), the processor (120) may display a fourth user interface (770). The changed user interface in the third user interface (750) may be the fourth user interface (770). In order to select a desired three-dimensional stereoscopic object, the user may compositely select the camera view change item (243), the depth movement item (241), and the screen panning item (245). When the processor (120) touches the screen panning item (245) and detects movement of the electronic device (101) while maintaining the touch, the processor (120) may change the user interface based on at least one of an x-axis movement value, a y-axis movement value, a pitch value, or a yaw value acquired from the sensing module (176).

[0129] Although FIG. 7 illustrates sequential selection of the camera view change item (243), depth movement item (241), and screen panning item (245), the depth movement item (241) may be selected and then the screen panning item (245) may be selected, or the camera view change item (243) may be selected and then the screen panning item (245) may be selected. The example of FIG. 7 illustrates an example of a user selecting a combination of option items of a control menu to select a desired three-dimensional stereoscopic object, and the present invention is not limited by the drawings and descriptions.

[0130] FIGS. 8A to 8C are drawings illustrating an example of selecting a three-dimensional object through a control menu in an electronic device according to one embodiment.

[0131] Referring to FIG. 8A, a processor (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an embodiment may detect a user input for a designated touch area (811) of a control menu while displaying a first user interface (810) on a display (e.g., the display (160) of FIG. 1). The first user interface (810) may include a pointer (801), a first three-dimensional stereoscopic object (805), or a second three-dimensional stereoscopic object (807). The first three-dimensional stereoscopic object (805) or the second three-dimensional stereoscopic object (807) may include an icon of an application included in a home screen or content (e.g., text, an image, a video) included in an execution screen of an application (e.g., a contact application, a gallery application).

[0132] The designated touch area (811) of the control menu may be an area excluding an area corresponding to an option item of the control menu. The highlighted area (803) is an area corresponding to the pointer (801), and the user can move the electronic device (101) so that a three-dimensional object corresponds to the highlighted area (803).

[0133] For example, the processor (120) can detect the movement of the electronic device (101) while maintaining a touch on a designated touch area (811) in the first user interface (810). The processor (120) can display a second user interface (820) based on the movement (e.g., z-direction movement) of the electronic device (101). The second user interface (820) may be a first three-dimensional stereoscopic object (805) or a second three-dimensional stereoscopic object (807) in which the z-axis position is changed. For example, the second user interface (820) may illustrate an example in which the second three-dimensional stereoscopic object (807) is included in the highlight area (803). The processor (120) can detect the movement of the electronic device (101) while maintaining a touch on a designated touch area (821) while displaying the second user interface (820).

[0134] If the second three-dimensional stereoscopic object (807) is included in the highlighted area (803), the user can release the touch on the designated touch area (831). If the touch on the designated touch area (831) is released, the processor (120) can display the third user interface (830). If the second three-dimensional stereoscopic object (807) is included in the highlighted area (803) like the third user interface (830), the processor (120) determines that the second three-dimensional stereoscopic object (807) has been selected and can perform a function corresponding to the second three-dimensional stereoscopic object (807). For example, if the second three-dimensional stereoscopic object (807) is a message of a message application, the processor (120) can display the selected message on the entire area of ​​the display (160).

[0135] Alternatively, as in the fourth user interface (840), the user may move the electronic device (101) so that the second three-dimensional stereoscopic object (807) corresponds to the designated touch area (841) of the control menu. When the second three-dimensional stereoscopic object (807) corresponds to the designated touch area (841) of the control menu, the processor (120) determines that the second three-dimensional stereoscopic object (807) has been selected and may perform a function corresponding to the second three-dimensional stereoscopic object (807).

[0136] Referring to FIG. 8B, the processor (120) may detect a user input for a designated touch area (851) of a control menu while displaying a fifth user interface (850) on the display (160). The fifth user interface (850) may include a pointer (801), a first three-dimensional stereoscopic object (805), or a second three-dimensional stereoscopic object (807). The first three-dimensional stereoscopic object (805) or the second three-dimensional stereoscopic object (807) may include an icon of an application included in a home screen or content (e.g., text, image, video) included in an execution screen of an application (e.g., a contact application, a gallery application). The highlight area (803) is an area corresponding to the pointer (801), and the user may move the electronic device (101) so that the three-dimensional stereoscopic object corresponds to the highlight area (803).

[0137] For example, the processor (120) can detect the movement of the electronic device (101) while maintaining a touch on a designated touch area (851) in the fifth user interface (850). The processor (120) can display a sixth user interface (860) based on the movement (e.g., z-direction movement) of the electronic device (101). For example, the sixth user interface (860) can illustrate an example in which a second three-dimensional stereoscopic object (807) corresponding to a first layer (e.g., first depth information) in the z-axis direction is automatically included in the highlight area (803). The processor (120) can select a second three-dimensional stereoscopic object (807) adjacent to the pointer (801) (e.g., closest x, y distance) while maintaining a touch on the designated touch area (861), and move the second three-dimensional stereoscopic object (807) to the highlight area (803).

[0138] In the fifth user interface (850), since the second three-dimensional stereoscopic object (807) is closer to the pointer (801) than the first three-dimensional stereoscopic object (805), the processor (120) can select the second three-dimensional stereoscopic object (807). When the touch on the designated touch area (871) is released, the processor (120) can display the seventh user interface (870). When the second three-dimensional stereoscopic object (807) is included in the highlight area (803) as in the seventh user interface (870), the processor (120) determines that the second three-dimensional stereoscopic object (807) has been selected and can perform a function corresponding to the second three-dimensional stereoscopic object (807).

[0139] Referring to FIG. 8C, the processor (120) can detect a user input for a designated touch area (881) of a control menu while displaying an eighth user interface (880) on the display (160). The eighth user interface (880) can include a pointer (801), a first three-dimensional stereoscopic object (805), or a second three-dimensional stereoscopic object (807). The highlighted area (803) is an area corresponding to the pointer (801), and the user can move the electronic device (101) so that the three-dimensional stereoscopic object corresponds to the highlighted area (803).

[0140] For example, the processor (120) can detect the movement of the electronic device (101) while maintaining a touch on a designated touch area (881) in the eighth user interface (880). The processor (120) can display a ninth user interface (890) based on the movement (e.g., z-direction movement) of the electronic device (101). For example, the ninth user interface (890) can illustrate an example in which a first three-dimensional stereoscopic object (805) corresponding to a second layer (e.g., second depth information) in the z-axis direction is automatically included in a highlight area (803). The processor (120) can select a first three-dimensional stereoscopic object (805) adjacent to the pointer (801) (e.g., closest x, y distance) while maintaining a touch on the designated touch area (861), and move the first three-dimensional stereoscopic object (805) to the highlight area (803).

[0141] In the eighth user interface (880), since the first three-dimensional stereoscopic object (805) is closer to the pointer (801) than the second three-dimensional stereoscopic object (807), the processor (120) can select the first three-dimensional stereoscopic object (805). When the touch on the designated touch area (891) is released, the processor (120) can display the tenth user interface (895). When the first three-dimensional stereoscopic object (805) is included in the highlight area (803) as in the tenth user interface (895), the processor (120) determines that the first three-dimensional stereoscopic object (805) has been selected and can perform a function corresponding to the first three-dimensional stereoscopic object (805).

[0142] FIGS. 9A and 9B are drawings illustrating an example of selecting a three-dimensional stereoscopic object through a control menu in an electronic device according to one embodiment.

[0143] Referring to FIG. 9A, a processor (e.g., the processor 120 of FIG. 1) of an electronic device (e.g., the electronic device 101 of FIG. 1) according to an embodiment may divide an area corresponding to a pointer (e.g., the pointer 801 of FIGS. 8A to 8C) into a first highlight area (901) and a second highlight area (903), such as a first user interface (910). For example, the processor (120) may move a three-dimensional (3D) object (921) to the first highlight area (901) or the second highlight area (903) based on the movement of the electronic device (101), such as a second user interface (920). The processor (120) may snap a three-dimensional (3D) object (931) to the first highlight area (901), such as a third user interface (930). The above snapping may mean moving the three-dimensional stereoscopic object (931) into the first highlight area (901). The processor (120), such as the fourth user interface (940), may include the three-dimensional stereoscopic object (941) in the first highlight area (901) and the second highlight area (903) based on the movement of the electronic device (101). When the three-dimensional stereoscopic object (941) is included in the first highlight area (901) and the second highlight area (903), the processor (120) may execute a function corresponding to the three-dimensional stereoscopic object (941).

[0144] Referring to FIG. 9B, when a recently executed list is selected while the processor (120) displays the fifth user interface (950), the processor (120) may display the sixth user interface (960). The sixth user interface (960) may display the execution screen (961) of the first application, the execution screen (963) of the second application, and the execution screen (965) of the third application as three-dimensional stereoscopic objects. The processor (120) may detect the movement of the electronic device (101) while a designated touch area of ​​the control menu is selected while the sixth user interface (960) is displayed. The processor (120) may select the three-dimensional stereoscopic object based on the movement (e.g., z direction) of the electronic device (101). For example, the processor (120) can select by moving the execution screen (961) of the first application to the range z+ to z-, or can select by moving the execution screen (963) of the second application and the execution screen (965) of the third application to the range z0 to z-.

[0145] For example, when the electronic device (101) moves left and right (970), the processor (120) can snap the execution screen (961) of the first application to the z+ highlight area (e.g., the first highlight area (901) of FIG. 9A). The snapping may mean moving the execution screen (961) of the first application to come into the z+ highlight area. In addition, when the electronic device (101) moves in the z-axis direction (980), the processor (120) can include the execution screen (961) of the first application in the first highlight area (901) and the second highlight area (903). When the execution screen (961) of the first application is included in the first highlight area (901) and the second highlight area (903), the processor (120) can display the execution screen (961) of the first application on the entire area of ​​the display (160).

[0146] FIGS. 10A to 10C are drawings illustrating an example of selecting a three-dimensional object through a control menu in an electronic device according to one embodiment.

[0147] Referring to FIG. 10A, a processor (e.g., a processor (120) of FIG. 1) of an electronic device (e.g., an electronic device (101) of FIG. 1) according to an embodiment may detect a user input for a designated touch area (1011) of a control menu while displaying a first user interface (1010) on a display (e.g., a display (160) of FIG. 1). The first user interface (1010) may include a pointer (1001), a first three-dimensional stereoscopic object (1005), or a second three-dimensional stereoscopic object (1007). The first three-dimensional stereoscopic object (1005) or the second three-dimensional stereoscopic object (1007) may include an icon of an application included in a home screen or content included in an execution screen of the application. The highlight area (1003) is an area corresponding to the pointer (1001), and the user can move the electronic device (101) so that a three-dimensional object corresponds to the highlight area (1003).

[0148] For example, the processor (120) can detect movement of the electronic device (101) while maintaining a touch on a designated touch area (1011) in the first user interface (1010). The processor (120) can display a second user interface (1020) based on the movement (e.g., z-direction movement) of the electronic device (101). For example, the second user interface (1020) can illustrate an example in which a second three-dimensional stereoscopic object (1007) corresponding to a first layer (e.g., first depth information) in the z-axis direction is automatically included in a highlight area (1003). The processor (120) can select a second three-dimensional stereoscopic object (1007) adjacent to the pointer (1001) (e.g., closest x, y distance) while maintaining a touch on the designated touch area (1021), and move the second three-dimensional stereoscopic object (1007) to the highlight area (1003). The processor (120) can also move the position of the first three-dimensional stereoscopic object (1005) based on the position movement of the second three-dimensional stereoscopic object (1007).

[0149] In the first user interface (1010), since the second three-dimensional stereoscopic object (1007) is closer to the pointer (1001) than the first three-dimensional stereoscopic object (1005), the processor (120) can select the second three-dimensional stereoscopic object (1007). When the touch on the designated touch area (1031) is released, the processor (120) can display the third user interface (1030). When the second three-dimensional stereoscopic object (1007) is included in the highlight area (1003) like the third user interface (1030), the processor (120) determines that the second three-dimensional stereoscopic object (1007) has been selected and can perform a function corresponding to the second three-dimensional stereoscopic object (1007).

[0150] Referring to FIG. 10B, the processor (120) may detect a user input for a designated touch area (1041) of a control menu while displaying a fourth user interface (1040) on the display (160). The fourth user interface (1040) may include a pointer (1001), a first three-dimensional stereoscopic object (1005), or a second three-dimensional stereoscopic object (1007). The first three-dimensional stereoscopic object (1005) or the second three-dimensional stereoscopic object (1007) may include an icon of an application included in a home screen or content included in an execution screen of the application. The highlight area (1003) is an area corresponding to the pointer (1001), and the user may move the electronic device (101) so that the three-dimensional stereoscopic object corresponds to the highlight area (1003).

[0151] For example, the processor (120) can detect the movement of the electronic device (101) while maintaining a touch on a designated touch area (1041) in the fourth user interface (1040). The processor (120) can display a fifth user interface (1050) based on the movement (e.g., z-direction movement) of the electronic device (101). For example, the fifth user interface (1050) can illustrate an example in which a first three-dimensional stereoscopic object (1005) corresponding to a second layer (e.g., second depth information) in the z-axis direction is automatically included in a highlight area (1003). The processor (120) can select a first three-dimensional stereoscopic object (1005) adjacent to the pointer (1001) (e.g., closest x, y distance) while maintaining a touch on the designated touch area (1051), and move the first three-dimensional stereoscopic object (1005) to the highlight area (1003). The processor (120) can also move the position of the second three-dimensional stereoscopic object (1007) based on the position movement of the first three-dimensional stereoscopic object (1005).

[0152] In the fourth user interface (1040), since the first three-dimensional stereoscopic object (1005) is closer to the pointer (1001) than the second three-dimensional stereoscopic object (1007), the processor (120) can select the first three-dimensional stereoscopic object (1005). When the touch on the designated touch area (1061) is released, the processor (120) can display the sixth user interface (1060). When the first three-dimensional stereoscopic object (1005) is included in the highlight area (1003) as in the sixth user interface (1060), the processor (120) determines that the first three-dimensional stereoscopic object (1005) has been selected and can perform a function corresponding to the first three-dimensional stereoscopic object (1005).

[0153] Referring to FIG. 10c, the processor (120) may detect a user input for a designated touch area (1071) of a control menu while displaying a seventh user interface (1070) on the display (160). The seventh user interface (1070) may illustrate an example of displaying a three-dimensional stereoscopic object (1073) adjacent to a pointer differently from the remaining three-dimensional stereoscopic objects. Like the seventh user interface (1070), the processor (120) may blur (or shade) the remaining three-dimensional stereoscopic objects except for the three-dimensional stereoscopic object (1073) adjacent to the pointer.

[0154] The processor (120) can detect the movement of the electronic device (101) while maintaining a touch on the touch area (1071) designated in the seventh user interface (1070). The processor (120) can display the eighth user interface (1080) based on the movement (e.g., movement in the +z direction) of the electronic device (101). For example, the eighth user interface (1080) may illustrate an example in which a three-dimensional stereoscopic object (1083) corresponding to a second layer (e.g., second depth information) in the +z direction is automatically included in a highlight area. The processor (120) can detect the movement of the electronic device (101) while maintaining a touch on the touch area (1081) designated in the eighth user interface (1080). The processor (120) can display the ninth user interface (1090) based on the movement (e.g., movement in the -z direction) of the electronic device (101). For example, the 9th user interface (1090) may be an example in which a 3D stereoscopic object (1093) corresponding to a first layer (e.g., first depth information) in the z-axis direction is automatically included in a highlight area. When the touch on the touch area (1091) designated in the 9th user interface (1090) is released, the processor (120) may display the 10th user interface (1095). When the 3D stereoscopic object (1097) is included in the highlight area, as in the 10th user interface (1095), the processor (120) may determine that the 3D stereoscopic object (1097) is selected and may perform a function corresponding to the 3D stereoscopic object (1097).

[0155] FIG. 11 is a flowchart (1100) illustrating a method for selecting a three-dimensional stereoscopic object through a control menu of an electronic device according to one embodiment.

[0156] Referring to FIG. 11, in operation 1101, a processor (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an embodiment may display a pointer and a control menu. The pointer (or cursor) may be for selecting a three-dimensional (3D) object displayed on a display (e.g., the display (160) of FIG. 1). The control menu may be an item for controlling a user interface including at least one three-dimensional (3D) object. The control menu (e.g., the control menu (237) of FIG. 2) may include at least one of a depth movement item (e.g., the depth movement item (241) of FIG. 2), a camera view change item (e.g., the camera view change item (243) of FIG. 2), or a screen panning item (e.g., the screen panning item (245) of FIG. 2). The control menu may be displayed in a two-dimensional plane or in a three-dimensional (3D) stereoscopic manner. The above control menu may further include a designated area (e.g., a designated touch area (1071) of FIG. 10c).

[0157] In operation 1103, the processor (120) can maintain a touch on a designated area of ​​the control menu. The user can move the electronic device (101) while touching the designated area of ​​the control menu to select a desired three-dimensional object.

[0158] In operation 1105, the processor (120) can detect the movement of the electronic device (101). The movement of the electronic device (101) can be acquired (or detected) using a sensor module (e.g., the sensor module (176) of FIG. 1) included in the electronic device (101). For example, the sensor module (176) can include a 6-axis sensor combining a gyro sensor and an acceleration sensor, or a 9-axis sensor combining a gyro sensor, an acceleration sensor, or a geomagnetic sensor. The user can move the electronic device (101) while maintaining the user input of touching a designated area of ​​the control menu without releasing it.

[0159] In operation 1107, the processor (120) may determine whether the pointer and the object are aligned. The alignment may mean aligning the pointer and the object side by side. The object may mean a three-dimensional object. The user may align the pointer and the three-dimensional object in the x, y, and z-axis directions to select the three-dimensional object. If the pointer and the object are aligned, the processor (120) may perform operation 1109, and if the pointer and the object are not aligned, the processor (120) may perform operation 1105. If the pointer and the object are not aligned, the processor (120) may return to operation 1105 to detect movement of the electronic device (101).

[0160] When the pointer and the object are aligned, in operation 1109, the processor (120) can select the object. When the pointer is located at the center of the three-dimensional stereoscopic object, the processor (120) can determine that the three-dimensional stereoscopic object has been selected. Alternatively, the processor (120) can determine that the three-dimensional stereoscopic object has been selected when the three-dimensional stereoscopic object is included in a highlight area corresponding to the pointer (e.g., a highlight area (803) of FIGS. 8A to 8C).

[0161] In operation 1111, the processor (120) may execute a function corresponding to the selected three-dimensional stereoscopic object. For example, if the selected three-dimensional stereoscopic object is a contact (e.g., a phone number), the processor (120) may attempt to call the contact. Alternatively, if the selected three-dimensional stereoscopic object is a photograph, the processor (120) may display the photograph in the entire area of ​​the display (160). If a three-dimensional stereoscopic object is included in a designated area of ​​the display (160), the processor (120) may provide guidance (e.g., a notification) that the three-dimensional stereoscopic object is selectable.

[0162] FIG. 12A is a diagram illustrating an example of selecting depth movement in a control menu in a VR mode of an electronic device according to one embodiment.

[0163] Referring to FIG. 12A, a processor (e.g., the processor 120 of FIG. 1) of an electronic device (e.g., the electronic device 101 of FIG. 1) according to an embodiment may display a first user interface (1210) on a display (e.g., the display 160 of FIG. 1) in a virtual space mode. The first user interface (1010) may include at least one of a three-dimensional object (e.g., an avatar), a pointer, or a control menu. The control menu may include at least one of a depth movement item (e.g., the depth movement item 241 of FIG. 2), a camera view change item (e.g., the camera view change item 243 of FIG. 2), or a screen panning item (e.g., the screen panning item 245 of FIG. 2).

[0164] The processor (120) can detect a user input (1021) for a depth movement item (241) while displaying a first user interface (1210). If the user input (1021) for the depth movement item (241) is maintained, the processor (120) can detect movement of the electronic device (101). That is, the user can move the electronic device (101) forward or backward from the user, or move the electronic device (101) up, down, left, or right of the user. The processor (120) can display a second user interface (1220) based on the movement of the electronic device (101).

[0165] The processor (120) can display various user interfaces (1230) according to the movement of the electronic device (101) while maintaining a touch on the depth movement item (241). The various user interfaces (1230) can include a first screen (1231) corresponding to movement to the left of the electronic device (101), a second screen (1233) corresponding to movement backward of the electronic device (101) (e.g., moving closer to the user), a third screen (1235) corresponding to movement to the right of the electronic device (101), and a fourth screen (1237) corresponding to movement forward of the electronic device (101) (e.g., moving away from the user).

[0166] FIG. 12b is a diagram illustrating an example of selecting a camera view change in a control menu in a VR mode of an electronic device according to one embodiment.

[0167] Referring to FIG. 12B, the processor (120) may display a fourth user interface (1240) on the display (160) in a virtual space mode. The fourth user interface (1040) may include at least one three-dimensional object (e.g., an avatar), a pointer, or a control menu. The processor (120) may detect a user input (1041) for a camera view change item (243) while displaying the fourth user interface (1240). If the user input (1041) for the camera view change item (243) is maintained, the processor (120) may detect a movement of the electronic device (101). That is, the user may move the electronic device (101) forward or backward from the user, or move the electronic device (101) up, down, left, or right of the user. The processor (120) may display a fifth user interface (1250) based on the movement of the electronic device (101). The processor (120) can change the camera view based on the movement of the electronic device (101) and display the fifth user interface (1250) in response to the changed camera view.

[0168] FIG. 12c is a diagram illustrating an example of selecting screen movement of a control menu in a VR mode of an electronic device according to one embodiment.

[0169] Referring to FIG. 12C, the processor (120) may display a seventh user interface (1270) on the display (160) in a virtual space mode. The seventh user interface (1070) may include at least one three-dimensional object (e.g., an avatar), a pointer, or a control menu. The processor (120) may detect a user input (1271) for a screen panning item (245) while displaying the seventh user interface (1270). When the user input (1271) for the screen panning item (245) is maintained, the processor (120) may detect a movement of the electronic device (101). That is, the user may move the electronic device (101) forward or backward from the user, or move the electronic device (101) up, down, left, or right of the user. The processor (120) may display an eighth user interface (1280) based on the movement of the electronic device (101). The processor (120) can change the user interface displayed on the display (160) based on the movement of the electronic device (101). The processor (120) can display the eighth user interface (1280) based on the movement of the electronic device (101) in the full screen configuration area (1290).

[0170] A method of operating an electronic device (101) according to an embodiment of the present disclosure may include an operation of detecting a user input for calling a control menu while displaying a three-dimensional stereoscopic object on a display (160) of the electronic device, an operation of displaying a control menu on the display in response to the detected user input, an operation of detecting a touch for selecting an option item of the control menu, an operation of detecting a movement of the electronic device using a sensor module (176) of the electronic device while maintaining the touch on the selected option item, and an operation of changing a user interface displayed on the display based on the detected movement of the electronic device when the touch on the selected option item is released.

[0171] The action of displaying the control menu may include an action of displaying the control menu in a corner edge area of ​​the display in response to the detected user input and displaying a pointer in the center of the display.

[0172] The method may further include an operation of acquiring first sensing data in the depth direction from the sensor module when depth movement is selected as an option item of the control menu, and an operation of changing depth information of a three-dimensional stereoscopic object displayed on the display according to the release of the touch based on the acquired first sensing data.

[0173] The method may further include, when a change of camera view is selected as an option item of the control menu, an operation of acquiring second sensing data in the depth direction from the sensor module, an operation of changing the camera view based on the acquired second sensing data, and an operation of changing the user interface based on the changed camera view.

[0174] The method may further include an operation of acquiring third sensing data from the sensor module when screen panning is selected as an option item of the control menu, and an operation of changing the user interface based on the acquired third sensing data.

[0175] The above method may further include an operation of selecting a three-dimensional stereoscopic object when the three-dimensional stereoscopic object corresponds to the pointer based on the acquired sensing data.

[0176] The method may further include an action of executing a function corresponding to the selected three-dimensional stereoscopic object when the touch for the selected option item is released.

[0177] The method may further include an operation of selecting a three-dimensional stereoscopic object located adjacent to a highlighted area corresponding to the pointer based on the acquired sensing data, and an operation of executing a function corresponding to the selected three-dimensional stereoscopic object when the touch release is detected.

[0178] The method may further include an operation of moving the three-dimensional stereoscopic object placed adjacent to the highlighted area to the highlighted area, and an operation of selecting the three-dimensional stereoscopic object when the three-dimensional stereoscopic object is aligned with the highlighted area.

[0179] The method may further include an operation of moving a first three-dimensional stereoscopic object positioned adjacent to the highlighted area to the highlighted area, and an operation of moving a position of a second three-dimensional stereoscopic object displayed on the display based on the movement of the first three-dimensional stereoscopic object.

[0180] The various embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to facilitate easy explanation of the technical content of the present invention and aid understanding thereof, and are not intended to limit the scope of the present invention. Therefore, the scope of the present invention should be interpreted to include all modifications or variations derived based on the technical concept of the present invention, in addition to the embodiments disclosed herein.

Claims

1. In an electronic device (101), display (160); Sensor module (176); Memory (130) for storing instructions; and Contains a processor (120), The above instructions, when executed by the processor, cause the electronic device to: While displaying a three-dimensional stereoscopic object on the above display, detecting user input invoking a control menu, In response to the detected user input, display a control menu on the display; Detects a touch to select an option item in the above control menu, While maintaining the touch on the selected option item, detecting the movement of the electronic device using the sensor module, An electronic device that changes a user interface displayed on the display based on the detected movement of the electronic device when the touch for the selected option item is released.

2. In the first paragraph, when the instructions are executed by the processor, the electronic device, An electronic device that displays the control menu in a corner edge area of ​​the display and displays a pointer in the center of the display in response to the detected user input.

3. In the first paragraph, when the instructions are executed by the processor, the electronic device, When depth movement is selected as an option item in the above control menu, first sensing data in the depth direction is acquired from the sensor module, An electronic device that changes depth information of a three-dimensional stereoscopic object displayed on the display based on the first sensing data acquired above according to the release of the touch.

4. In the first paragraph, when the instructions are executed by the processor, the electronic device, When the change camera view is selected as an option item in the above control menu, the second sensing data in the depth direction is acquired from the sensor module, Change the camera view based on the second sensing data acquired above, An electronic device that changes the user interface based on the changed camera view.

5. In the first paragraph, when the instructions are executed by the processor, the electronic device, When screen panning is selected as an option item in the above control menu, the third sensing data is acquired from the sensor module, An electronic device that changes the user interface based on the third sensing data acquired above.

6. In the second paragraph, the instructions, when executed by the processor, cause the electronic device to: An electronic device that selects a three-dimensional stereoscopic object when the three-dimensional stereoscopic object corresponds to the pointer based on the acquired sensing data.

7. In the sixth paragraph, the instructions, when executed by the processor, cause the electronic device to: An electronic device that executes a function corresponding to the selected three-dimensional stereoscopic object when the touch for the selected option item is released.

8. In the second paragraph, the instructions, when executed by the processor, cause the electronic device to: Based on the sensing data acquired above, if a 3D stereoscopic object is located adjacent to the highlighted area corresponding to the pointer, the located 3D stereoscopic object is selected, An electronic device that, when the above touch release is detected, causes a function corresponding to the selected three-dimensional stereoscopic object to be executed.

9. In the 8th paragraph, the instructions, when executed by the processor, cause the electronic device to: Move the three-dimensional stereoscopic object placed adjacent to the above highlighted area to the above highlighted area, An electronic device that selects a three-dimensional stereoscopic object when the three-dimensional stereoscopic object is aligned with the highlighted area.

10. In the 8th paragraph, the instructions, when executed by the processor, cause the electronic device to: Moving a first three-dimensional stereoscopic object positioned adjacent to the above highlighted area to the above highlighted area, An electronic device that moves the position of a second three-dimensional stereoscopic object displayed on the display based on the movement of the first three-dimensional stereoscopic object.

11. In the operating method of an electronic device (101), An action of detecting a user input invoking a control menu while displaying a three-dimensional stereoscopic object on a display (160) of the electronic device; An action of displaying a control menu on the display in response to the detected user input; An action to detect a touch for selecting an option item in the above control menu; An operation of detecting movement of the electronic device using a sensor module (176) of the electronic device while maintaining the touch on the selected option item; and A method comprising an action of changing a user interface displayed on the display based on the detected movement of the electronic device when the touch for the selected option item is released.

12. In paragraph 11, the action of displaying the control menu is: A method comprising the action of displaying the control menu in a corner edge area of ​​the display in response to the detected user input and displaying a pointer in the center of the display.

13. In paragraph 11, When depth movement is selected as an option item of the above control menu, an operation of acquiring first sensing data in the depth direction from the sensor module; and A method further comprising an action of changing depth information of a three-dimensional stereoscopic object displayed on the display based on the first sensing data acquired above according to the release of the touch.

14. In paragraph 11, An operation of acquiring second sensing data in the depth direction from the sensor module when changing the camera view is selected as an option item of the above control menu; An operation of changing the camera view based on the second sensing data acquired above; and A method further comprising an action of changing the user interface based on the changed camera view.

15. In paragraph 11, When screen panning is selected as an option item of the above control menu, an operation of acquiring third sensing data from the sensor module; and A method comprising an action of changing the user interface based on the third sensing data acquired above.

Images