Method for providing work space in extended reality and electronic device therefor

Abstract

Disclosed is an electronic device including at least one sensor, at least one camera, a microphone, a display, a memory, and at least one processor. The electronic device may acquire at least one first image, and identify an object of interest among surrounding objects of the electronic device from the at least one first image, on the basis of an interaction with a user of the electronic device. The electronic device may acquire context information on the basis of at least one of sensor data and the at least one first image. The electronic device may identify a recommended work space among a plurality of work spaces stored in the memory on the basis of the context information and an application associated with the object of interest, and provide the recommended work space through the display.

Description

Method for providing a workspace within augmented reality and electronic device for the same

[0001] The embodiments disclosed in this document relate to a method for providing a workspace in extended reality (XR) and an electronic device for the same.

[0002] Electronic devices for providing extended reality (XR) are being researched. XR may include augmented reality (AR), virtual reality (VR), and / or mixed reality (MR). Electronic devices can provide extended reality by synthesizing virtual objects and / or information onto a real-world environment. For example, electronic devices can provide extended reality by displaying graphic objects on a transparent display. For example, electronic devices can provide extended reality by displaying graphic objects along with images of the real environment on an opaque display (e.g., via video-see-through, VST methods).

[0003] Within augmented reality, users of electronic devices can run various applications. The application execution screens can be provided within augmented reality through electronic devices. Users can place execution screens at specific locations within the real environment. Users can place multiple execution screens within augmented reality and perform various tasks within it.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art in relation to the present disclosure.

[0005] An electronic device according to one embodiment disclosed in this document may include at least one sensor, at least one camera, a microphone, a display, a memory, and at least one processor. The at least one processor may be communicatively connected to the microphone, the display, and the memory, and may include at least one processing circuit. The memory may store instructions that, when executed individually or in combination by the at least one processor, enable the electronic device to acquire at least one first image and identify an object of interest among the surrounding objects of the electronic device from the at least one first image based on interaction with a user of the electronic device. The memory may store instructions that, when executed individually or in combination by the at least one processor, enable the electronic device to acquire context information based on at least one of sensor data or the at least one first image. When the above memory is executed individually or in combination by the at least one processor, the electronic device may store instructions that identify a recommended workspace among a plurality of workspaces stored in memory based on an application associated with an object of interest and context information, and provide the recommended workspace through a display.

[0006] Additionally, a method for providing a recommended workspace of an electronic device according to an embodiment disclosed in this document may include: acquiring at least one first image using at least one camera of the electronic device; identifying an object of interest among surrounding objects of the electronic device from the at least one first image based on interaction with a user of the electronic device; acquiring context information based on at least one of sensor data acquired using at least one sensor of the electronic device or the at least one first image; identifying a recommended workspace among a plurality of workspaces stored in the memory of the electronic device based on an application associated with the object of interest and the context information; and providing the recommended workspace through a display of the electronic device.

[0007] A computer-readable storage medium according to one embodiment disclosed in this document may store instructions that cause the electronic device to perform the method for providing the recommended workspace when executed by the processor of the electronic device.

[0008] Figure 1 illustrates an example of augmented reality.

[0009] FIG. 2 illustrates a block diagram of an electronic device according to one embodiment.

[0010] FIG. 3a shows a perspective view of an electronic device according to one embodiment.

[0011] FIG. 3b illustrates an example of internal hardware of an electronic device according to one embodiment.

[0012] FIG. 4a shows a rear perspective view of an electronic device according to one embodiment.

[0013] FIG. 4b shows a front perspective view of an electronic device according to one embodiment.

[0014] FIG. 5a shows a front perspective view of a detachable electronic device according to one embodiment.

[0015] FIG. 5b shows a rear perspective view of a detachable electronic device according to one embodiment.

[0016] FIG. 6 illustrates examples of electronic devices according to an embodiment.

[0017] FIG. 7 illustrates a block diagram of software configurations of an electronic device according to one embodiment.

[0018] FIG. 8a is a flowchart of a novel work area storage method of an electronic device according to one embodiment.

[0019] FIG. 8b is a flowchart of a method for updating the work area of ​​an electronic device according to one embodiment.

[0020] Figure 9 illustrates an example of a real space.

[0021] FIG. 10 illustrates an extended reality according to one embodiment.

[0022] FIG. 11a illustrates a work area changed from the work area of ​​FIG. 10.

[0023] FIG. 11b illustrates an update UI (user interface) according to one embodiment.

[0024] FIG. 11c illustrates a work area save UI according to one embodiment.

[0025] FIG. 11d illustrates an application-based workspace editing UI according to one embodiment.

[0026] FIG. 11e illustrates an object-based workspace editing UI according to one embodiment.

[0027] FIG. 12 illustrates an immersive type work area according to one embodiment.

[0028] FIGS. 13a and 13b are flowcharts of a work area restoration method according to one embodiment.

[0029] FIG. 14a illustrates a recommendation UI according to one embodiment.

[0030] FIG. 14b illustrates a restored work area according to one embodiment.

[0031] FIGS. 15a and 15b illustrate examples of the restoration of a working area of ​​an electronic device according to one embodiment.

[0032] FIG. 16 is a flowchart of a method for providing a recommended workspace for an electronic device according to one embodiment.

[0033] FIG. 17 is a block diagram of an exemplary electronic device capable of performing the operations described in this document.

[0034] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0035] Hereinafter, various embodiments of the present invention are described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention.

[0036] Figure 1 illustrates an example of augmented reality.

[0037] Referring to FIG. 1, according to one embodiment, an electronic device (10) may be configured to provide extended reality (e.g., augmented reality, mixed reality, and / or virtual reality). For example, the electronic device (10) may provide augmented reality and / or mixed reality by mapping and displaying virtual objects mapped to the real space where the user (1) is located. For example, the electronic device (10) may provide augmented reality by displaying virtual objects within any virtual space. For example, the electronic device (10) may provide extended reality by configuring at least some of the surrounding spaces of the user (1) as virtual space and the remaining parts as real space.

[0038] In the present disclosure, “actual space” may include physically existing space and physical objects within the physical space. In the present disclosure, “virtual space” may be referred to as a virtual space provided by an electronic device (10). The virtual space and the actual space may be distinguished based on a visual representation provided by the electronic device (10). For example, if the electronic device (10) captures and provides the actual space, the provided space may be referred to as the actual space. For example, if the electronic device (10) visually reconstructs the actual space, the provided space may be referred to as the virtual space.

[0039] In the present disclosure, “virtual object” may be referred to as a graphic object that does not exist in actual space but is displayed by an electronic device (10). The electronic device (10) may display the virtual object in actual space and / or virtual space.

[0040] In the example of FIG. 1, the electronic device (10) can display a first virtual object (8a) and a second virtual object (8b) in real space. For example, the electronic device (10) can display the first virtual object (8a) and the second virtual object (8b) mapped to a real object (e.g., a laptop (3) on a desk (2)). In the present disclosure, “mapping display” may include displaying virtual objects by mapping them to a real object or a real location. For example, the electronic device (10) can display the first virtual object (8a) and the second virtual object (8b) by mapping them to the laptop (3). The electronic device (10) can display the first virtual object (8a) and the second virtual object (8b) at a location adjacent to the laptop (3) when viewed by the user (1). The electronic device (10) can display the first virtual object (8a) and the second virtual object (8b) such that the first virtual object (8a) and the second virtual object (8b) have a depth similar to that of the laptop (3). For example, the electronic device (10) can set the depth of the first virtual object (8a) and the second virtual object (8b) such that the distance from the user (1) to the first virtual object (8a) and the second virtual object (8b) is similar to the distance from the user (1) to the laptop (3).

[0041] According to one embodiment, the electronic device (10) may support multiple modalities. For example, the electronic device may be configured to receive and process various types of inputs. For example, 'modality' may refer to a channel for interaction between the electronic device and the user. In this case, voice input and text input through an interface (e.g., a virtual keyboard) may be considered different modalities. For example, 'modality' may refer to the format of data input to an artificial intelligence model (e.g., a generative artificial intelligence model). For example, the user's voice input may be converted into text data through STT (speech to text) and NLU (natural language understanding) and input to the artificial intelligence model. In this case, from the perspective of the artificial intelligence model, voice input and text input through an interface (e.g., a virtual keyboard) may be considered substantially the same modality. Hereinafter, 'modality' may refer to a channel between the user and the electronic device and / or the format of input data to the artificial intelligence model. In the present disclosure, 'modality' may be referred to as 'input type'.

[0042] In one example, the electronic device (10) may be a wearable device. The electronic device (10) may acquire voice input from a user using a microphone. The electronic device (10) may acquire image input using a camera. The electronic device (10) may acquire user input using an interface (e.g., a button and / or a touchpad). The electronic device (10) may acquire user input from an external device (not shown) that is communicably connected. The electronic device (10) may be configured to acquire voice input, image input and / or user input, and to process the acquired input.

[0043] According to one embodiment, the electronic device (10) may be configured to provide a “work area.” For example, the first virtual object (8a) and / or the second virtual object (8b) may correspond to the execution screen of at least one application. The user (1) may execute an application related to the work through the electronic device (10) while performing work through the laptop (3). The electronic device (10) may display the first virtual object (8a) and the second virtual object (8b), which are the execution screens of the application. For example, the first virtual object (8a) and the second virtual object (8b) may form a “work area” together with the laptop (3) and / or the desk (2).

[0044] In the present disclosure, a “work area” may include at least one application executed by a user (1) for a specific purpose or for a specific task. For example, a work area may consist of at least one application mapped to a real space (e.g., a location within the real space, the configuration of the real space, and / or a real object). Information in the work area may include surrounding environment information and application information. Information in the work area may be described later in relation to FIG. 7. In the present disclosure, the term “work area” may be referred to as the term “work space.” Additionally, “work” is not limited to the user’s (1) purpose. For example, “work” may include not only tasks or work related to a job, but also work related to rest.

[0045] According to one embodiment, the electronic device (10) stores information about a work area and can restore the work area based on the information about the work area. For example, the electronic device (10) can store a work area including surrounding environment information (e.g., context information) and application information. The electronic device (10) can restore the work area using the information about the stored work area. For example, the user (1) can move the location of the execution screen (e.g., the first virtual object (8a)) during work. The electronic device (10) can restore the location of the execution screen using the information about the stored work area. For example, the user (1) can perform work using the laptop (3) at a location different from FIG. 1. The electronic device (10) can provide the work area at the changed location using the information about the stored work area.

[0046] In the following, methods for providing a working area of ​​an electronic device (10) may be described with reference to various drawings. The example described above in relation to FIG. 1 is for illustrative purposes only and is not limited to the embodiments of the present disclosure. For example, in the example of FIG. 1, the working area is described with the user (1)'s field of view area (9) as the center, but the working area may include an area other than the reagent area (9). However, unless otherwise described, the details described above in relation to FIG. 1 may be applied to the embodiments described below.

[0047] FIG. 2 illustrates a block diagram of an electronic device according to one embodiment.

[0048] Referring to FIG. 2, according to one embodiment, the electronic device (10) may include a processor (120), memory (130), sensor circuit (140), display (160), camera (170), interface (180), and / or communication circuit (190). The electronic device (10) may correspond to the electronic device (300) of FIG. 3a and 3b, the electronic device (400) of FIG. 4a and 4b, the electronic device (500) of FIG. 5a and 5b, the smart glasses (600) of FIG. 6, the wearable device (610), and / or the electronic device (1700) of FIG. 17. The electronic device (10) may include a configuration similar to the electronic device (1700) described below in relation to FIG. 17. For example, the processor (120) may correspond to at least one processor (1710) of FIG. 17. For example, the memory (130) may correspond to the memory (1720) of FIG. 17. For example, the sensor circuit (140) may correspond to the sensor interface (1719) and / or sensor (1770) of FIG. 17. For example, the display (160) may correspond to the display (1740) of FIG. 17. For example, the camera (170) may include the image sensor (1750) of FIG. 17. For example, the communication circuit (190) may correspond to the communication circuit (1760) of FIG. 17. The configuration of the electronic device (10) shown in FIG. 2 is exemplary and the configuration of the electronic device (10) is not limited thereto. For example, the electronic device (10) may further include a configuration not shown in FIG. 2 (e.g., at least one of the configurations of the electronic device (1700) of FIG. 17). For example, the electronic device (10) may not include at least one of the configurations shown in FIG. 2.

[0049] The processor (120) may be connected communically, electrically, operatively, or functionally to memory (130), sensor circuit (140), display (160), camera (170), interface (180), and / or communication circuit (190). In various embodiments of the present disclosure, when one component is connected “operatively” to another component, it may mean that the component is connected to enable the other component to operate. For example, the component may enable the other component by transmitting a control signal to the other component directly or through another component. In various embodiments of the present disclosure, when one component is connected “functionally” to another component, it may mean that the component is connected to enable the function of the other component. For example, the component may enable the function of the other component by transmitting a control signal to the other component directly or through another component.

[0050] The processor (120) may include at least one processor. The at least one processor may include at least one processing circuit. For example, the processor (120) may include an application processor (AP), a central processing unit (CPU), an image signal processor (ISP), a graphics processing unit (GPU), a neural processing unit (NPU), a tensor processing unit (TPU), and / or a communication processor (CP). The processor (120) may include at least one chip or one chipset. In the present disclosure, the processor (120) may be referred to as a hardware component having an architecture by at least one processing circuit. For example, the processor (120) may be placed on a substrate (e.g., a printed circuit board) located inside the electronic device (10) and may communicate with other components of the electronic device (10) through at least one conductive path formed in the substrate, a flexible printed circuit board (FPCB), a cable, and / or any communication channel (e.g., a wireless communication channel).

[0051] The memory (130) can store instructions. When the instructions are executed by the processor (120), they can cause the electronic device (10) to perform various operations. For example, the instructions can cause the electronic device (10) to perform various operations by being executed individually or collectively by at least one processor. In various embodiments of the present disclosure, the operation of the electronic device (10) may be referred to as an operation performed by the processor (120) by executing instructions stored in the memory (130). The memory (130) may be referred to as a hardware component for storing data.

[0052] The sensor circuit (140) may be configured to detect information associated with the electronic device (10), e.g., context information. The context information may include optical information of the surroundings of the electronic device (10), movement information of the electronic device (10), attachment status information of the electronic device (10), location information of the electronic device (10), and / or information of adjacent objects. For example, the sensor circuit (140) may include an image sensor (141), an inertial sensor (143), a position sensor (145), and / or a proximity sensor (147). The configuration of the sensor circuit (140) shown in FIG. 2 is exemplary and the embodiments of the present disclosure are not limited thereto.

[0053] The image sensor (141) may be configured to detect optical signals. For example, the image sensor (141) may be configured to detect light signals, infrared signals, and / or ultraviolet signals. The image sensor (141) may be configured to acquire depth information. For example, the image sensor (141) may include a light detection and ranging (LiDAR).

[0054] The inertial sensor (143) may be configured to detect movement of the electronic device (10). For example, the inertial sensor (143) may include an inertial measurement unit (IMU) configured to detect the orientation, magnetic force, angular rate, and / or specific force of the electronic device (10). The inertial sensor (143) may include an accelerometer, a gyroscope, and / or a magnetometer.

[0055] The location sensor (145) may include any sensor configured to acquire the geographical location of the electronic device (10). For example, the location sensor (145) may acquire geographical location information based on a global navigation satellite system (GNSS). For example, the location sensor (145) may acquire geographical location information based on the reception strength and / or angle of arrival of an external signal. In one example, the electronic device (10) may acquire geographical location information based on reception information from a base station.

[0056] The proximity sensor (147) may be configured to detect an external object in close proximity to the electronic device (10). The electronic device (10) may use the proximity sensor (147) to detect the blockage status of the electronic device (10). For example, the proximity sensor (147) may include a capacitive sensor, a photoelectric sensor, a touch switch, an illuminance sensor, and / or an inductive sensor.

[0057] The display (160) may include at least one pixel configured to display an image. In one example, the display (160) may include a plurality of displays. For example, the display (160) may include a left-eye display and a right-eye display. The display (160) may include a front display and / or a rear display. The display (160) may include at least one of a see-through display, a flexible display, a rollable display, a foldable display and / or a rigid display. In one example, the display (160) may include at least one projector for projecting an image.

[0058] The camera (170) may include at least one camera. When the electronic device (10) includes multiple cameras, each of the multiple cameras may differ in at least one of the facing direction, magnification, or field of view. In one example, the electronic device (10) may select a camera from the multiple cameras to acquire an image. The electronic device (10) may select a camera to acquire an image based on the user's context information (e.g., speech and / or direction of movement). For example, the electronic device (10) may identify an image containing an object corresponding to the user's speech among the multiple cameras through image recognition. The electronic device (10) may acquire an image corresponding to the voice input using the camera that acquired the identified image. For example, the electronic device (10) may select a camera among the multiple cameras that faces the direction of movement of the electronic device (10). In one example, the electronic device (10) may select one camera among the multiple cameras according to user input.

[0059] The interface (180) may include at least one device configured to receive input. For example, the interface (180) may include a touch circuit (181) (e.g., a touch screen display) configured to receive touch input. The interface (180) may include at least one microphone (e.g., a first microphone (182) and / or a second microphone (183)) configured to receive voice input. In one example, the electronic device (10) can identify the location of the speaker of the voice input (e.g., the relative direction of the speaker to the electronic device (10)) by performing beamforming using the first microphone (182) and the second microphone (183). The interface (180) may include at least one device for output. For example, the interface (180) may include a haptic module for tactile output, at least one speaker for sound output (e.g., a speaker (184)), and / or an indicator. The interface (180) may include any human interface device (HID). For example, the interface (180) may include a button (185). According to one embodiment, the processor (120) may be configured to receive input using the interface (180) and to process the received input.

[0060] The communication circuit (190) may be configured to perform short-range wireless communication and / or long-range wireless communication. The communication circuit (190) may include a network interface card (NIC). The processor (120) may communicate with other external electronic devices based on wireless communication and / or wired communication, for example, using the communication circuit (190). The processor (120) may communicate with external devices via an internet protocol (IP) network, for example, using the communication circuit (190).

[0061] According to one embodiment, the electronic device (10) may be configured to store a work area and provide the stored work area. When performing work in a three-dimensional space (e.g., virtual space and / or real space), the electronic device (10) may place multiple execution screens for the work within the work area. After interrupting the work, the user may resume the work. In this case, the user may need to rearrange the multiple execution screens. When performing work in a mixed reality space, the user's experience may be degraded due to the repetitive execution and restoration of the application because the execution screens must be placed in the three-dimensional space. The electronic device (10) according to an embodiment of the present disclosure may reduce user inconvenience through the restoration of the work area.

[0062] For example, the electronic device (10) can determine whether to save a work area by analyzing at least one of the surrounding environment, surrounding objects, or user posture. The electronic device (10) can save information necessary for restoring the work area (e.g., running applications, 3D objects, surrounding environment, and / or user posture). The electronic device (10) can periodically analyze the surrounding environment, surrounding objects, and / or user posture. For example, when the user visits a new place, the electronic device (10) can compare the results of the analysis with the saved work area. If the electronic device (10) determines that restoration of the work area is necessary based on at least one of the saved work areas, it can restore the work area to match the surrounding environment.

[0063] For example, the electronic device (10) can restore spatial data from when the work area was saved during the work area restoration process. For example, the user can decide whether to restore spatial data along with the work area restoration. If spatial data is restored along with the work area, the electronic device (10) can restore spatial data from when the work area was saved (e.g., the user's room in the house) along with the work area. In this case, the user can feel as if they are located in the space (e.g., the user's room) at the time the work area was saved. Through the restoration of spatial data, the user's concentration on work can be further increased.

[0064] FIG. 3a shows a perspective view of an electronic device according to one embodiment.

[0065] Referring to FIG. 3a, the electronic device (300) may correspond to an example of the electronic device (10) of FIG. 1. For example, the electronic device (300) may include at least one display (350) (e.g., the display (160) of FIG. 2) and a frame supporting at least one display (350). For example, the electronic device (300) may be worn on a part of a user's body. For example, the electronic device (300) may include a glasses-type device configured to provide augmented reality.

[0066] The frame may be formed as a physical structure that allows the electronic device (300) to be worn on the user's body. The frame may be configured so that when the user wears the electronic device (300), the first display (350-1) and the second display (350-2) can be positioned corresponding to the user's left and right eyes. The frame may support at least one display (350).

[0067] The frame may include a first rim (301) covering at least a portion of a first display (350-1), a second rim (302) covering at least a portion of a second display (350-2), a bridge (303) positioned between the first rim (301) and the second rim (302), a first pad (311) positioned along a portion of the edge of the first rim (301) from one end of the bridge (303), a second pad (312) positioned along a portion of the edge of the second rim (302) from the other end of the bridge (303), a first temple (304) extending from the first rim (301) and fixed to a portion of the wearer's ear, and a second temple (305) extending from the second rim (302) and fixed to a portion of the ear opposite to the ear. The first pad (311) and the second pad (312) may come into contact with a part of the user's nose, and the first temple (304) and the second temple (305) may come into contact with a part of the user's face and a part of the ear. The temples (304, 305) may be rotatably connected to the rim through the hinge units (306, 307) of FIG. 3B. The first temple (304) may be rotatably connected to the first rim (301) through a first hinge unit (306) positioned between the first rim (301) and the first temple (304). The second temple (305) may be rotatably connected to the second rim (302) through a second hinge unit (307) positioned between the second rim (302) and the second temple (305).

[0068] The frame may include a contact area (320) in which at least a portion comes into contact with a part of the user's body when the user wears the electronic device (300). For example, the contact area (320) may include a nose pad (310), a first temple (304), and a second temple (305).

[0069] The electronic device (300) can provide visual information to a user through at least one display (350). For example, at least one display (350) may include a transparent or translucent lens. At least one display (350) may include a first display (350-1) and / or a second display (350-2) spaced apart from the first display (350-1). For example, the first display (350-1) and the second display (350-2) may be positioned at locations corresponding to the user's left and right eyes, respectively.

[0070] FIG. 3b illustrates an example of internal hardware of an electronic device according to one embodiment.

[0071] Referring to FIGS. 3a and 3b, according to one embodiment, an electronic device (300) may include hardware that performs various functions (e.g., hardware described above in relation to the block diagram of FIG. 2). For example, the hardware may include a battery module (370), an antenna module (375), optical devices (382, 384), speakers (392-1, 392-2), microphones (394-1, 394-2, 394-3), a light-emitting module (not shown), and / or a printed circuit board (390). The various hardware may be placed within a frame.

[0072] At least one display (350) (e.g., the display (160) of FIG. 2) may form a display area on a lens to provide a user wearing an electronic device (300) with visual information that is distinct from the visual information, along with the visual information contained in the external light passing through the lens. The lens may be formed based on at least one of a Fresnel lens, a pancake lens, or a multi-channel lens. The display area formed by the at least one display (350) may be formed on the second surface (332) of the first surface (331) and the second surface (332) of the lens. When the user wears the electronic device (300), the external light may be transmitted to the user by being incident on the first surface (331) and transmitted through the second surface (332). As another example, the at least one display (350) may display a virtual reality image to be combined with a real-world image transmitted through the external light. The virtual reality image output from at least one display (350) can be transmitted to the user's eye through one or more hardware included in the electronic device (300) (e.g., optical devices (382, 384) and / or at least one waveguide (333, 334)).

[0073] According to one embodiment, an electronic device (300) may include waveguides (333, 334) that diffract light transmitted from at least one display (350) and relayed by optical devices (382, 384) and transmit it to a user. The waveguides (333, 334) may be formed based on at least one of glass, plastic, or polymer. A nano pattern may be formed on the exterior or at least a portion of the interior of the waveguides (333, 334). The nano pattern may be formed based on a polygonal and / or curved grating structure. Light incident on one end of the waveguides (333, 334) may be propagated to the other end of the waveguides (333, 334) by the nano pattern. Waveguides (333, 334) may include at least one diffractive element (e.g., DOE (diffractive optical element), HOE (holographic optical element)) and at least one reflective element (e.g., a reflective mirror). For example, waveguides (333, 334) may be placed within an electronic device (300) to guide a screen displayed by at least one display (350) to the user's eye. For example, the screen may be transmitted to the user's eye based on total internal reflection (TIR) ​​occurring within the waveguides (333, 334).

[0074] According to one embodiment, microphones (394-1, 394-2, 394-3) of an electronic device (300) (e.g., the first microphone (182) and / or the second microphone (183) of FIG. 2) are positioned on at least a portion of a frame to acquire a sound signal. Although the first microphone (394-1) positioned on the nose pad (310), the second microphone (394-2) positioned on the second rim (302), and the third microphone (394-3) positioned on the first rim (301) are shown in FIG. 3b, the number and position of the microphones (394) are not limited to the embodiment of FIG. 3b. If there are two or more microphones (394) included in the electronic device (300), the electronic device (300) can identify the direction of the sound signal using a plurality of microphones positioned on different portions of the frame.

[0075] In one embodiment, the camera (340) (e.g., the camera (170) of FIG. 2) may include an eye tracking camera (ET CAM) (340-1a, 340-1b), a motion recognition camera (340-2), and / or a shooting camera (340-3). The shooting camera (340-3), the eye tracking camera (340-1a, 340-1b), and the motion recognition camera (340-2) may be positioned at different locations on the frame and may perform different functions. The eye tracking camera (340-1a, 340-1b) may output data representing the gaze of a user wearing the electronic device (300). For example, the electronic device (300) may detect the gaze from an image containing the user's pupils obtained through the eye tracking camera (340-1a, 340-1b). An example in which the eye-tracking cameras (340-1a, 340-1b) are positioned toward both eyes of the user is illustrated in FIG. 3b, but the embodiment is not limited thereto, and the eye-tracking cameras (340-1a, 340-1b) may be positioned alone toward the user's left or right eye.

[0076] A shooting camera (340-3) can capture a real image or background to be matched with a virtual image in order to implement augmented reality or mixed reality content. The shooting camera can capture an image of a specific object located at the position where the user is looking and provide the image to at least one display (350). The at least one display (350) can display a single image in which information regarding a real image or background including the image of the specific object obtained using the shooting camera is superimposed with a virtual image provided through optical devices (382, 384). In one embodiment, the shooting camera may be placed on a bridge (303) positioned between the first rim (301) and the second rim (302).

[0077] The eye tracking cameras (340-1a, 340-1b) can achieve more realistic augmented reality by tracking the gaze of a user wearing the electronic device (300), thereby matching the user's gaze with visual information provided to at least one display (350). For example, when the user looks straight ahead, the electronic device (300) can naturally display environmental information related to the user's front on at least one display (350) at the location where the user is situated. The eye tracking cameras (340-1a, 340-1b) may be configured to capture an image of the user's pupil to determine the user's gaze. For example, the eye tracking cameras (340-1a, 340-1b) may receive a gaze detection light reflected from the user's pupil and track the user's gaze based on the position and movement of the received gaze detection light. In one embodiment, the eye tracking cameras (340-1a, 340-1b) may be positioned at locations corresponding to the user's left and right eyes. For example, the eye-tracking cameras (340-1a, 340-1b) may be positioned within the first rim (301) and / or the second rim (302) to face the direction in which the user wearing the electronic device (300) is located.

[0078] A motion recognition camera (340-2) can provide a specific event on a screen provided on at least one display (350) by recognizing the movement of the user's entire body or part thereof, such as the user's torso, hands, or face. The motion recognition camera (340-2) can recognize the user's motion (e.g., gesture recognition), acquire a signal corresponding to said motion, and provide a display corresponding to said signal on at least one display (350). A processor can perform a designated function based on identifying the signal corresponding to said motion. In one embodiment, the motion recognition camera (340-2) may be placed on the first rim (301) and / or the second rim (302).

[0079] According to one embodiment, an electronic device (300) can analyze an object included in a real-world image collected through a shooting camera (340-1a, 340-1b), combine a virtual object corresponding to an object among the analyzed objects that is the target of augmented reality provision, and display it on at least one display (350). The virtual object may include at least one of text and an image regarding various information related to the object included in the real-world image. The electronic device (300) can analyze the object based on a multi-camera such as a stereo camera. For the object analysis, the electronic device (300) can perform time-of-flight (ToF) and / or simultaneous localization and mapping (SLAM) supported by the multi-camera.

[0080] The camera (340) included in the electronic device (300) is not limited to the eye-tracking camera (340-1a, 340-1b) and / or motion recognition camera (340-2) described above. For example, the electronic device (300) can identify external objects included within the field of view (FoV) by using a shooting camera (340-3) positioned toward the user's field of view (FoV). The identification of external objects by the electronic device (300) can be performed based on a sensor for identifying the distance between the electronic device (300) and the external object, such as a depth sensor and / or a time of flight (ToF) sensor. The camera (340) positioned toward the FoV can support an autofocus function and / or an optical image stabilization (OIS) function. For example, the electronic device (300) may include a camera (340) (e.g., a face tracking camera) positioned toward the face to acquire an image including the face of a user wearing the electronic device (300).

[0081] The battery module (370) can supply power to the electronic components of the electronic device (300). In one embodiment, the battery module (370) may be placed within the first temple (304) and / or the second temple (305). For example, the battery module (370) may be a plurality of battery modules (370). The plurality of battery modules (370) may each be placed in the first temple (304) and the second temple (305). In one embodiment, the battery module (370) may be placed at the end of the first temple (304) and / or the second temple (305).

[0082] The antenna module (375) can transmit a signal or power to the outside of the electronic device (300) or receive a signal or power from the outside. The antenna module (375) can be electrically and / or operatively connected to a communication circuit within the electronic device (300) (e.g., communication circuit (190) of FIG. 2). In one embodiment, the antenna module (375) may be placed within the first temple (304) and / or the second temple (305). For example, the antenna module (375) may be placed close to one side of the first temple (304) and / or the second temple (305).

[0083] Speakers (392-1, 392-2) (e.g., speaker (184) of FIG. 2) can output an acoustic signal to the outside of the electronic device (300). In one embodiment, the speakers (392-1, 392-2) may be placed within a first temple (304) and / or a second temple (305) to be positioned adjacent to the ears of a user wearing the electronic device (300). For example, the electronic device (300) may include a second speaker (392-2) positioned adjacent to the user's left ear within the first temple (304), and a first speaker (392-1) positioned adjacent to the user's right ear within the second temple (305).

[0084] The electronic device (300) may include a printed circuit board (390) on a PCB. The PCB (390) may be included in at least one of a first temple (304) or a second temple (305). The PCB (390) may include an interposer disposed between at least two sub-PCBs. One or more hardware components included in the electronic device (300) may be disposed on the PCB (390). The electronic device (300) may include a flexible PCB (FPCB) for interconnecting the hardware components.

[0085] The electronic device (300) may further include configurations not illustrated in relation to FIGS. 3a and 3b. For example, the electronic device (300) may further include a light source (e.g., an LED (light emitting diode)) that emits light toward a subject (e.g., a user's eye, face, and / or an object outside the FoV) being photographed using the camera (340). The light source may include an LED of infrared wavelength. The light source may be placed in at least one of the frame and hinge units (306, 307).

[0086] For example, the electronic device (300) can identify an external object touching the frame (e.g., a user's fingertip) and / or a gesture performed by said external object by using a touch sensor, a grip sensor, and / or a proximity sensor formed on at least a portion of the surface of the frame.

[0087] For example, the electronic device (300) may include at least one of a gyroscope sensor, a gravity sensor, and / or an accelerometer sensor (e.g., sensor circuit (140) of FIG. 2) for detecting the posture of the electronic device (300) and / or the posture of a body part (e.g., head) of a user wearing the electronic device (300).

[0088] FIG. 4a shows a rear perspective view of an electronic device according to one embodiment. FIG. 4b shows a front perspective view of an electronic device according to one embodiment.

[0089] Referring to FIGS. 4a and 4b, the electronic device (400) may correspond to an example of the electronic device (10) of FIG. 1.

[0090] Referring to FIG. 4a, according to one embodiment, a first surface (410) of an electronic device (400) may have a wearable form on a part of a user's body (e.g., the user's face). Although not illustrated, the electronic device (400) may further include a strap for securing to a part of a user's body and / or one or more temples (e.g., a first temple (304) and / or a second temple (305) of FIG. 3a and FIG. 3b). A first display (450-1) for outputting an image to the user's left eye and a second display (450-2) for outputting an image to the user's right eye may be disposed on the first surface (410). The electronic device (400) may further include a rubber or silicone packing formed on the first surface (410) to prevent interference by light different from light emitted from the first display (450-1) and the second display (450-2) (e.g., ambient light).

[0091] According to one embodiment, the electronic device (400) may include cameras (440-1, 440-2) (e.g., camera (170) of FIG. 2) for photographing and / or tracking both eyes of an adjacent user. The cameras (440-1, 440-2) may be referred to as ET (eye tracking) cameras. According to one embodiment, the electronic device (400) may include cameras (440-3, 440-4) (e.g., camera (170) of FIG. 2) for photographing and / or recognizing the face of a user. The cameras (440-3, 440-4) may be referred to as FT cameras.

[0092] Referring to FIG. 4b, on a second surface (420) opposite to the first surface (410) of FIG. 4a, a camera (e.g., cameras (440-5, 440-6, 440-7, 440-8, 440-9, 440-10)) (e.g., camera (170) of FIG. 2)) and / or a sensor (e.g., depth sensor (430)) may be placed to acquire information related to the external environment of the electronic device (400). For example, cameras (440-5, 440-6, 440-7, 440-8, 440-9, 440-10) may be placed on the second surface (420) to recognize external objects different from the electronic device (400). For example, by using cameras (440-9, 440-10), the electronic device (400) can acquire images and / or media to be transmitted to each of the user's two eyes. Camera (440-9) may be placed on the second surface (420) of the electronic device (400) to acquire an image to be displayed through a second display (450-2) corresponding to the right eye among the two eyes. Camera (440-10) may be placed on the second surface (420) of the electronic device (400) to acquire an image to be displayed through a first display (450-1) corresponding to the left eye among the two eyes.

[0093] According to one embodiment, the electronic device (400) may include a depth sensor (430) (e.g., sensor circuit (140)) disposed on a second surface (420) to identify the distance between the electronic device (400) and an external object. Using the depth sensor (430), the electronic device (400) can obtain spatial information (e.g., depth map) for at least a portion of the FoV of a user wearing the electronic device (400).

[0094] Although not shown, a microphone (e.g., the first microphone (182) and / or the second microphone (183) of FIG. 2) for acquiring sound output from an external object may be placed on the second surface (420) of the electronic device (400). The number of microphones may be one or more depending on the embodiment.

[0095] As described above, according to one embodiment, the electronic device (400) may have a form factor for being worn on a user's head. The electronic device (400) may provide a user experience based on augmented reality, virtual reality, and / or mixed reality while being worn on the head. Using a first display (450-1) and a second display (450-2) positioned toward each of the user's two eyes, the electronic device (400) may display a screen containing an external object. Within the screen, the electronic device (400) may register the external object or display the result of executing a function assigned to the external object.

[0096] FIG. 5a shows a front perspective view of a detachable electronic device according to one embodiment. FIG. 5b shows a rear perspective view of a detachable electronic device according to one embodiment.

[0097] Referring to FIGS. 5a and 5b, according to one embodiment, the electronic device (500) may be configured to be detachably attached to the housing (501). For example, the electronic device (500) may be inserted into a slot (510) of the housing (501) or may be physically coupled to the housing (501) by any structure of the housing (501).

[0098] When combined, the display (550) of the electronic device (500) (e.g., the display (160) of FIG. 2) can be viewed through a left eye lens (551-1) and a right eye lens (551-2). The electronic device (500) can display a left eye image in an area of ​​the display (550) corresponding to the left eye lens (551-1) and display a right eye image in an area of ​​the display (550) corresponding to the right eye lens (551-2).

[0099] Although not illustrated, the housing (501) may further include a strap for securing to a part of the user's body and / or one or more temples (e.g., the first temple (304) and / or the second temple (305) of FIG. 3a and FIG. 3b). The housing (501) may further include rubber or silicone packing to prevent interference by ambient light.

[0100] According to one embodiment, the electronic device (500) may include a camera (540) (e.g., camera (170) of FIG. 2) and / or a sensor (e.g., a depth sensor (not shown)) for acquiring information related to the external environment of the electronic device (500). The housing (501) may include a camera hole (541) for the camera (540) to be exposed to the outside when combined with the electronic device (500). For example, the electronic device (500) may acquire spatial information (e.g., a depth map) for at least a portion of the FoV of a user wearing the electronic device (500) using a plurality of cameras.

[0101] Although not shown, the electronic device (500) may include a microphone (e.g., the first microphone (182) and / or the second microphone (183) of FIG. 2) for acquiring sound output from an external object.

[0102] The housing (501) may provide a structure for physical coupling with the electronic device (500) and for user wearing. In one example, the housing (501) may be electrically coupled with the electronic device (500). In this case, the housing (501) may include at least some of the components of the electronic device (10) described above in relation to FIG. 2. The electronic device (500) may receive information obtained by the components of the housing (501) through communication with the housing (501) or output information using the components of the housing (501).

[0103] FIG. 6 illustrates examples of electronic devices according to an embodiment.

[0104] With reference to FIG. 6, the functions of the electronic device (10) described above in relation to FIG. 2 may be implemented through a plurality of electronic devices. In the examples of FIG. 3a through FIG. 5b, the electronic device is described as a head-mounted device (HMD), but embodiments of the present disclosure are not limited thereto. In the example of FIG. 6, a user (601) may use a plurality of electronic devices connected to communicate with each other.

[0105] For example, visual information may be acquired by a camera of a wearable device (610) (e.g., artificial intelligence (AI)), a mobile device (620), and / or smart glasses (600). For example, auditory information may be acquired by a microphone of smart glasses (600), a wearable device (610), a mobile device (620), and / or an audio device (630). For example, visual information may be provided through a display of a device having a display (e.g., smart glasses (600) and / or a mobile device (620)). For example, auditory information may be provided through a device having a speaker (e.g., smart glasses (600), a wearable device (610), a mobile device (620), and / or an audio device (630)).

[0106] The “electronic device” of the present disclosure may be referred to as a device that stores a work area and determines the restoration of a work area. For example, a mobile device (620) may acquire information using other devices and store a work area based on the acquired information. The mobile device (620) may provide a restored work area through smart glasses (600) based on the stored work area. Even if one electronic device does not support some of the functions described above in relation to the electronic device (10) of FIG. 2, the device that determines the storage and restoration of a work area may be referred to as the electronic device of the present disclosure.

[0107] The examples described above in connection with FIGS. 3a through 6 are examples of forms of electronic devices and are for illustrative purposes only and are not intended to limit the embodiments of the present disclosure. A person skilled in the art will understand that, in addition to the examples described above in connection with FIGS. 3a through 5b, various forms of electronic devices may perform the embodiments of the present disclosure. Furthermore, a person skilled in the art will understand that, as described above in connection with FIG. 6, a plurality of electronic devices may perform the embodiments of the present disclosure through communication.

[0108] FIG. 7 illustrates a block diagram of software configurations of an electronic device according to one embodiment.

[0109] Referring to FIGS. 2 and FIGS. 7, according to one embodiment, the electronic device (10) can store a workspace, provide recommendations based on the stored workspace, and restore a workspace based on the stored workspace. The components of the electronic device (10) described in relation to FIG. 7 may be software modules (e.g., threads, functions, databases, and / or programs) implemented by the electronic device (10) executing instructions stored in memory (130) using a processor (120). In one example, some of the components of the electronic device (10) may be implemented by an external device (e.g., a server). For example, some of the workspace management module (750) may be implemented by an external device. For example, the generative AI module (721) may be implemented by an external device. In this case, the electronic device (10) may transmit and receive information by communicating with the external device using a communication circuit (190).

[0110] In the example of FIG. 7, the electronic device (10) may include an operating system (700) and a workspace management module (750). For example, the workspace management module (750) may correspond to an application or framework running on the operating system (700).

[0111] The operating system (700) may be configured to provide basic system services associated with the electronic device (10) and to perform hardware-dependent tasks. For example, the operating system (700) may include a location calculation module (711), an environment analysis module (713), an object recognition module (715), a body tracking module (717), a system UI (user interface) module (719), and / or a generative AI module (721). The configurations of the operating system (700) illustrated in FIG. 7 are exemplary, and at least some of the configurations of the operating system (700) may correspond to applications or frameworks running on the operating system (700). At least some of the configurations of the operating system (700) illustrated in FIG. 7 may be omitted.

[0112] The position calculation module (711) can identify the position and orientation (e.g., the direction of the electronic device (10) relative to the direction of gravity) of the electronic device (10) in space using at least one camera (e.g., camera (170)) and sensor circuit (140). For example, the position calculation module (711) can construct a spatial coordinate system based on the result of processing acceleration, gyroscope, and gravity sensor input data and calculate the position of the electronic device (10) in the spatial coordinate system in real time.

[0113] The environment analysis module (713) can acquire surrounding environment information of the electronic device (10) using a camera (170) and / or a sensor circuit (140). The surrounding environment information may include information about the surrounding space where the electronic device (10) is located. The surrounding environment information may include, for example, indoor, outdoor, geographical location (e.g., latitude and longitude, geofence), category of the surrounding space (e.g., office, home, inside a car, public transportation, and / or park), relationship between other spaces and the surrounding space (e.g., a room connected to a living room), image of the surrounding space, 3D configuration of the surrounding space (e.g., depth map), information on objects placed in the surrounding space, and / or space visit history. For example, the environment analysis module (713) can identify the geographical location (e.g., latitude and longitude) of the electronic device (10) using a location sensor (145). The environment analysis module (713) can acquire surrounding environment information based on an image acquired using the camera (170). For example, the environment analysis module (713) can obtain surrounding environment information by processing images of the surrounding space. The environment analysis module (713) can obtain surrounding environment information based on user input. The environment analysis module (713) can obtain priority information of surrounding objects based on information of surrounding objects recognized by the object recognition module (715) and user interactions associated with surrounding objects. For example, interactions may include the user's gaze toward an object, manipulation of an object by the user, placement of a graphic object adjacent to an object within a work area by the user, and / or designation of an object by the user.

[0114] The object recognition module (715) can identify the type, location, and / or size of surrounding objects of the electronic device (10). For example, the object recognition module (715) can recognize surrounding objects based on image recognition. At least some of the information about surrounding objects recognized by the object recognition module (715) may be included in the surrounding environment information.

[0115] The body tracking module (717) can identify the user's body parts (e.g., pupils, torso, hands, fingers, arms, head and / or legs) and posture of the electronic device (10). For example, the body tracking module (717) can identify whether the user is sitting or standing. The body tracking module (717) can identify surrounding objects that the user of the electronic device (10) is looking at. For example, the body tracking module (717) can identify the direction in which the user's gaze is directed based on pupil tracking or based on the direction in which the user's head is facing. The operating system (700) can identify surrounding objects that the user is looking at by recognizing surrounding objects located in the direction in which the gaze is directed. The body tracking module (717) can identify the user's gestures based on the user's body parts and / or posture. For example, the body tracking module (717) can determine whether the user is raising or lowering their arms. For example, the body tracking module (717) can identify gestures by identifying the movement of parts of the user's body.

[0116] The system UI module (719) may be configured to provide a UI related to various applications and operating systems running on the electronic device (10). For example, a user of the electronic device (10) may perform tasks such as launching or closing an application, resizing the application, moving its position, or minimizing it through the UI provided by the system UI module (719). The system UI module (719) may be configured to provide information about the currently running application.

[0117] The generative AI module (721) can generate information using at least one AI model. For example, at least one AI model may include a large language model (LLM), a large multi-modal model (LMM), and / or a large vision model (LVM). For example, the generative AI model may include a generative adversarial network (GAN) for image generation, a variational auto encoder (VAE), and / or a diffusion-based generative model. The diffusion-based generative model may utilize a VAE and a transformer structure. For example, the artificial intelligence model for generating text data,

[0118] The generative AI module (721) can generate a prompt based on surrounding environment information (e.g., images and / or text) and generate information about the surrounding space by inputting the prompt into an AI model. For example, the generative AI module (721) can generate information about a work area based on surrounding environment information. For example, the information about the work area may include the name of the work area, the classification of the work area, the classification of the space, information about applications to be executed within the work area (e.g., names and / or classifications), a description of the work area, and / or an image of the work area.

[0119] The generative AI module (721) can identify a recommended work area corresponding to the surrounding environment based on surrounding environment information. The generative AI module (721) can generate a prompt based on the surrounding environment information and identify at least one recommended work area by comparing the result based on the generated prompt with the information of the stored work area.

[0120] For example, the workspace management module (750) may be configured to perform the storage, management, and restoration of workspaces. The workspace management module (750) may include a workspace comparison module (751), a workspace storage module (753), a workspace restoration module (755), and / or a spatial data capture module (757).

[0121] In the present disclosure, a “work area” (e.g., workspace) may be referred to as a virtual space formed by a user for a specific purpose or specific task. For example, information in the work area may include information about an executed application, the spatial configuration of the execution screen, surrounding environment information, and / or surrounding object information. For example, information about an executed application may include the name, package name, version, type, and / or purpose of the executed application. In one example, information about an executed application may include information related to the execution status (e.g., information about the last edited file, list of tabs in an internet browser). For example, the spatial configuration of the execution screen may include the relative position between the execution screen and the user, and / or the absolute position of the execution screen within the space. For example, surrounding environment information may include captured spatial data, priority of surrounding objects, spatial classification (e.g., category), and / or the relative position (e.g., height) of an electronic device (10) within the space. For example, surrounding object information may include the name, type, and / or spatial position of objects placed in the surroundings.

[0122] In one example, the information of the work area may include additional information. The additional information may include, for example, information for saving, comparing, and / or restoring the work area. For example, the additional information may include the posture of the user of the electronic device (10), the user's gestures, information of other users located nearby, a summary of the work, the name of the work area, a summary of the work area, and / or a summary image of the work area (e.g., a thumbnail). The user's posture may include, for example, standing, sitting in a chair, sitting on the floor, lying down, walking, and / or a specific posture.

[0123] The electronic device (10) may include information regarding a plurality of work areas. The information regarding the plurality of work areas may not be mutually exclusive. For example, at least a portion (e.g., an application) of the information of a first work area and the information of a second work area may match.

[0124] The work area comparison module (751) can compare context information with information of at least one work area stored in the electronic device (10) (e.g., information of an executed application, spatial configuration of an execution screen, surrounding environment information, surrounding object information, and / or additional information). For example, the context information may include information of an executed application, surrounding environment information, surrounding object information, and / or additional information. For example, the context information may include surrounding environment information obtained by the environment analysis module (713). For example, the context information may include surrounding object information obtained by the object recognition module (715). For example, the context information may include additional information obtained by the position calculation module (711) and / or the body tracking module (717) (e.g., the position and orientation of the electronic device (10) in space, the user's posture, and / or the user's gesture).

[0125] According to one embodiment, the work area comparison module (751) can perform a prioritized search for comparing work areas. For example, the work area comparison module (751) can compare work areas by comparing surrounding objects with high priority (e.g., highest). For example, the work area comparison module (751) can perform a prioritized search based on information about surrounding people. For example, there may be cases where the user works alone in the same space and cases where the user works with another person. When the user and another person are together, the electronic device (10) can prioritize searching and comparing work areas associated with the other person. When there are no other people besides the user, the electronic device (10) can prioritize searching and comparing work areas associated with the user. For example, the work area comparison module (751) can perform a prioritized search based on categories of surrounding spaces (e.g., a room in a house, a living room, a garage, an office, a library, an airplane, a bus, inside a vehicle, and / or a subway). The work area comparison module (751) can prioritize searching for and comparing work areas associated with surrounding spaces of the same category.

[0126] The electronic device (10) may determine whether to save and / or update the work area based on the comparison result of the work area comparison module (751). The electronic device (10) may determine whether to save the current user's work area based on information obtained from the environment analysis module (713), the object recognition module (715), and / or the body tracking module (717), and / or information related to application usage obtained from the system UI module (719). The electronic device (10) may save the work area periodically or based on user input. The electronic device (10) may determine whether to save the work area based on predefined conditions and / or user settings. A method for determining whether to save the work area may be described later in relation to FIGS. 8a and 8b.

[0127] When the electronic device (10) determines that a work area is to be saved, it can use a work area saving module (753) to save and / or update the current work area. The electronic device (10) can save information of the work area associated with the work area to be saved. As described above, the information of the work area may include information of the executed application, configuration within the space of the execution screen, surrounding environment information, surrounding object information, and / or additional information.

[0128] The electronic device (10) can determine whether to restore the work area based on the comparison result of the work area comparison module (751). The electronic device (10) can determine whether to restore the stored work area to the current space based on information obtained from the environment analysis module (713), the object recognition module (715), and / or the body tracking module (717), and / or information related to application usage obtained from the system UI module (719). The electronic device (10) can determine whether to restore the work area periodically or based on user input. The electronic device (10) can determine the restoration of the work area based on predefined conditions and / or user settings. A method for determining the restoration of the work area may be described later in relation to FIGS. 13a and 13b.

[0129] When the restoration of a work area is determined, the work area restoration module (755) can restore the work area based on the information of the stored work area. The restoration of the work area may include the execution of a preset application and / or the placement of the execution screen. The restoration of the work area may include the restoration of the execution state of a preset application (e.g., execution of an internet browser and access to a specific web page). For example, the work area restoration module (755) may determine a fixed reference point by comparing the information of the current space with the information of the stored work area, and place a 3D object (e.g., execution screen) corresponding to the executed application in space based on the fixed reference point. For example, the restoration of the work area may include the restoration of spatial data. The work area restoration module (755) may display the space in the current space using spatial data stored together with the work area (e.g., an image of the space and / or 3D spatial configuration information). By displaying an image of the stored space (e.g., a 2D and / or 3D image), a sense of immersion may be provided to the user of the electronic device (10).

[0130] The spatial data capture module (757) can acquire three-dimensional spatial data of the surrounding space using a camera (170) and a sensor circuit (140). The three-dimensional spatial data may include, for example, textures, vertices, a three-dimensional mesh, Gaussian information, images of the surrounding space, and / or a three-dimensional depth map. In one example, the spatial data capture module (757) can acquire three-dimensional spatial data using techniques for three-dimensional spatial reconstruction such as Gaussian splatting and Nerf. The three-dimensional spatial data may be included in the surrounding environment information described above.

[0131] In the following disclosure, the term “current work area” may be referred to as the work area currently being used by the user of the electronic device (10). The term “applied work area” may be referred to as the work area most recently applied to the current electronic device (10). For example, it may be assumed that a first work area has been applied to the electronic device (10). After the first work area is restored, the user may change the position of the execution screen as they perform a task. Depending on the change in the position of the execution screen, the first work area may be changed to a second work area. In this case, the second work area may be referred to as the “current work area,” and the first work area may be referred to as the “applied work area.” If there is no change after application, the current work area and the applied work area may be the same.

[0132] FIG. 8a is a flowchart of a novel work area storage method of an electronic device according to one embodiment.

[0133] Referring to FIG. 2 and FIG. 8a, according to one embodiment, the electronic device (10) can determine whether to save the current work area. For example, the electronic device (10) can determine whether to save based on a specified period. In the example of FIG. 8a, the work area function of the electronic device (10) may be assumed to be in a disabled state.

[0134] The operations described below in relation to FIG. 8a may be referred to as operations performed in the processor (120) of the electronic device (10) of FIG. 2. The order of the operations described below in relation to FIG. 8a is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be performed differently from the order of FIG. 8a or may be performed substantially simultaneously with other operations of FIG. 8a.

[0135] According to one embodiment, in operation 805, the electronic device (10) can determine whether the first time interval has expired. For example, the electronic device (10) can determine whether the first time interval has expired from a specific point in time. The specific point in time may include, for example, a point in time when the electronic device (10) is activated or a point in time when a specified application is executed on the electronic device (10). For example, the electronic device (10) may operate a timer having the length of the first time interval and determine that the first time interval has expired when the timer expires. The first time interval may be a preset value. For example, the first time interval may be a manufacturer or user-set value. The first time interval may be a value set based on the remaining battery level of the electronic device (10). The first time interval may be set to become longer as the remaining battery level of the electronic device (10) decreases.

[0136] If the first time interval has not expired (e.g., operation 805-NO), the electronic device (10) may wait for the first time interval to expire. If the first time interval has expired (e.g., operation 805-YES), the electronic device (10) may perform operation 810. Operation 805 is intended to explain that the electronic device (10) may perform operation 810 at a specified interval, but the electronic device (10) may perform operation 810 based on other events, such as user input, rather than at a specified interval. In this case, the electronic device (10) may omit the performance of operation 805.

[0137] According to one embodiment, in operation 810, the electronic device (10) can determine whether the current work area needs to be saved. For example, the electronic device (10) can obtain context information using the position calculation module (711), environment analysis module (713), object recognition module (715), body tracking module (717), and / or system UI module (719) of FIG. 7. The electronic device (10) can determine whether the current work area needs to be saved based on the context information.

[0138] According to one embodiment, the electronic device (10) may store a work area including the space of the electronic device (10) and the at least one application when at least one application has been executed for a specified period of time or longer. For example, if there is at least one application that has been executed for a specified period of time or longer, the electronic device (10) may determine whether to store the work area based on user interaction with the at least one application (e.g., user gaze and / or user input regarding the at least one application). For example, if the frequency of interaction during a specified period is greater than the specified frequency, the duration of the interaction is greater than the specified time, or the number of interactions is greater than the specified number, the electronic device (10) may decide to store the current work area. For example, the electronic device (10) may decide to store the current work area when the above-described conditions are satisfied and the electronic device (10) is located in a physical space during the specified period of time. Even if the electronic device (10) is in motion, if the space where the electronic device (10) is located forms a physical space (e.g., inside a moving vehicle), it can be considered as a single actual space.

[0139] According to one embodiment, the electronic device (10) may determine to save the current work area if at least one of the conditions described below is satisfied—when the change in position of an application (e.g., execution screen of an application) and / or a 3D object placed within the current work area is less than a specified distance for a specified time; when the user moves more than a certain distance from the application and / or 3D object while the placement of the application and 3D object is maintained; when the user uses the current work area for more than a specified time, performs a task for a predefined time (e.g., 30 minutes), and terminates the display of the application and / or 3D object for the whole or a specified percentage (e.g., 60%) or more (e.g., termination of execution); and / or if the electronic device (10) is a wearable device, when the wearable device is removed during a task through the work area. In addition to the conditions described above, the electronic device (10) may determine to save the current work area based on user input, user settings, and / or other work area saving conditions.

[0140] For example, if saving the current work area is determined, the electronic device (10) may determine that saving the current work area is necessary (e.g., operation 810-YES). In one example, if saving the current work area is determined, the electronic device (10) may compare the current work area with the saved work area. For example, the electronic device (10) may compare the current work area with the saved work area using the work area comparison module (751) of FIG. 7. The comparison of work areas may be described later in relation to operation 875 of FIG. 8b. If there is no saved work area corresponding to the current work area, the electronic device (10) may save the current work area. If there is a saved work area corresponding to the current work area, the electronic device (10) may update the corresponding work area based on the current work area. In one example, if saving the current work area is determined, the electronic device (10) may provide a UI to the user for saving the current work area. The UI for saving may include at least one of the details of the work area to be saved, the name of the work area, and / or an image. The electronic device (10) can save the current workspace based on user input for the provided UI.

[0141] If it is determined that the current workspace needs to be saved (e.g., operation 810-YES), in operation 815, the electronic device (10) can activate the workspace function and save the current workspace as a new workspace. For example, the electronic device (10) can generate information about the current workspace using the generative AI module (721) of FIG. 7 and save the generated information. For example, the electronic device (10) can automatically generate and save a name and / or image (e.g., thumbnail) for the current workspace.

[0142] For example, information about the work area may include information about the executed application, the spatial configuration of the execution screen, surrounding environment information (e.g., the user's relative position from an object corresponding to the application, the user's absolute position within the space, captured spatial data, classification of the space, the spatial relative position of the electronic device (10) (e.g., height)), surrounding object information (e.g., information about objects placed around the user), and / or additional information (e.g., the user's body posture, information related to people located nearby, a thumbnail corresponding to the work area, a description of the work area, and / or the name of the work area). For example, the description of the work area may include a description of the user's operation. The description of the operation may be generated based on the user's interaction and / or application information.

[0143] For example, a user of the electronic device (10) may use a composition application while playing the piano in a room. In this case, the electronic device (10) may determine the storage of the current work area in accordance with the above description in relation to operation 810. The electronic device (10) may capture spatial data of the user's room and store it in work area information, and may store the piano as surrounding object information associated with the work area. Additionally, if the user plays the piano while sitting, the electronic device (10) may store the user's sitting posture as part of the work area information.

[0144] FIG. 8b is a flowchart of a method for updating the work area of ​​an electronic device according to one embodiment.

[0145] Referring to FIG. 2 and FIG. 8b, according to one embodiment, the electronic device (10) can determine whether to update the current work area. For example, the electronic device (10) can determine whether to save based on a specified period. In the example of FIG. 8b, the work area function of the electronic device (10) may be assumed to be in an active state.

[0146] The operations described below in relation to FIG. 8b may be referred to as operations performed in the processor (120) of the electronic device (10) of FIG. 2. The order of the operations described below in relation to FIG. 8b is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be performed differently from the order of FIG. 8b, or may be performed substantially simultaneously with other operations of FIG. 8b.

[0147] According to one embodiment, in operation 855, the electronic device (10) may determine whether the second time interval has expired. The second time interval may be shorter than the first time interval of FIG. 8a. For example, the electronic device (10) may determine whether the second time interval has expired from a specific point in time. The specific point in time may include, for example, the point in time when the electronic device (10) is activated or the point in time when the work area function is activated. For example, the electronic device (10) may operate a timer having the length of the second time interval and determine that the second time interval has expired when the timer expires. The second time interval may be a preset value. For example, the second time interval may be a manufacturer or user-set value. The second time interval may be a value set based on the remaining battery level of the electronic device (10). The second time interval may be set to become longer as the remaining battery level of the electronic device (10) decreases.

[0148] If the second time interval has not expired (e.g., operation 855-NO), the electronic device (10) may wait for the second time interval to expire. If the second time interval has expired (e.g., operation 855-YES), the electronic device (10) may perform operation 860. Operation 855 is intended to explain that the electronic device (10) may perform operation 860 at a specified interval, but the electronic device (10) may perform operation 860 based on other events, such as user input, rather than at a specified interval. In this case, the electronic device (10) may skip performing operation 855.

[0149] According to one embodiment, in operation 860, the electronic device (10) may determine whether saving the current work area is necessary. For example, the electronic device (10) may determine whether to save the current work area in accordance with the above description in relation to operation 810. The electronic device (10) may determine to save the current work area if at least one of the conditions described below is satisfied—when the change in position of an application (e.g., execution screen of an application) and / or a 3D object placed within the current work area is less than a specified distance for a specified time; when the user moves more than a certain distance from the application and / or 3D object while the placement of the application and 3D object is maintained; when the user uses the current work area for more than a specified time, performs a task for a predefined time (e.g., 30 minutes), and terminates the display of the application and / or 3D object for the whole or a specified percentage (e.g., 60%) or more (e.g., termination of execution); and / or if the electronic device (10) is a wearable device, when the wearable device is removed during a task through the work area. In addition to the conditions described above, the electronic device (10) may determine the storage of the current workspace based on user input, user settings, and / or other workspace storage conditions.

[0150] If saving the current work area is required (e.g., operation 860-YES), in operation 865, the electronic device (10) can determine whether the current work area corresponds to an applied work area (e.g., the most recently applied work area).

[0151] For example, the similarity between the current work area and the applied work area may be determined based on the spatial type of the current work area and the applied work area, captured spatial data, surrounding objects (e.g., objects located in front of the user), the user's posture, and / or the height of the electronic device (10). If the similarity between the current work area and the applied work area exceeds a threshold, the electronic device (10) may determine that the current work area corresponds to the applied work area. If the similarity between the current work area and the applied work area is less than the threshold, the electronic device (10) may determine that the current work area does not correspond to the applied work area.

[0152] If the current work area corresponds to the applied work area (e.g., operation 865-YES), in operation 870, the electronic device (10) can update the applied work area based on the current work area. The electronic device (10) can update the information of the stored applied work area using the work area information of the current work area. After updating the work area information, the electronic device (10) can wait for a second time interval according to reference point A.

[0153] If the current work area does not correspond to the applied work area (e.g., operation 865-NO), in operation 875, the electronic device (10) can determine whether the current work area corresponds to a stored work area. The electronic device (10) can determine whether the current work area corresponds to a stored work area based on information of the executed application, the spatial configuration of the execution screen, surrounding environment information, surrounding object information, and / or additional information.

[0154] For example, the electronic device (10) may determine the similarity between the current work area and the stored work area based on the spatial type of the current work area and the applied work area, captured spatial data, surrounding objects (e.g., objects located in front of the user), the user's posture, and / or the height of the electronic device (10). If the similarity between the current work area and the stored work area exceeds a threshold, the electronic device (10) may determine that the current work area corresponds to the stored work area. If there is no stored work area for which the similarity with the current work area exceeds the threshold, the electronic device (10) may determine that the current work area does not correspond to the stored work area.

[0155] If the current work area does not correspond to the stored work area (e.g., operation 875-NO), in operation 885, the electronic device (10) may store the current work area as a new work area. For example, the electronic device (10) may store the new work area as described above in relation to operation 815 of FIG. 8a. After storing the new work area information, the electronic device (10) may wait for a second time interval according to reference point A. In this case, the new work area may correspond to the current work area and the applied work area.

[0156] If the current work area corresponds to a stored work area (e.g., operation 875-YES), in operation 880, the electronic device (10) can update the corresponding stored work area based on the current work area. The electronic device (10) can update the information of the corresponding stored work area based on information of the executed application, spatial configuration of the execution screen, surrounding environment information, surrounding object information, and / or additional information. After updating the work area information, the electronic device (10) can wait for a second time interval according to reference point A.

[0157] If saving the current workspace is not required (e.g., operation 860-NO), in operation 890, the electronic device (10) may determine whether to disable the workspace function. For example, the electronic device (10) may determine to disable the workspace function if the user's work is interrupted, if the electronic device (10) is unworn, or if the application in the workspace is terminated. For example, the electronic device (10) may ask the user whether to disable the workspace and identify the disablement of the workspace based on the user's response. If the disablement of the workspace is determined (e.g., operation 890-YES), in operation 895, the electronic device (10) may disable the workspace function. If the maintenance of the workspace's activation is determined (e.g., operation 890-NO), the electronic device (10) may wait for a second time interval according to reference point A. In one example, if storage of the work area is not required (e.g., operation 860-NO), the electronic device (10) does not perform operation 890, and the electronic device (10) may wait for a second time interval according to reference point A.

[0158] As described below in relation to FIGS. 13a and 13b, in one example, the electronic device (10) may apply another stored work area instead of the current work area if restoration of another stored work area is required.

[0159] Figure 9 illustrates an example of a real space.

[0160] Referring to FIGS. 2 and FIGS. 9, for example, the real space (900) of FIGS. 9 may correspond to an image of the real space shown through the display (160) of the electronic device (10). For example, the real space (900) may correspond to an image acquired and displayed by a camera (170) or an image shown through the display (160) of the electronic device (10). In the example of FIGS. 9, the real space (900) may include a music keyboard (910), a desk (920), a light (930), a flowerpot (940), and a picture frame (950).

[0161] According to one embodiment, the electronic device (10) can recognize surrounding objects placed in the surrounding environment using a camera (170) and a sensor circuit (140). For example, the electronic device (10) can recognize a music keyboard (910), a desk (920), a light (930), a flowerpot (940), and a picture frame (950) based on image recognition.

[0162] According to one embodiment, the electronic device (10) can classify and store recognized surrounding objects according to categories. For example, the electronic device (10) can classify surrounding objects based on their use. The use of the surrounding objects may typically refer to the use and / or purpose for which the object is widely used. For example, the use of a music keyboard (910) may be music. The use of a flowerpot (940) and a picture frame (950) may be decoration. The use of a desk (920) may be furniture. In one example, the electronic device (10) may use the categories of surrounding objects when comparing two work areas. Instead of comparing the recognized objects themselves, by determining similarity based on the use of the objects, the electronic device (10) can identify two work areas that have similar purposes.

[0163] According to one embodiment, the electronic device (10) can store priority information regarding surrounding objects. The electronic device (10) can determine priority based on interaction with surrounding objects and / or association of work with surrounding objects.

[0164] For example, the electronic device (10) can detect interactions between a user and surrounding objects using the body tracking module (717) of FIG. 7. For example, interactions may include the user’s gaze toward an object, manipulation of the object by the user, placement of a graphic object adjacent to an object within a work area by the user, and / or designation of the object by the user. The electronic device (10) can determine priorities based on the frequency, number, and / or time of the interactions. If the frequency of interactions with a specific object is high, the number is large, and / or the time is long, the electronic device (10) can set a relatively high priority for the specific object.

[0165] For example, the electronic device (10) can obtain information about the currently running application using the system UI module (719) of FIG. 7. If there is a surrounding object that has a use and / or purpose highly related to the use and / or purpose of the running application, the electronic device (10) can set a relatively high priority for the surrounding object.

[0166] In the example of FIG. 9, the user may be playing a music keyboard (910). Based on the interaction of playing the music keyboard (910), the electronic device (10) may set the priority for the music keyboard (910) higher than that of the other objects. In the example of FIG. 9, the user may be composing music. The electronic device (10) may set the priority for the music keyboard (910), which is highly associated with the purpose of composing music, higher than that of the other objects.

[0167] For example, the electronic device (10) can store the priority of surrounding objects within the surrounding environment information. As another example, the electronic device (10) can store the priority information of surrounding objects as part of the object information. The priority information of surrounding objects can be stored as information of the corresponding work area. For example, the priority of the music keyboard (910) may be set high for the first work area, but the priority of the music keyboard (910) may be set low for the second work area.

[0168] According to one embodiment, the electronic device (10) can compare work areas based on the priority of surrounding objects. For example, the electronic device (10) can identify an object that has the highest priority among each work area and determine that two work areas are similar if the two objects are the same or the categories of the two objects are the same.

[0169] FIG. 10 illustrates an extended reality according to one embodiment.

[0170] Referring to FIGS. 2 and FIGS. 10, for example, the augmented reality (1000) may include at least one graphic object placed in real space. For example, the augmented reality (1000) may include an application A (1001), an application B (1002), an application C (1003), an application D (1004), an application E (1005), and / or an application F (1006) placed adjacent to the music keyboard (910). For example, the applications may correspond to the execution screen and / or graphic object of an application placed by the user. For example, the user of the electronic device (10) may perform tasks related to composition using the music keyboard (910).

[0171] According to one embodiment, the electronic device (10) may store information about a work area within the extended reality. For example, the electronic device (10) may store information about a work area as described above in relation to operation 815 of FIG. 8a. In this case, the applied work area may include AF applications (1001-1006).

[0172] FIG. 11a illustrates a work area changed from the work area of ​​FIG. 10.

[0173] Referring to FIG. 2 and FIG. 11a, the modified work area (1100) of FIG. 11 may have a different configuration from the work area of ​​the augmented reality (1000) of FIG. 10. For example, as the user proceeds with the work, the user may change the location of the application and terminate some applications.

[0174] According to one embodiment, the electronic device (10) can update information of the applied work area based on information of the changed work area (1100) (e.g., current work area). For example, the information of the updated work area may include applications A, C, E, and F (1001, 1003, 1005, 1006). For example, the electronic device (10) can update the work area based on operation 870 of FIG. 8b. Depending on the update of the work area, the updated work area (1100) may correspond to the current work area and the applied work area.

[0175] FIG. 11b illustrates an update UI (user interface) according to one embodiment.

[0176] Referring to FIG. 2 and FIG. 11b, according to one embodiment, an electronic device (10) may provide an update UI (1110) based on an update of a work area. For example, the electronic device (10) may display the update UI (1110) when an update of the work area is determined.

[0177] For example, the update UI (1110) may include information (1111) of the workspace being updated (e.g., the updated workspace (1100)). The information (1111) may include a summary, description, and / or title of the updated workspace (1100). In one example, the electronic device (10) may generate the information (1111) using a generative artificial intelligence model.

[0178] For example, the update UI (1110) may include a first affordance (1113). When the electronic device (10) receives input for the first affordance (1113), it may change the update method from automatic update to manual update. For example, when the electronic device (10) receives input for the first affordance (1113), it may provide the UI described below in relation to FIG. 11d and / or FIG. 11e. In the case of automatic update, the electronic device (10) may save the current workspace (e.g., the updated workspace (1100)). In the case of manual update, the electronic device (10) may change the components of the saved workspace based on user input.

[0179] FIG. 11c illustrates a work area save UI according to one embodiment.

[0180] Referring to FIG. 2 and FIG. 11c, according to one embodiment, the electronic device (10) may provide a save UI (1120) based on an update of the work area. For example, the electronic device (10) may display the save UI (1120) when an update of the work area is determined.

[0181] For example, the save UI (1120) may include information (1121) of the updated workspace (e.g., the updated workspace (1100)). The information (1121) may include a summary, description, and / or title of the updated workspace (1100). In one example, the electronic device (10) may generate the information (1121) using a generative artificial intelligence model.

[0182] For example, the save UI (1120) may include a second affordance (1123) and a third affordance (1125). When an input for the second affordance (1123) is received, the electronic device (10) may cancel the update of the workspace. In one example, when an input for the third affordance (1125) is received, the electronic device (10) may update the information of the workspace based on the current workspace. In one example, when an input for the third affordance (1125) is received, the electronic device (10) may provide the update UI (1110) of FIG. 11b, the application-based workspace editing UI of FIG. 11d, or the object-based workspace editing UI of FIG. 11e.

[0183] For example, the save UI (1120) may include additional configurations not shown in FIG. 11c. For example, the save UI (1120) may include a graphic object for editing the work area to be saved. For example, the electronic device (10) may provide the editing UI of FIG. 11d or 11e when input for the graphic object is received, and may save the current work area as is when input for the third affordance (1125) is received. In one example, the electronic device (10) may provide an interface for editing the work area information (1121) when input for the work area information (1121) is received. For example, the electronic device (10) may provide a voice input interface and / or a virtual keyboard and may modify the work area information (1121) based on user input.

[0184] FIG. 11d illustrates an application-based workspace editing UI according to one embodiment.

[0185] Referring to FIG. 2 and FIG. 11d, according to one embodiment, the electronic device (10) may provide an application-based workspace editing UI based on an update of the workspace. For example, the electronic device (10) may display an application-based workspace editing UI when an update of the workspace is determined while the device is set to manual update.

[0186] An application-based workspace editing UI may include an editing UI (1130). The editing UI (1130) may include a fourth affordance (1131) and a fifth affordance (1133). While the editing UI (1130) is provided, the electronic device (10) may provide an editing affordance (1190) for editing for each application. For example, when the electronic device (10) receives input for the editing affordance (1190) of application A (1001), it may exclude application A (1001) from the information of the workspace to be stored. When input for the fourth affordance (1131) is received, the electronic device (10) may stop updating the workspace and terminate editing the workspace. When input for the fifth affordance (1133) is received, the electronic device (10) may perform an update of the workspace and terminate editing the workspace. Based on the end of editing of the work area, the electronic device (10) can stop displaying the application-based work area editing UI.

[0187] An application-based workspace editing UI may include a trigger type UI (1140). In one example, the electronic device (10) may display the trigger type UI (1140) during editing of the workspace. The trigger type UI (1140) may include an object affordance (1141) and an application affordance (1143). The trigger type UI (1140) may indicate the type of trigger used for editing the current workspace. In the example of FIG. 11d, the trigger type UI (1140) may indicate that editing of the workspace based on the application is being performed. Based on input to the trigger type UI (1140), the electronic device (10) may change the type of trigger that is the target of editing the workspace. For example, when input to the trigger type UI (1140) is received, the electronic device (10) may display the object-based workspace editing UI of FIG. 11e.

[0188] FIG. 11e illustrates an object-based workspace editing UI according to one embodiment.

[0189] Referring to FIG. 2 and FIG. 11e, according to one embodiment, the electronic device (10) may provide an object-based workspace editing UI based on an update of the workspace. For example, the electronic device (10) may display an object-based workspace editing UI when an update of the workspace is determined while the device is set to manual update.

[0190] The object-based workspace editing UI may include an editing UI (1130) and a trigger UI (1140). While the editing UI (1130) is provided, the electronic device (10) may provide an editing affordance (1190) for editing for each object. For example, when the electronic device (10) receives input regarding the editing affordance (1190) of a picture frame (950), the picture frame (950) may be excluded from the information of the workspace to be stored. In the example of FIG. 11e, the trigger type UI (1140) may indicate that editing of the object-based workspace is performed.

[0191] In the examples of FIGS. 11d and 11e, the electronic device (10) can change the display type of the object to which an input for an editing affordance (1190) is received. In the example of FIG. 11d, when an input for an editing affordance (1190) of application A (1001) is received, the electronic device (10) can display application A (1001) in a translucent state while editing is being performed. When an input for application A (1001) in a translucent state is received, the electronic device (10) can include application A (1001) in the information of the work area to be saved and display application A in an opaque state. In the example of FIG. 11e, when an input for an editing affordance (1190) of a picture frame (950) is received, the electronic device (10) can display a visual effect for the picture frame (950) while editing is being performed. For example, the electronic device (10) may display an image of a specified color overlaid on an area corresponding to the frame (950), or display the outline of the frame (950) in a specified color. Then, when input regarding the frame (950) is received again, the electronic device (10) may include the frame (950) again in the information of the work area to be stored.

[0192] In the example implemented in relation to FIGS. 11c to 11e, the electronic device (10) may terminate the editing if no user input is received for more than a specified time while the editing is not completed, or if the electronic device (10) leaves its current location. For example, upon termination of editing, the electronic device (10) may save the currently edited work area or cancel the editing of the work area.

[0193] FIG. 12 illustrates an immersive type work area according to one embodiment.

[0194] Referring to FIGS. 2 and FIGS. 12, according to one embodiment, the electronic device (10) can display a background image (1210) of a work area. In the examples of FIGS. 11b through 11e, an example is described in which the electronic device (10) displays a virtual object on an image of real space. In this case, the electronic device (10) can operate in a pass-through mode. In the example of FIG. 12, the electronic device (10) can display a background image (1210) on at least a portion of the image of real space. In this case, the electronic device (10) can operate in an immersion mode.

[0195] In one example, the background image (1210) may include a designated image or an image of a stored space. For example, the background image (1210) may correspond to an image of the space at the time the applied work area is stored. For example, the electronic device (10) may reconstruct the space based on spatial data (e.g., an image of the space and / or three-dimensional spatial configuration information) of the image of the space at the time the applied work area is stored, and display the reconstructed space as a background image (1210).

[0196] In one example, the electronic device (10) may display a background image (1210) based on the direction of the user's gaze or the direction of the user's face. When the direction of the user's gaze or the direction of the user's face goes beyond a certain range within the background image (1210) (e.g., the boundary of the background image (1210)), the electronic device (10) may adjust the display method of the background image (1210) so that an image of the actual space is shown. For example, the electronic device (10) may increase the transparency of the background image (1210) or stop the display of the background image (1210).

[0197] The electronic device (10) can store information of an applied background image (1210) in the information of the work area in the storage of the work area.

[0198] FIG. 13a is a flowchart of a method for restoring a work area according to one embodiment.

[0199] Referring to FIG. 2 and FIG. 13a, according to one embodiment, an electronic device (10) may determine whether to restore a work area. For example, the electronic device (10) may determine whether to restore based on a specified period. In the example of FIG. 13a, the work area function of the electronic device (10) may be assumed to be in a disabled state. In the example of FIG. 13a, restoration of the work area by the electronic device (10) may include applying a work area stored within the current extended reality.

[0200] The operations described below in relation to FIG. 13a may be referred to as operations performed in the processor (120) of the electronic device (10) of FIG. 2. The order of the operations described below in relation to FIG. 13a is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be performed differently from the order of FIG. 13a or may be performed substantially simultaneously with other operations of FIG. 13a.

[0201] According to one embodiment, in operation 1305, the electronic device (10) can determine whether the third time interval has expired. For example, the electronic device (10) can determine whether the third time interval has expired from a specific point in time. The specific point in time may include, for example, the point in time when the electronic device (10) is activated or the point in time when a specified application is executed on the electronic device (10). For example, the electronic device (10) may operate a timer having the length of the third time interval and determine that the third time interval has expired when the timer expires. The third time interval may be a preset value. For example, the third time interval may be a manufacturer or user-set value. The third time interval may be a value set based on the remaining battery level of the electronic device (10). The third time interval may be set to become longer as the remaining battery level of the electronic device (10) decreases.

[0202] If the third time interval has not expired (e.g., operation 1305-NO), the electronic device (10) may wait for the third time interval to expire. If the third time interval has expired (e.g., operation 1305-YES), the electronic device (10) may perform operation 1310. Operation 1305 is intended to explain that the electronic device (10) may perform operation 1310 at a specified interval, but the electronic device (10) may perform operation 1310 based on other events, such as user input, rather than at a specified interval. In this case, the electronic device (10) may skip performing operation 1305.

[0203] According to one embodiment, in operation 1310, the electronic device (10) can determine whether restoration of the work area is required. For example, the electronic device (10) can obtain context information using the position calculation module (711), environment analysis module (713), object recognition module (715), body tracking module (717), and / or system UI module (719) of FIG. 7. The electronic device (10) can determine whether to restore the current work area based on the context information.

[0204] The electronic device (10) may determine the need for restoration of a work area based on at least one positive condition. For example, the electronic device (10) may determine the restoration of a work area corresponding to an object associated with at least one of the stored work areas where an identified surrounding object is associated. For example, the electronic device (10) may determine the restoration of a work area if the location of the electronic device (10) corresponds to a location associated with at least one of the stored work areas. For example, the electronic device (10) may determine the restoration of a work area if the user's posture of the electronic device (10) and the location of the electronic device (10) correspond to at least one of the stored work areas. For example, the electronic device (10) may determine the restoration of a work area when a user input instructing the restoration of a work area is received. For example, the electronic device (10) may determine the restoration of a work area if the executed application is an application associated with at least one of the stored work areas.

[0205] The electronic device (10) may determine the need to restore the work area based on at least one negative condition. In one example, the electronic device (10) may determine not to perform restoration of the work area if the function of the work area is active. In one example, the electronic device (10) may determine not to restore the work area if there is a previously executed application (e.g., a specified application). If at least one negative condition is satisfied, the electronic device (10) may determine not to perform restoration of the work area even if other positive conditions are satisfied.

[0206] The electronic device (10) may decide not to perform restoration of the work area if at least N of the above-described positive conditions are not satisfied or if at least one negative condition is satisfied. In this case (e.g., operation 1310-NO), the electronic device (10) may wait for a third time interval. The electronic device (10) may decide that restoration of the work area is required if at least N (e.g., N is a natural number) of the above-described positive conditions are satisfied and the negative condition is not satisfied. In this case (e.g., operation 1310-YES), the electronic device (10) may perform operation 1315.

[0207] In operation 1315, the electronic device (10) can activate the work area function and restore the work area based on specified conditions. For example, the electronic device (10) can restore the work area if at least M (e.g., M is a natural number) of the specified conditions described below are satisfied—when the spatial classification of the current space and the spatial classification of the stored work area are the same; when the similarity between the captured spatial information of the current space and the spatial information of the stored work area exceeds a threshold; when objects around the user of the electronic device (10) are similar to objects in the stored work area; and / or when the posture of the current user and / or the height of the electronic device (10) is similar to that of the stored work area. For example, the electronic device (10) can identify a work area satisfying the specified conditions by comparing the current work area and the stored work area using the work area comparison module (751) of FIG. 7. The conditions described above in relation to operation 1310 may be referred to as conditions for the electronic device (10) to trigger the restoration of the work area. The specified conditions described in relation to operation 1315 may be referenced as conditions for determining the work area corresponding to the current space.

[0208] For example, as described above, the electronic device (10) may store spatial classification information (e.g., category information) of the work area as surrounding environment information when storing the work area. For example, the spatial classification information may be generated based on an artificial intelligence model or specified by a user. The electronic device (10) may identify the spatial classification of the current space. For example, the electronic device (10) may determine the spatial classification of the current space based on context information. In one example, the electronic device (10) may identify the spatial classification by inputting context information into an artificial intelligence model. By comparing the spatial classification of the current space with the spatial classification of the stored work area, the electronic device (10) may determine the similarity between the current space and the work area based on the spatial classification information. In one example, the electronic device (10) may determine that the spatial classification of the current space and the spatial classification of the stored work area are the same if the similarity exceeds a specified value. The spatial classification may include, for example, a room in a house, a living room, a garage, an office, a library, an airplane, a bus, inside a vehicle, and / or a subway. The electronic device (10) can identify at least one work area having the same spatial classification as the spatial classification of the current space.

[0209] For example, the electronic device (10) can capture spatial information of the current space using the spatial data capture module (757) of FIG. 7. The electronic device (10) can identify similarity between the captured spatial information and the spatial information of the stored work area. For example, the electronic device (10) can identify similarity based on any algorithm for calculating similarity. The electronic device (10) can identify at least one work area among a plurality of work areas that has a similarity with the current space exceeding a threshold value.

[0210] For example, the electronic device (10) can identify surrounding objects in the current space. The electronic device (10) can identify a work area containing an object corresponding to the identified surrounding objects. For example, an object with the same purpose as the identified surrounding objects can be referenced as an object corresponding to the surrounding objects.

[0211] For example, the electronic device (10) can identify a work area containing a corresponding object based on a high-priority object among surrounding objects in the current space. For example, the electronic device (10) can determine an object located in front of the user of the electronic device (10) and / or an object with a high frequency of interaction as a high-priority object. For example, if multiple objects are located in front of the user, the electronic device (10) can identify the highest-priority object among the multiple objects based on the priority or interaction of the pre-set objects. The electronic device (10) can identify at least one work area containing an object corresponding to the high-priority object by comparing the high-priority object with objects associated with stored work areas.

[0212] For example, the electronic device (10) can identify the user's posture using the body tracking module (717) of FIG. 7. The electronic device (10) can identify the height of the electronic device (10) using the position calculation module (711) of FIG. 7. The electronic device (10) can identify at least one work area having a posture and height similar to the identified posture and height.

[0213] If a single work area satisfying a specified condition is identified, the electronic device (10) can perform restoration on the identified work area. If multiple work areas satisfying a specified condition exist, the electronic device (10) can attempt restoration based on the similarity between the multiple work areas and the current space. For example, the electronic device (10) can restore the work area having the highest similarity among the multiple work areas.

[0214] In restoring a work area, the electronic device (10) may inquire with the user whether to restore it. The electronic device (10) may restore the work area based on the reception of user input instructing restoration. If there are multiple work areas satisfying specified conditions, the electronic device (10) may provide information on the multiple work areas and restore the work area selected by the user. The information on the multiple work areas may be sorted, for example, based on similarity.

[0215] If there is no work area satisfying the specified conditions described above, the electronic device (10) may not perform restoration of the work area.

[0216] The electronic device (10) can identify a work area to be restored and restore the identified work area. For example, the electronic device (10) can restore the work area based on the work area information of the identified work area. The electronic device (10) can execute and place an application based on the information of the application to be executed in the work area and the location of the application. The electronic device (10) can place an application based on the location of surrounding objects. For example, if the execution screen of application A in the saved work area is placed adjacent to object B, the electronic device (10) can place the execution screen of application A adjacent to object C in real space (e.g., an object corresponding to object B).

[0217] The electronic device (10) can restore a spatial image of a work area. For example, the electronic device (10) can restore a spatial image of a work area by displaying a stored spatial image of a work area, such as the background image (1210) of FIG. 12. In restoring the spatial image, the electronic device (10) can perform spatial image restoration for a full-area space or a partial space (e.g., a part in front of the electronic device (10)). For example, the spatial image restoration range may be limited by the range of the stored spatial image. For example, the spatial image restoration range may be based on user settings. In one example, the electronic device (10) may receive input for setting the spatial image restoration range (e.g., input via a physical interface or external electronic device and / or input to a 3D UI).

[0218] The electronic device (10) can restore the work area based on spatial data of the work area. For example, the work area information may indicate that the execution screen of application A is located on the front wall. In this case, the electronic device (10) can place the execution screen of application A on the front wall of the actual space.

[0219] The electronic device (10) can restore a work area based on an absolute position in space or a position relative to the user. For example, the electronic device (10) can use an absolute position or a relative position depending on the setting of the work area to be restored. For example, the electronic device (10) can use an absolute position or a relative position based on the mobility of the electronic device (10). For example, if the mobility of the electronic device (10) exceeds a threshold value, the electronic device (10) can use a relative position.

[0220] When restoring a workspace, the electronic device (10) may terminate or run applications running in the foreground before restoration. If an application running in the foreground is included in the workspace to be restored, the electronic device (10) may not terminate or switch the running application to the background.

[0221] In one example, the workspace to be restored may include an application deleted from the electronic device (10). In this case, the electronic device (10) may restore the workspace excluding the deleted application, or provide a notification providing information about the deleted application. For example, the electronic device (10) may display a UI that provides a download of the deleted application. If the UI containing information about the deleted application is terminated from display according to user input without re-downloading the deleted application, the electronic device (10) may update the information of the workspace so that the workspace no longer contains information about the deleted application.

[0222] According to one embodiment, the electronic device (10) can restore the execution state of an application to restore a work area. For example, the execution state of the application before restoration and the execution state of the application to be restored may be different. In this case, the electronic device (10) can perform the restoration of the execution state based on user settings or user input.

[0223] For example, a workspace may be restored while websites A and B are open through a web browser. The workspace to be restored may include web pages corresponding to websites B and C. During the process of restoring the workspace, the electronic device (10) may provide the user with an inquiry regarding whether to restore the websites. For example, the electronic device (10) may inquire whether to 1) open website C in addition to the currently open websites A and B, 2) maintain the currently open websites A and B and update the information in the workspace according to websites A and B, 3) close the currently open websites A and B and open websites B and C, and / or 4) open websites B and C and reopen websites A and B upon closing the workspace.

[0224] When input regarding the inquiry described above is received, the electronic device (10) may inquire whether to perform the same method of processing for similar situations in the future. When user input agreeing to the same method of processing is received, the electronic device (10) may apply a user-configured method to the restoration of applications with different execution states. When user input rejecting the same method of processing is received, the device may inquire of the user's choice whenever the restoration of applications with different execution states is performed.

[0225] FIG. 13b is a flowchart of a method for restoring a work area according to one embodiment.

[0226] Referring to FIG. 2 and FIG. 13b, according to one embodiment, the electronic device (10) can determine whether to restore the applied work area. For example, the electronic device (10) can determine whether to restore based on a specified period. In the example of FIG. 13b, the work area function of the electronic device (10) may be assumed to be in an activated state.

[0227] The operations described below in relation to FIG. 13b may be referred to as operations performed in the processor (120) of the electronic device (10) of FIG. 2. The order of the operations described below in relation to FIG. 13b is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be performed differently from the order of FIG. 13b or may be performed substantially simultaneously with other operations of FIG. 8b.

[0228] According to one embodiment, in operation 1350, the electronic device (10) can determine whether the fourth time interval has expired. The fourth time interval may be shorter than the third time interval of FIG. 13a. For example, the electronic device (10) may determine whether the fourth time interval has expired from a specific point in time. The specific point in time may include, for example, the point in time when the electronic device (10) is activated, the point in time when the work area function is activated, or the point in time when the applied work area is applied. For example, the electronic device (10) may operate a timer having the length of the fourth time interval and determine that the fourth time interval has expired when the timer expires. The fourth time interval may be a preset value. For example, the fourth time interval may be a manufacturer or user-set value. The fourth time interval may be a value set based on the remaining battery level of the electronic device (10). The fourth time interval may be set to become longer as the remaining battery level of the electronic device (10) decreases.

[0229] If the fourth time interval has not expired (e.g., operation 1350-NO), the electronic device (10) may wait for the fourth time interval to expire. If the fourth time interval has expired (e.g., operation 1350-YES), the electronic device (10) may perform operation 1355. Operation 1350 is intended to explain that the electronic device (10) may perform operation 1355 at a specified interval, but the electronic device (10) may perform operation 1355 based on other events, such as user input, rather than at a specified interval. In this case, the electronic device (10) may omit the performance of operation 1350.

[0230] According to one embodiment, in operation 1355, the electronic device (10) can determine whether restoration of the work area is required. For example, the electronic device (10) can compare the current work area with the applied work area. For example, the similarity between the current work area and the applied work area can be determined based on the spatial type of the current work area and the applied work area, captured spatial data, surrounding objects (e.g., objects located in front of the user), the user's posture, and / or the height of the electronic device (10).

[0231] The electronic device (10) may determine that restoration of the work area is not necessary if the similarity between the current work area and the applied work area exceeds a threshold. The electronic device (10) may determine that restoration of the work area is not necessary if user input rejecting restoration of the work area is received. In this case (e.g., operation 1355-NO), the electronic device (10) may wait for a fourth time interval according to reference point B.

[0232] The electronic device (10) may determine that restoration of the work area is necessary if the similarity between the current work area and the applied work area is below a threshold value. The electronic device (10) may determine that restoration of the work area is necessary if user input instructing restoration of the work area is received. In this case (e.g., operation 1355-YES), the electronic device (10) may perform operation 1360.

[0233] In operation 1360, the electronic device (10) can restore a work area based on specified conditions. For example, the electronic device (10) can restore a work area based on work area information of an applied work area. For example, the electronic device (10) can identify a work area satisfying the specified conditions described above in relation to operation 1315 of FIG. 13a, and restore a work area based on work area information of the identified work area.

[0234] According to one embodiment, the electronic device (10) can independently perform the storage of a work area and the restoration of a work area. For example, a user may start a composition work while performing a task. In this case, the electronic device (10) may identify a composition work area as the work area to be restored and identify a work work area as the work area to be saved. For example, the electronic device (10) may temporarily store information of the work area to be stored in memory (130), and when the storage of the work area is determined, it may use the temporary data stored in memory (130) to store the corresponding work area in memory (130). In one example, the electronic device (10) may substantially simultaneously perform the storage of a work area and the restoration of a work area. To prevent information of the work area currently being restored from being included in the information of the stored work area, the electronic device (10) may perform the restoration of the work area after completing the storage of the existing work area, or exclude the information of the work area to be restored from the information of the work area to be saved based on the information of the work area to be restored.

[0235] FIG. 14a illustrates a recommendation UI according to one embodiment.

[0236] Referring to FIG. 2 and FIG. 14a, according to one embodiment, an electronic device (10) can display a recommendation UI (1460) within an extended reality (1401). For example, the electronic device (10) can display the recommendation UI (1460) when a work area to be restored is identified (e.g., when a work area satisfying the specified conditions of operation 1315 of FIG. 13a is identified).

[0237] For example, a piano (1410) may be located in the real space of the extended reality (1401). For example, the electronic device (10) may determine the restoration of the work area based on the user's posture when the user is sitting in front of the piano (1410) or playing the piano (1410) (action 1310-YES in FIG. 13a).

[0238] The electronic device (10) can identify a work area containing a surrounding object corresponding to the piano (1410) as the work area to be restored. For example, the electronic device (10) can identify the work area of ​​FIG. 10 as the work area to be restored. The electronic device (10) can identify a work area containing a music keyboard corresponding to the piano (1410) intended for music as the work area to be restored.

[0239] For example, the recommendation UI (1460) may include information about the workspace to be restored. The information about the workspace may include an image (1461) of the workspace (e.g., a thumbnail) and / or a title (1463) of the workspace.

[0240] For example, the recommendation UI (1460) may include a spatial image restoration affordance (1465) that indicates whether to restore the spatial image. If the spatial image restoration affordance (1465) indicates the restoration of the spatial image, the electronic device (10) may restore the spatial image of the work area to be restored together.

[0241] The electronic device (10) can determine whether to restore the workspace based on input to the fifth affordance (1467). For example, if user input indicates canceling the restoration of the workspace, the electronic device (10) can close the display of the recommendation UI (1460) and cancel the restoration of the workspace. For example, if user input indicates restoring the workspace, the electronic device (10) can close the display of the recommendation UI (1460) and restore the workspace. For example, the electronic device (10) can restore the workspace as in the example of FIG. 14b.

[0242] FIG. 14b illustrates a restored work area according to one embodiment.

[0243] Referring to FIG. 2 and FIG. 14b, according to one embodiment, the electronic device (10) can restore a work area. The electronic device (10) can restore and provide a work area within the extended reality (1402). For example, the electronic device (10) can restore the work area described above in relation to FIG. 10.

[0244] Restoration of the work area may include the execution of an application and the arrangement of the execution screen. In the example of FIG. 14b, the electronic device (10) may execute an application according to the execution application of the work area of ​​FIG. 10. The electronic device (10) may arrange the execution screen according to the arrangement of the execution screen of FIG. 10. In the example of FIG. 10, the execution screen is arranged above the music keyboard (910). In the example of FIG. 14b, the electronic device (10) may arrange the execution screens above the piano (1410).

[0245] FIGS. 15a and 15b illustrate examples of the restoration of a working area of ​​an electronic device according to one embodiment.

[0246] Referring to FIG. 15a, within the extended reality (1501), a work area (e.g., the work area of ​​FIG. 10) may be applied to a first location (1510) (e.g., an absolute location) in space. For example, as the user of the electronic device (10) moves to a different location, the electronic device (10) may move away from the first location (1510).

[0247] Referring to FIG. 15b, within the extended reality (1502), the location of the work area can be changed to a second location (1520). For example, the electronic device (10) can determine the restoration of the work area and restore the work area by changing the location of the work area to a second location (1520).

[0248] In one example, the electronic device (10) can restore the work area by changing the absolute position of the work area within the extended reality (1502). In this case, the restored work area may be assumed to be fixed at a second position (1520).

[0249] In one example, the electronic device (10) can change the location of the work area to a second location (1520) by changing the fixed reference point of the work area to a relative location (e.g., a location relative to the user). The electronic device (10) can restore the work area by changing the fixed reference point from an absolute location to a relative location. When a fixed reference point based on a location relative to the user's location (e.g., the location of the electronic device (10)) is used, the work area can be moved within the extended reality according to the movement of the electronic device (10).

[0250] The electronic device (10) can determine a change in the fixed reference point based on mobility. For example, if the mobility of the electronic device (10) exceeds a threshold value, the electronic device (10) can change the fixed reference point from an absolute position to a relative position.

[0251] The electronic device (10) can change the fixed reference point based on the location of the electronic device (10). For example, if the electronic device (10) is located in a space with a long dwell time or low mobility, such as a home or office, the electronic device (10) can set the fixed reference point of the work area based on the absolute location. For example, if the electronic device (10) is located in an external space, the fixed reference point of the work area can be set based on the relative location (e.g., relative location to the user). In one example, the electronic device (10) can use the fixed reference point based on the absolute location in a pre-configured space (e.g., user-configured space), and the fixed reference point based on the relative location in other spaces.

[0252] FIG. 16 is a flowchart of a method for providing a recommended workspace for an electronic device according to one embodiment.

[0253] Referring to FIG. 2 and FIG. 16, according to one embodiment, an electronic device (10) may provide a recommended workspace (e.g., a recommended work area). For example, the electronic device (10) may identify a workspace to be recommended (e.g., a work area) and provide the identified recommended workspace.

[0254] The operations described below in relation to FIG. 16 may be referred to as operations performed in the processor (120) of the electronic device (10) of FIG. 2. The order of the operations described below in relation to FIG. 16 is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be performed differently from the order of FIG. 16 or may be performed substantially simultaneously with other operations of FIG. 16.

[0255] For example, the electronic device (10) may include at least one sensor (e.g., sensor circuit (140), camera (170), microphone (e.g., first microphone (182) and / or second microphone (183)), memory (130), and at least one processor (e.g., processor (120)). At least one processor may be communicated to at least one sensor (e.g., sensor circuit (140)), camera (170), microphone, and memory (130). The memory (130) may store instructions that cause the electronic device (10) to perform the operations described below when executed individually or collectively by at least one processor. For example, the electronic device (10) may include a head-mounted device (HMD), and the user may include a wearer of the electronic device (10).

[0256] According to one embodiment, in operation 1605, the electronic device (10) may acquire at least one first image. The electronic device (10) may acquire at least one first image using at least one camera. The at least one first image may include an image corresponding to the surrounding space of the electronic device (10) and / or an image including at least a part of the body of the user of the electronic device (10). For example, the electronic device (10) may acquire at least one first image based on a specified period (e.g., the third time interval in FIG. 13a) or based on user input.

[0257] According to one embodiment, in operation 1610, the electronic device (10) can identify an object of interest from at least one first image based on interaction. For example, the electronic device (10) can identify surrounding objects based on image recognition. For example, the electronic device (10) can recognize surrounding objects using the object recognition module (715) of FIG. 7.

[0258] The electronic device (10) can identify an object of interest among surrounding objects based on interaction with the user. For example, the electronic device (10) can identify the object with the highest priority or the highest priority object described above as the object of interest. The electronic device (10) can identify the surrounding object with the highest frequency of interaction or the surrounding object with the longest duration of interaction as the object of interest. The interaction may include, for example, at least one of the user's gaze toward the object of interest, manipulation of the object of interest, or designation of the object of interest by the user. The electronic device (10) can identify the designation of the object of interest based on user gestures (e.g., gestures pointing toward the object of interest) or user input (e.g., input using a controller connected to communicate with the electronic device (10)).

[0259] According to one embodiment, in operation 1620, the electronic device (10) may acquire context information based on at least one of sensor data or at least one first image. For example, the electronic device (10) may acquire sensor data using at least one sensor. For example, the electronic device (10) may acquire context information according to the method described above in relation to FIG. 7. For example, the context information may include at least one of the user's posture, the height of the electronic device (10), or the configuration of the surrounding space.

[0260] Context information may include, for example, information on an executed application, surrounding environment information, surrounding object information, and / or additional information. Information on an executed application may include the name, package name, version, type, and / or purpose of an application running on the electronic device (10). Information on the surrounding environment may include indoor, outdoor, geographical location, category of surrounding space, relationship between other spaces and surrounding space, image of surrounding space, 3D configuration of surrounding space, priority of objects in surrounding space, and / or space visit history.

[0261] For example, surrounding object information may include the name, type, and / or spatial location of objects placed in the surroundings.

[0262] For example, additional information may include the posture of the user of the electronic device (10), the user's gestures, information about other users located nearby, a summary of the current space, and / or a summary image of the current space. The user's posture may include, for example, standing, sitting in a chair, sitting on the floor, lying down, walking, and / or a specific posture.

[0263] According to one embodiment, in operation 1625, the electronic device (10) can identify a recommended workspace based on the application and context information associated with the object of interest. The electronic device (10) can identify a recommended workspace corresponding to the application and context information among a plurality of workspaces stored in memory (130). For example, the plurality of workspaces may correspond to work areas stored based on the method described above in relation to FIGS. 8a to 12.

[0264] For example, the associated application may include at least one of an application corresponding to the attribute (e.g., use) of the object of interest, an application pre-configured for the object of interest, or an application placed adjacent to the object of interest within an extended space. In one example, when an object configured to be triggered (e.g., the triggering object of FIG. 11e) is identified, the associated application may identify an application associated with the identified object. The electronic device (10) may identify a workspace containing the associated application among a plurality of workspaces stored in memory (130) as a recommended workspace.

[0265] For example, the electronic device (10) can identify a recommended workspace corresponding to context information. The electronic device (10) can identify a workspace (e.g., a work area) that satisfies specified conditions according to the method described above in relation to operation 1315 of FIG. 13a as a recommended workspace.

[0266] In one embodiment, in operation 1630, the electronic device (10) may provide a recommended workspace. For example, the electronic device (10) may display a plurality of execution screens included in the recommended workspace by arranging them within the augmented reality according to the settings of the recommended workspace. For example, the electronic device (10) may provide a recommended workspace by restoring the recommended workspace. Restoring the recommended workspace may include at least one of restoring the arrangement of the plurality of execution screens within the augmented reality or restoring the execution state associated with the plurality of execution screens.

[0267] In one example, the electronic device (10) may provide a recommended workspace based on user input. In one example, the electronic device (10) may refrain from providing a guide if an application other than the application associated with the object of interest is running in the foreground. The electronic device (10) may provide a guide (e.g., the recommendation UI (1460) of FIG. 14a) that recommends a recommended workspace to the user based on the identification of the recommended workspace. The electronic device (10) may provide a recommended workspace based on the user's response to the guide (e.g., input acknowledging the recommendation).

[0268] In one example, the guide may include summary information about the recommended workspace (e.g., an image (1461) of the workspace in FIG. 14a and / or a title (1463) of the workspace). The electronic device (10) may generate summary information using a generative artificial intelligence model.

[0269] The operations of the electronic device (10) described above in relation to FIG. 16 are part of the embodiments described above in relation to FIG. 1 through 15, and a person skilled in the art will understand that the electronic device (10) can perform the embodiments described above in relation to FIG. 1 through 15. For example, the method of FIG. 16 corresponds to the work area restoration method of FIG. 13a, but a person skilled in the art will understand that the electronic device (10) can also perform the remaining embodiments described above in relation to FIG. 1 through 15.

[0270] FIG. 17 is a block diagram of an exemplary electronic device (1700) capable of performing the operations described in this document.

[0271] Referring to FIG. 17, the electronic device (1700) may be one of various forms of electronic devices, such as a notebook (1790), smartphones (1791) having various form factors (e.g., a bar-type smartphone (1791-1), a foldable-type smartphone (1791-2), or a sliderable (or rollable)-type smartphone (1791-3)), a tablet (1792), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 17 are illustrative only and are not intended to limit the implementations described or claimed herein. The electronic device (1700) may be referred to as a mobile device, a user device, a multifunction device, a portable device, or a server.

[0272] The electronic device (1700) may include components comprising at least one processor (1710) (hereinafter referred to as processor (1710)), at least one memory (1720) (hereinafter referred to as memory (1720)), at least one display (1740) (hereinafter referred to as display (1740)), at least one image sensor (1750) (hereinafter referred to as image sensor (1750)), at least one communication circuit (1760) (hereinafter referred to as communication circuit (1760)), and / or at least one sensor (1770) (hereinafter referred to as sensor (1770)). The components are merely exemplary. For example, the electronic device (1700) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuit, antenna, rechargeable battery, or input / output interface). For example, some components may be omitted from the electronic device (1700). For example, some components can be integrated into a single component.

[0273] The processor (1710) may be implemented as one or more integrated circuit (or circuitry) chips and may perform various data processing operations. The processor (1710) may include at least one electrical circuit and may process instructions (or programs, data) stored in memory (1720) individually or collectively in a distributed manner. The processor (1710) may include a processor assembly comprising one or more processing circuits. The processor (1710) may include any processing circuit that is operative to control the performance and operations of one or more components of the electronic device (1700) (e.g., memory (1720), display (1740), image sensor (1750), communication circuit (1760), and / or sensor (1770)). For example, the processor (1710) (e.g., application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or chipset). For example, the processor (1710) may be implemented with a plurality of cores (or at least one core circuit), a plurality of chips, or a plurality of chipsets. For example, the processor (1710) may include one or more processing circuits. For example, the processor (1710) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively. By example, without limitation, at least a portion of the processor (1710) may be included in a first chip of the electronic device (1700), and at least another portion of the processor (1710) may be included in a second chip of the electronic device (1700) different from the first chip of the electronic device (1700).

[0274] For example, the processor (1710) may include a central processing unit (1711), a graphics processing unit (1712), a neural processing unit (1713), an image signal processor (1714), a display controller (1715), a memory controller (1716), a storage controller (1717), a communication processor (1718), and / or a sensor interface (1719). These components of the processor (1710) are merely exemplary. For example, the processor (1710) may include other components. For example, some components of the processor (1710) may be omitted from the processor (1710). For example, some components of the processor (1710) may be included as separate components of the electronic device (1700) outside of the processor (1710). For example, some components of the processor (1710) (e.g., memory controller (1716)) may be included within other components (e.g., at least part of memory (1720), an interface (e.g. available for connection to at least one component of the electronic device (100)), a display (1740) and / or an image sensor (1750)).

[0275] The processor (1710) may cause other components of the electronic device (1700) to perform various operations by executing instructions stored in memory (1720). The CPU (1711) (or central processing circuit) may be configured to control the components of the processor (1710) based on the execution of instructions stored in memory (1720) (e.g., volatile memory (1721) and / or non-volatile memory (1722)). The GPU (1712) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (1713) (or neural processing circuit, or AI (artificial intelligence) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). An ISP (1714) (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor (1750) into a format suitable for a component within an electronic device (1700) or a component of a processor (1710). A display controller (1715) (or display control circuit, or DPU (display processing unit)) may be configured to process an image acquired from a CPU (1711), GPU (1712), ISP (1714), or memory (1720) (e.g., volatile memory (1721)) into a format suitable for a display (1740). A memory controller (1716) (or memory control circuit) may be configured to control reading data from volatile memory (1721) and writing data to volatile memory (1721). The storage controller (1717) (or storage control circuit) may be configured to control reading data from non-volatile memory (1722) and writing data to non-volatile memory (1722).The CP (1718) (communication processing circuit) may be configured to process data obtained from a component of the processor (1710) into a format suitable for transmitting to another electronic device via the communication circuit (1760), or to process data obtained from another electronic device via the communication circuit (1760) into a format suitable for processing by the component of the processor (1710). For example, the communication circuit (1760) may include one or more communication circuits. The sensor interface (1719) (or sensing data processing circuit, sensor hub) may be configured to process data regarding the state of the electronic device (1700) and / or the state around the electronic device (1700), obtained through the sensor (1770), into a format suitable for the component of the processor (1710).

[0276] Memory (1720) may include one or more storage media (or one or more storage devices). For example, memory (1720) may include a memory assembly comprising one or more storage media. For example, the one or more storage media may include a hard drive, a flash memory, a permanent memory such as ROM (read-only memory) (e.g., non-volatile memory (1722)), a semi-permanent memory such as RAM (random access memory) (e.g., volatile memory (1721)), any other suitable type of storage (or storage assembly), or any combination thereof. Memory (1720) may include a cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (1700). As an example not limited to, the cache memory may be included within the processor (1710). The memory (1720) may be fixedly embedded within the electronic device (1700) or incorporated into one or more suitable types of components (e.g., a SIM (subscriber identity module) card and / or an SD (secure digital) card) that can be repeatedly inserted into and removed from the electronic device (1700).

[0277] For example, memory (1720) may store one or more software applications, such as operating system (or system) software applications, firmware software applications, driver software applications, plugin (e.g., add-in, add-on, and / or applet) software applications, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (1710). For example, memory (1720) may store instructions that can be called by an application programming interface (API). For example, memory (1720) may store instructions within a library.

Claims

1. In an electronic device (10), At least one sensor (140); At least one camera (170); Mike(182, 183); Display (160); Memory (130); and It includes at least one processor (120) that is communicatively connected to the above at least one sensor, the above at least one camera, the above microphone, the above display, and the above memory and includes at least one processing circuit, and When the above memory is executed individually or collectively by the above at least one processor, the electronic device: At least one first image is obtained using the above at least one camera, and Based on interaction with a user of the electronic device, an object of interest is identified among the surrounding objects of the electronic device from at least one first image, and Context information is obtained based on sensor data obtained using the at least one sensor or at least one of the at least one first image, and Identifying a recommended workspace among a plurality of workspaces stored in the memory based on the application associated with the object of interest and the context information, and An electronic device that stores instructions for providing the above recommended workspace through the above display.

2. In Paragraph 1, When the above instructions are executed individually or in combination by the at least one processor, the electronic device identifies the object of interest based on at least one of the interaction frequency or the time of interaction, and An electronic device wherein the interaction comprises at least one of the user’s gaze toward the object of interest, manipulation of the object of interest, or designation of the object of interest.

3. In Paragraph 1, The above recommended workspace includes multiple execution screens, and An electronic device that, when the above instructions are executed individually or in combination by the at least one processor, allows the electronic device to arrange and provide the plurality of execution screens in an extended reality according to information in the recommended workspace.

4. In Paragraph 3, When the above instructions are executed individually or in combination by the at least one processor, the electronic device: By providing the above recommended workspace, the restoration of the above recommended workspace is performed, and An electronic device in which the restoration of the recommended workspace includes the restoration of the alignment of the plurality of execution screens within the augmented reality and the restoration of the execution state associated with the plurality of execution screens.

5. In Paragraph 1, An electronic device comprising at least one of an application associated with the object of interest, an application corresponding to the attributes of the object of interest, an application pre-configured for the object of interest, or an application placed adjacent to the object of interest in an extended reality space.

6. In Paragraph 5, The above context information includes at least one of the user's posture, surrounding space configuration, or the height of the electronic device, and When the above instructions are executed individually or in combination by the at least one processor, the electronic device: An electronic device for identifying the recommended workspace corresponding to the context information and the application associated with the object of interest among the plurality of workspaces.

7. In Paragraph 1, When the above instructions are executed individually or in combination by the at least one processor, the electronic device, Based on the identification of the recommended workspace above, a guide is provided to recommend the recommended workspace to the user, and An electronic device that provides the recommended workspace based on the user's response to the above guide.

8. In Paragraph 7, When the above instructions are executed individually or in combination by the at least one processor, the electronic device, An electronic device that refrains from providing the guide when an application other than the application associated with the object of interest is running in the foreground.

9. In Paragraph 7, When the above instructions are executed individually or in combination by the at least one processor, the electronic device, Generate summary information for the aforementioned multiple workspaces using a generative artificial intelligence model, and The above guide is an electronic device that includes summary information about the above recommended workspace.

10. In Paragraph 1, The above electronic device includes a head-mounted device (HMD), and The above user is an electronic device corresponding to the wearer of the above electronic device.

11. A method for providing a recommended workspace for an electronic device (10), The operation of acquiring at least one first image using at least one camera of the electronic device; An operation of identifying an object of interest among surrounding objects of the electronic device from at least one first image based on interaction with a user of the electronic device; An operation of acquiring context information based on at least one of sensor data acquired using at least one sensor of the electronic device or at least one of the at least one first image; An operation to identify a recommended workspace among a plurality of workspaces stored in the memory of the electronic device based on the application associated with the object of interest and the context information; and A method comprising the operation of providing the above-mentioned recommended workspace through the display of the electronic device.

12. In Paragraph 11, The operation of identifying the object of interest includes the operation of identifying the object of interest based on at least one of the interaction frequency or the time of interaction, and A method wherein the above interaction comprises at least one of the user’s gaze toward the object of interest, manipulation of the object of interest, or designation of the object of interest.

13. In Paragraph 11, The above recommended workspace includes multiple execution screens, and A method comprising the operation of providing the recommended workspace through the display of the electronic device, wherein the operation of arranging and providing the plurality of execution screens in an augmented reality according to the information of the recommended workspace.

14. In Paragraph 13, The operation of providing the recommended workspace through the display of the electronic device includes the operation of performing restoration of the recommended workspace by providing the recommended workspace. A method comprising the restoration of the recommended workspace, the restoration of the alignment of the plurality of execution screens within the augmented reality and the restoration of the execution state associated with the plurality of execution screens.

15. In Paragraph 11, A method in which an application associated with the object of interest comprises at least one of an application corresponding to the attributes of the object of interest, an application pre-configured for the object of interest, or an application placed adjacent to the object of interest in an extended reality space.

Images