Method and device for storing information for accessing pages

Abstract

This electronic device comprises: a display; one or more processors including a processing circuit; and a memory including one or more storage media in which instructions are stored, wherein, when executed by the one or more processors, the instructions can cause the electronic device to: display a preceding page; display a target page after displaying the preceding page; acquire a user bookmark input about bookmarking of the target page; store, in the memory, bookmark information including information for accessing the target page; and, on the basis of the relevance between the preceding page and the target page, add the information for accessing the preceding page to the bookmark information.

Description

Method and device for storing information for accessing pages

[0001] Below, a technology for storing information for accessing a page is disclosed.

[0002] Recently, virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies that utilize computer graphics technology are being developed. VR technology uses computers to create virtual spaces that don't exist in the real world and then make them feel real, while AR or MR technologies superimpose computer-generated information on top of the real world. In other words, they combine the real and virtual worlds to enable real-time user interaction.

[0003] Among these, AR and MR technologies are being integrated and utilized in various fields, such as broadcasting, medical technology, and gaming technology. Representative examples of AR being integrated into broadcasting include the natural change of weather maps in front of a weathercaster giving a weather forecast on TV, or the insertion of non-existent advertising images into the stadium during a sports broadcast, making them appear as if they were actually there.

[0004] A representative service that provides users with augmented reality or mixed reality is the metaverse. This metaverse, a portmanteau of "meta," meaning "fictional" or "abstract," and "universe," signifies a three-dimensional virtual world. A more advanced concept than the existing term "virtual reality environment," the metaverse provides an augmented reality environment where virtual worlds like the web and the internet are integrated into the real world.

[0005] In an electronic device, a display, at least one processor including a processing circuit, and a memory including one or more storage media storing instructions, wherein when the instructions are executed by the at least one processor, the electronic device can cause the electronic device to display a preceding page, display a target page after displaying the preceding page, obtain a bookmark input of a user for bookmarking the target page, store bookmark information including information for accessing the target page in the memory, and add information for accessing the preceding page to the bookmark information based on a relevance between the preceding page and the target page.

[0006] A method performed by an electronic device may include an action of displaying a preceding page through a display, an action of displaying a target page after displaying the preceding page, an action of obtaining a user's bookmark input for bookmarking the target page, an action of storing bookmark information including information for accessing the target page in a memory, and an action of adding information for accessing the preceding page to the bookmark information based on a relevance between the preceding page and the target page.

[0007] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0008] FIG. 2 illustrates an example of an optical see-through device according to various embodiments.

[0009] FIG. 3 illustrates examples of optical systems for an eye tracking camera, a transparent member, and a display according to various embodiments.

[0010] FIGS. 4A and 4B are drawings showing examples of the front and back of an electronic device according to various embodiments.

[0011] FIG. 5 illustrates examples of construction of a virtual space, input from a user within the virtual space, and output to the user according to various embodiments.

[0012] FIG. 6 is a diagram illustrating an example of a method of bookmarking a page performed by an electronic device according to various embodiments.

[0013] FIG. 7 is a diagram illustrating an example of an operation of an electronic device providing a display history of a page according to various embodiments.

[0014] FIG. 8 is a drawing illustrating an example of an operation of displaying a target page when an electronic device receives a display command for a target page from an external device according to various embodiments.

[0015] FIG. 9 illustrates an example of an operation in which an electronic device, according to various embodiments, bookmarks a page for at least some of a plurality of operations constituting a function and then re-performs the function.

[0016] FIG. 10 is a drawing illustrating an example of an operation in which an electronic device according to various embodiments performs multiple operations constituting one function.

[0017] FIGS. 11A and 11B illustrate example interfaces for displaying multiple pages together according to various embodiments.

[0018] FIG. 12 is a diagram illustrating an example of an operation of an electronic device according to various embodiments to determine relevance between search screens based on relevance between search requests.

[0019] FIG. 13 is a diagram illustrating an example of an operation of an electronic device providing an interface for comparing multiple pages according to various embodiments.

[0020] FIGS. 14a, 14b, 14c, and 14d are diagrams illustrating examples of screen layouts for displaying a results page including search results according to various embodiments.

[0021] FIG. 15 is a diagram illustrating an example of an operation of an electronic device according to various embodiments to provide a comparison interface between a plurality of target pages and a plurality of preceding pages.

[0022] FIG. 16 is a diagram illustrating various embodiments of an operation of an electronic device to obtain user input after bookmarking a target page.

[0023] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0024] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0025] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0026] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0027] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0028] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0029] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0030] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0031] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0032] A display module (160) (e.g., a display) can visually provide information to an external device (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0033] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0034] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0035] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0036] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0037] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0038] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0039] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0040] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0041] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data relation (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0042] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0043] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0044] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0045] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0046] According to one embodiment, commands or data can be transmitted or received between an electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199).

[0047] Each of the external electronic devices (102, 103) and the server (108) may be the same type of device as or different from the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 103) or the server (108). For example, when the electronic device (101) needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of executing the function or service itself or in addition, request one or more external electronic devices to execute at least a part of the function or service. The one or more external electronic devices that receive the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a part of a response to the request. In this specification, an example is mainly described in which an electronic device (101) is an augmented reality device (e.g., an electronic device (201) of FIG. 2, an electronic device (301) of FIG. 3, or an electronic device (401) of FIG. 4), and an external electronic device (102, 103) or a server (108) among the servers transmits the results of executing a virtual space and additional functions or services related to the virtual space to the electronic device (101).

[0048] The server (108) may include a processor (181), a communication module (182), and a memory (183). The processor (181), the communication module (182), and the memory (183) may be configured similarly to the processor (120), the communication module (190), and the memory (130) of the electronic device (101). For example, the processor (181) may provide a virtual space and interaction between users within the virtual space by executing instructions stored in the memory (183). The processor (181) may generate at least one of visual information, auditory information, or tactile information of the virtual space and objects within the virtual space. For example, as visual information, the processor (181) may generate rendering data (e.g., visual rendering data) that renders the appearance (e.g., shape, size, color, or texture) of the virtual space and the appearance (e.g., shape, size, color, or texture) of objects positioned within the virtual space. In addition, the processor (181) may generate rendering data that renders a change (e.g., a change in the appearance of an object, a sound generation, or a tactile generation) based on at least one of an interaction between objects (e.g., a physical object, a virtual object, or an avatar object) in a virtual space, or a user's input to an object (e.g., a physical object, a virtual object, or an avatar object). The communication module (182) may establish communication with a first electronic device (e.g., an electronic device (101)) of a user and a second electronic device (e.g., an electronic device (102)) of another user. The communication module (182) may transmit at least one of the visual information, the tactile information, or the auditory information described above to the first electronic device and the second electronic device. For example, the communication module (182) may transmit rendering data.

[0049] For example, the server (108) renders content data executed in an application and transmits it to the electronic device (101), and the electronic device (101) receiving the data can output the content data to the display module (160). If the electronic device (101) detects user movement through an IMU sensor or the like, the processor (120) of the electronic device (101) can correct the rendering data received from the external electronic device (102) based on the movement information and output it to the display module (160). Alternatively, the movement information can be transmitted to the server (108) to request rendering so that the screen data is updated accordingly. However, the present invention is not limited thereto, and the aforementioned rendering can be performed by various forms of external electronic devices (102, 103), such as a smartphone or a case device capable of storing and charging the electronic device (101). Rendering data corresponding to the aforementioned virtual space generated by the external electronic device (102, 103) can be provided to the electronic device (101). For another example, the electronic device (101) may receive virtual space information (e.g., vertex coordinates, texture, color defining the virtual space) and object information (e.g., vertex coordinates, texture, color defining the appearance of an object) from the server (108) and perform rendering on its own based on the received data.

[0050] FIG. 2 illustrates an example of an optical see-through device according to various embodiments.

[0051] The electronic device (201) may include at least one of a display (e.g., a display module (160) of FIG. 1), a vision sensor, a light source (230a, 230b), an optical element, or a substrate. An electronic device (201) that has a transparent display and provides an image through the transparent display may be referred to as an optical see-through device (OST device).

[0052] The display may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED).

[0053] In one embodiment, when the display is formed of one of a liquid crystal display (LCD), a digital mirror display (DMD), or a silicon liquid crystal display (SLCD), the electronic device (201) may include a light source (230a, 230b) that irradiates light to a screen output area of ​​the display (e.g., a screen display unit (215a, 215b)). In another embodiment, when the display can generate light on its own, for example, when it is formed of one of an organic light emitting diode (OLED) or a micro LED, the electronic device (201) may provide a good quality virtual image to the user even without including a separate light source (230a, 230b). In one embodiment, if the display is formed of an organic light emitting diode (OLED) or a micro LED, the light source (230a, 230b) is unnecessary, and thus the electronic device (201) may be lightweight.

[0054] Referring to FIG. 2, the electronic device (201) may include a display, a first transparent member (225a) and / or a second transparent member (225b), and a user may use the electronic device (201) while wearing it on his or her face. The first transparent member (225a) and / or the second transparent member (225b) may be formed of a glass plate, a plastic plate, or a polymer, and may be manufactured to be transparent or translucent. According to one embodiment, the first transparent member (225a) may be arranged to face the user's right eye, and the second transparent member (225b) may be arranged to face the user's left eye. The display may include a first display (205) that outputs a first image (e.g., a right image) corresponding to the first transparent member (225a), and a second display (210) that outputs a second image (e.g., a left image) corresponding to the second transparent member (225b). In one embodiment, if each display is transparent, each display and transparent member may be positioned at a position facing the user's eyes to form a screen display unit (215a, 215b).

[0055] In one embodiment, light emitted from the display (205, 210) may be guided along an optical path through an input optical member (220a, 220b) into a waveguide. Light traveling within the waveguide may be guided toward a user's eyes through an output optical member (e.g., the output optical member (340) of FIG. 3). The screen display units (215a, 215b) may be determined based on the light emitted toward the user's eyes.

[0056] For example, light emitted from the display (205, 210) may be reflected in the grating region of the waveguide formed in the input optical member (220a, 220b) and the screen display portions (215a, 215b) and transmitted to the user's eyes.

[0057] The optical element may include at least one of a lens or an optical waveguide.

[0058] The lens can adjust the focus so that the screen outputted to the display can be seen by the user. The lens may include, for example, at least one of a Fresnel lens, a pancake lens, or a multi-channel lens.

[0059] An optical waveguide can transmit image light generated from a display to a user's eyes. For example, the image light may represent light emitted by a light source (230a, 230b) that passes through a screen output area of ​​the display. The optical waveguide can be made of glass, plastic, or polymer. The optical waveguide can include a nano-pattern formed on a portion of an internal or external surface, for example, a grating structure having a polygonal or curved shape. An exemplary structure of the optical waveguide is described below in FIG. 3.

[0060] The vision sensor may include at least one of a camera sensor or a depth sensor.

[0061] The first camera (265a, 265b) is a recognition camera, and may be a camera used for 3DoF, 6DoF head tracking, hand detection, hand tracking, and spatial recognition. The first camera (265a, 265b) may mainly include a GS (Global shutter) camera. Since a stereo camera is required for head tracking and spatial recognition, the first camera (265a, 265b) may include two or more GS cameras. The GS camera may have superior performance compared to the RS (Rolling shutter) camera in terms of detecting and tracking fine movements such as rapid hand movements and fingers. For example, the GS camera may have low image blur. The first camera (265a, 265b) may capture image data used for spatial recognition for 6DoF and SLAM function through depth shooting. Additionally, a user gesture recognition function can be performed based on image data captured by the first camera (265a, 265b).

[0062] The second camera (270a, 270b) is an ET (Eye Tracking) camera and can be used to capture image data for detecting and tracking the user's pupils. The second camera (270a, 270b) is described later in FIG. 3.

[0063] The third camera (245) may be a camera for photography. The third camera (245) may include a high-resolution camera for capturing HR (High Resolution) or PV (Photo Video) images. The third camera (245) may include a color camera equipped with functions for obtaining high-quality images, such as an AF function and optical image stabilization (OIS). The third camera (245) may be a GS camera or an RS camera.

[0064] The fourth camera unit (e.g., the face recognition camera (425, 426) of FIG. 4 below) is a face recognition camera, and the FT (Face Tracking) camera can be used to detect and track the user's facial expressions.

[0065] A depth sensor (not shown) may be a sensor that senses information for determining the distance to an object, such as Time of Flight (TOF). TOF is a technology that measures the distance to an object using signals (e.g., near-infrared, ultrasound, laser, etc.). A depth sensor based on TOF technology can measure the time of flight of a signal by transmitting a signal and measuring the signal at a receiver.

[0066] The light source (230a, 230b) (e.g., illumination module) may include a device (e.g., light emitting diode (LED)) that irradiates light of various wavelengths. The illumination module may be attached to various locations depending on the application. As one example, a first illumination module (e.g., LED device) attached around the frame of the augmented reality glasses device may emit light to assist in gaze detection when tracking eye movements with an ET camera. The first illumination module may include, for example, an IR LED of an infrared wavelength. As another example, a second illumination module (e.g., LED device) may be attached adjacent to a camera mounted around a hinge (240a, 240b) connecting the frame and the temple, or around a bridge connecting the frames. The second illumination module may emit light to supplement the ambient brightness when the camera is taking pictures. If subject detection is not easy in a dark environment, the second lighting module may illuminate.

[0067] A substrate (235a, 235b) (e.g., a printed circuit board (PCB)) can support the aforementioned components.

[0068] A printed circuit board (PCB) may be disposed on the temple portion of the glasses. The FPCB may transmit electrical signals to each module (e.g., a camera, a display, an audio module, a sensor module) and other printed circuit boards. In one embodiment, at least one printed circuit board may include a first substrate, a second substrate, and an interposer disposed between the first substrate and the second substrate. Electrical signals may be transmitted to each module and other printed circuit boards.

[0069] Other components may include, for example, at least one of a plurality of microphones (e.g., a first microphone (250a), a second microphone (250b), a third microphone (250c)), a plurality of speakers (e.g., a first speaker (255a), a second speaker (255b)), a battery (260), an antenna, or a sensor (e.g., an acceleration sensor, a gyro sensor, a touch sensor, etc.).

[0070] FIG. 3 illustrates examples of optical systems for an eye tracking camera, a transparent member, and a display according to various embodiments.

[0071] FIG. 3 is a diagram for explaining the operation of an eye-tracking camera included in an electronic device according to one embodiment. Referring to FIG. 3, a process is illustrated in which an eye-tracking camera (310) of an electronic device (301) according to one embodiment (e.g., the first eye-tracking camera (270a) and the second eye-tracking camera (270b) of FIG. 2) tracks a user's eye (309), that is, the user's gaze, using light (e.g., infrared light) output from a display (320) (e.g., the first display (205) and the second display (210) of FIG. 2).

[0072] The second camera (e.g., the second cameras (270a, 270b) of FIG. 2) may be an eye tracking camera (310) that collects information to position the center of a virtual image projected onto the electronic device (301) according to the direction in which the pupil of the wearer of the electronic device (301) is looking. The second camera may also include a GS camera to detect the pupil and track rapid eye movement. ET cameras may also be installed for the left and right eyes, respectively, and cameras with the same performance and specifications may be used. The eye tracking camera (310) may include a gaze tracking sensor (315). The gaze tracking sensor (315) may be included inside the eye tracking camera (310). Infrared light output from the display (320) may be transmitted to the user's eyes (309) as infrared reflected light (303) by a half mirror. The gaze tracking sensor (315) can detect infrared reflected light (303) and infrared transmitted light (305) reflected from the user's eye (309). The eye tracking camera (310) can track the user's eye (309), or in other words, the user's gaze, based on the detection result of the gaze tracking sensor (315).

[0073] The display (320) may include a plurality of visible light pixels and a plurality of infrared pixels. The visible light pixels may include R, G, and B pixels. The visible light pixels may output visible light corresponding to a virtual object image. The infrared pixels may output infrared light. The display (320) may include, for example, micro light emitting diodes (LEDs) or organic light emitting diodes (OLEDs).

[0074] The display optical pipe (350) and the eye tracking camera optical pipe (360) may be included inside a transparent member (370) (e.g., the first transparent member (225a) and the second transparent member (225b) of FIG. 2). The transparent member (370) may be formed of a glass plate, a plastic plate, or a polymer, and may be manufactured to be transparent or translucent. The transparent member (370) may be positioned to face the user's eyes. At this time, the distance between the transparent member (370) and the user's eyes (309) may be referred to as 'eye relief' (380).

[0075] The transparent member (370) may include optical waveguides (350, 360). The transparent member (370) may include an input optical member (330) and an output optical member (340). In addition, the transparent member (370) may include an eye-tracking splitter (375) that separates input light into multiple waveguides.

[0076] According to one embodiment, light incident on one end of the display light pipe (350) can be propagated inside the display light pipe (350) by the nano-pattern and provided to the user. In addition, the display light pipe (350) composed of a free-form prism can provide image light to the user through the reflected light. The display light pipe (350) can include at least one diffractive element (e.g., a Diffractive Optical Element (DOE), a Holographic Optical Element (HOE)) or at least one reflective element (e.g., a reflective mirror). The display light pipe (350) can guide display light (e.g., image light) emitted from a light source to the user's eyes by using at least one diffractive element or reflective element included in the display light pipe (350). For reference, also, in FIG. 3, the output optical member (340) is expressed as being separate from the eye-tracking optical waveguide (360), but the output optical member (340) may be included inside the eye-tracking optical waveguide (360).

[0077] According to various embodiments, the diffractive element may include an input optical member (330) and an output optical member (340). For example, the input optical member (330) may mean an input grating region. The output optical member (340) may mean an output grating region. The input grating region may serve as an input terminal that diffracts (or reflects) light output from (e.g., a Micro LED) to transmit the light to a transparent member (e.g., a first transparent member, a second transparent member) of a screen display unit. The output grating region may serve as an outlet that diffracts (or reflects) light transmitted to a transparent member (e.g., a first transparent member, a second transparent member) of a waveguide to a user's eye.

[0078] According to various embodiments, the reflective element may include a total internal reflection (TIR) ​​optical element or a total internal reflection waveguide. For example, total internal reflection may refer to a method of guiding light such that light (e.g., a virtual image) entering through an input grating region is 100% reflected from one surface (e.g., a specific surface) of the waveguide, thereby transmitting 100% of the light to the output grating region.

[0079] In one embodiment, light emitted from the display (320) may be guided along an optical path through an input optical element (330) into a waveguide. Light traveling within the waveguide may be guided toward the user's eyes through an output optical element (340). The screen display may be determined based on the light emitted toward the user's eyes.

[0080] FIGS. 4A and 4B are diagrams illustrating examples of the front and back of an electronic device according to various embodiments. FIG. 4A may be an external appearance of the electronic device (401) as viewed from a first direction (①), and FIG. 4B may be an external appearance of the electronic device (401) as viewed from a second direction (②). When a user wears the electronic device (401), the external appearance viewed by the user's eyes may be FIG. 4B.

[0081] Referring to FIG. 4A, according to various embodiments, an electronic device (401) (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, and the electronic device (301) of FIG. 3) may provide a service that provides an extended reality (XR) experience to a user. For example, an XR or XR service may be defined as a service that collectively refers to virtual reality (VR), augmented reality (AR), and / or mixed reality (MR).

[0082] According to one embodiment, the electronic device (401) may refer to a head-mounted device or a head-mounted display worn on the user's head, but may also be configured in the form of at least one of glasses, goggles, a helmet, or a hat. The electronic device (401) may include an OST (optical see-through) type configured to allow external light to reach the user's eyes through the glasses when worn, or a VST (video see-through) type configured to allow light emitted from the display to reach the user's eyes but block external light so that external light does not reach the user's eyes when worn.

[0083] According to one embodiment, the electronic device (401) may be worn on the user's head and may provide the user with an image related to an extended reality (XR) service. For example, the electronic device (401) may provide XR content (hereinafter referred to as an XR content image) that outputs at least one virtual object to be superimposed on a display area or an area determined to be the user's field of view (FoV). According to one embodiment, the XR content may refer to an image or image that appears to have at least one virtual object superimposed on an image related to a real space acquired through a camera (e.g., a camera for taking pictures) or a virtual space. According to one embodiment, the electronic device (401) may provide XR content based on a function being performed by the electronic device (401) and / or a function being performed by one or more external electronic devices (e.g., the electronic devices (102, 104) of FIG. 1, the server (108) of FIG. 1).

[0084] According to one embodiment, the electronic device (401) is at least partially controlled by an external electronic device (e.g., electronic devices (102 or 104) of FIG. 1), and may perform at least one function under the control of the external electronic device, but may also perform at least one function independently.

[0085] Referring to FIG. 4A, a vision sensor may be placed on a first surface of a housing of a main body (410) of an electronic device (401). The vision sensor may include cameras (e.g., cameras for second functions (411, 412), cameras for first functions (415)) and / or a depth sensor (417) for obtaining information related to the surrounding environment of the electronic device (401).

[0086] In one embodiment, the second function cameras (411, 412) can obtain images related to the surrounding environment of the electronic device (401). The first function cameras (415) can obtain images when the wearable electronic device is worn by the user. The first function cameras (415) can be used for hand detection and tracking, and recognition of user gestures (e.g., hand movements). The first function cameras (415) can be used for 3DoF, 6DoF head tracking, position (spatial, environmental) recognition, and / or movement recognition. In one embodiment, the second function cameras (411, 412) can also be used for hand detection and tracking, and user gestures.

[0087] In one embodiment, the depth sensor (417) may be configured to transmit a signal and receive a signal reflected from a subject, and may be used for purposes such as time of flight (TOF) to determine the distance to an object. Instead of or in addition to the depth sensor (417), cameras (411, 412, 415, 416) may determine the distance to an object.

[0088] Referring to FIG. 4b, a camera (425, 426) for facial recognition and / or a display (421) (and / or lens) may be placed on the second surface (420) of the housing of the main body (410).

[0089] In one embodiment, a face recognition camera (425, 426) adjacent to the display may be used to recognize the user's face, or may recognize and / or track the user's two eyes.

[0090] In one embodiment, the display (421) (and / or lens) may be disposed on the second side (420) of the electronic device (401). In one embodiment, the electronic device (401) may not include some of the plurality of cameras (415). Although not shown in FIGS. 4A and 4B , the electronic device (401) may further include at least one of the configurations illustrated in FIG. 2 .

[0091] According to one embodiment, the electronic device (401) may include a main body (410) that mounts at least some of the components of FIG. 1, a display (421) (e.g., a display module (160) of FIG. 1) disposed in a first direction (①) of the main body (410), a first function camera (e.g., a recognition camera) (415) disposed in a second direction (②) of the main body (410), a second function camera (e.g., a shooting camera) (411, 412) disposed in a second direction (②), a third function camera (e.g., a gaze tracking camera) (428) disposed in the first direction (①), a fourth function camera (e.g., a face recognition camera) (425, 426) disposed in the first direction (①), a depth sensor (417) disposed in the second direction (②), and a touch sensor (413) disposed in the second direction (②). Although not shown in the drawing, the main body (410) includes a memory (e.g., memory (130) of FIG. 1) and a processor (e.g., processor (120) of FIG. 1), and may further include other components shown in FIG. 1.

[0092] According to one embodiment, the display (421) may include a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED).

[0093] In one embodiment, when the display (421) is formed of one of a liquid crystal display (LCD), a digital mirror display (DMD), or a silicon liquid crystal display (SiLCD), the electronic device (401) may include a light source that irradiates light to a screen output area of ​​the display (421). In another embodiment, when the display (421) can generate light on its own, for example, when the electronic device (401) is formed of one of an organic light emitting diode (OLED) or a micro LED, the electronic device (401) may provide a good quality XR content image to the user even without including a separate light source. In one embodiment, when the display (421) is formed of an organic light emitting diode (OLED) or a micro LED, a light source is unnecessary, and thus the electronic device (401) may be lightweight.

[0094] According to one embodiment, the display (421) may include a first transparent member (421a) and / or a second transparent member (421b). The user may use the electronic device (401) while wearing it on his or her face. The first transparent member (421a) and / or the second transparent member (421b) may be formed of a glass plate, a plastic plate, or a polymer, and may be manufactured to be transparent or translucent. According to one embodiment, the first transparent member (421a) may be arranged to face the user's left eye in the fourth direction (④), and the second transparent member (421b) may be arranged to face the user's right eye in the third direction (③). According to various embodiments, when the display (421) is transparent, it may be arranged at a position facing the user's eyes to form a display area.

[0095] According to one embodiment, the display (421) may include a lens including a transparent waveguide. The lens may serve to adjust the focus so that the screen (e.g., XR content image) output to the display (421) can be viewed by the user's eyes. For example, light emitted from the display panel may pass through the lens and be transmitted to the user through a waveguide formed within the lens. The lens may be configured as a Fresnel lens, a pancake lens, or a multi-channel lens.

[0096] An optical waveguide (e.g., a waveguide) may serve to transmit light generated by the display (421) to the user's eyes. The optical waveguide may be made of glass, plastic, or a polymer, and may include nano-patterns formed on a portion of the inner or outer surface, for example, a grating structure having a polygonal or curved shape. According to one embodiment, light incident on one end of the optical waveguide, that is, an output image of the display (421), may be propagated within the optical waveguide and provided to the user. In addition, an optical waveguide composed of a free-form prism may provide the incident light to the user through a reflective mirror. The optical waveguide may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or at least one reflective element (e.g., a reflective mirror). The optical waveguide may guide an image output from the display (421) to the user's eyes by using at least one diffractive element or reflective element included in the optical waveguide.

[0097] According to one embodiment, the diffractive element may include an input optical member / output optical member (not shown). For example, the input optical member may mean an input grating region, and the output optical member (not shown) may mean an output grating region. The input grating region may serve as an input terminal that diffracts (or reflects) light output from a light source (e.g., a Micro LED) to transmit the light to a transparent member (e.g., a first transparent member (421a), a second transparent member (421b)) of the display area. The output grating region may serve as an outlet that diffracts (or reflects) light transmitted to a transparent member (e.g., a first transparent member, a second transparent member) of the optical waveguide to a user's eye.

[0098] In some embodiments, the reflective element may comprise a total internal reflection (TIR) ​​optical element or waveguide for total internal reflection. For example, total internal reflection may refer to a method of directing light such that light (e.g., a virtual image) entering through an input grating region is reflected substantially 100% from one surface (e.g., a specific surface) of the optical waveguide, thereby transmitting substantially 100% of the light to the output grating region at an angle of incidence.

[0099] In one embodiment, light emitted from the display (421) may be guided along an optical path through an input optical element into a waveguide. Light traveling within the optical waveguide may be guided toward the user's eyes through an output optical element. The display area may be determined based on the light emitted toward the user's eyes.

[0100] According to one embodiment, the electronic device (401) may include a plurality of cameras. For example, the cameras may include a first function camera (e.g., a recognition camera) (415) disposed in the second direction (②) of the main body (410), a second function camera (e.g., a shooting camera) (411, 412) disposed in the second direction (②), a third function camera (e.g., a gaze tracking camera) (428) disposed in the first direction (①), and / or a fourth function camera (e.g., a face recognition camera) (425, 426) disposed in the first direction (①), but may further include cameras for other functions not shown.

[0101] The first function camera (e.g., recognition camera) (415) can be used for the purpose of detecting user movement or recognizing user gestures. The first function camera (415) can support at least one of head tracking, hand detection and hand tracking, and spatial recognition. For example, the first function camera (415) mainly uses a GS (global shutter) camera, which has superior performance compared to an RS (rolling shutter) camera, to detect and track fine movements of hand movements and fingers, and can be configured as a stereo camera including two or more GS cameras for head tracking and spatial recognition. The first function camera (415) can perform a SLAM (simultaneous localization and mapping) function to recognize information (e.g., location and / or direction) related to the surrounding space through spatial recognition for 6DoF and depth shooting.

[0102] A second function camera (e.g., a camera for shooting) (411, 412) can be used to capture the outside and generate an image or video corresponding to the outside and transmit it to a processor (e.g., a processor (120) of FIG. 1). The processor can display the image provided from the second function camera (411, 412) on a display (421). The second function camera (411, 412) may be referred to as HR (high resolution) or PV (photo video) and may include a high-resolution camera. For example, the second function camera (411, 412) may include a color camera equipped with functions for obtaining high-quality images, such as an AF (auto focus) function and an OIS (optical image stabilizer), but is not limited thereto, and the second function camera (411, 412) may also include a GS camera or an RS camera.

[0103] A third function camera (e.g., a gaze tracking camera) (428) may be positioned on the display (421) (or inside the main body) so that the camera lens faces the user's eyes when the user wears the electronic device (401). The third function camera (428) may be used for the purpose of detecting and tracking (ET: eye tracking) the pupil. The processor may track the movements of the user's left and right eyes in the images received from the third function camera (428) to determine the gaze direction. By tracking the position of the pupil in the images, the processor may ensure that the center of the XR content image displayed in the display area is positioned according to the direction in which the pupil is looking. As an example, a GS camera may be used as the third function camera (428) to detect the pupil and track the movement of the pupil. The third function cameras (428) may be installed respectively for the left and right eyes, and each camera having the same performance and specifications may be used.

[0104] A fourth functional camera (e.g., a camera for facial recognition) (425, 426) can be used to detect and track (FT: face tracking) the user's facial expression when the user wears the electronic device (401).

[0105] According to one embodiment, the electronic device (401) may include a lighting unit (e.g., LED) (not shown) as an auxiliary means for the cameras. For example, the third function camera (425) may use lighting included in the display to direct the emitted light (e.g., IR LED of infrared wavelength) toward the user's both eyes as an auxiliary means to facilitate gaze detection when tracking eye movements. As another example, the second function cameras (411, 412) may further include a lighting unit (e.g., flash) as an auxiliary means to supplement the surrounding brightness when taking external pictures.

[0106] According to one embodiment, a depth sensor (or depth camera) (417) can be used for the purpose of checking the distance to an object (e.g., an object), such as time of flight (TOF). TOF (time of flight) is a technology that measures the distance to an object using a signal (e.g., near-infrared, ultrasound, or laser). After a signal is transmitted from a transmitter, a signal is measured at a receiver, and the distance to an object can be measured based on the flight time of the signal.

[0107] According to one embodiment, the touch sensor (413) may be arranged in the second direction (②) of the main body (410). For example, when a user wears the electronic device (401), the user's eyes may look in the first direction (①) of the main body. The touch sensor (413) may be implemented as a single type or a left / right separated type depending on the shape of the main body (410), but is not limited thereto. For example, when the touch sensor (413) is implemented as a left / right separated type as illustrated in FIG. 4A, when a user wears the electronic device (401), the first touch sensor (413a) may be arranged at the user's left eye position, such as in the fourth direction (④), and the second touch sensor (413b) may be arranged at the user's right eye position, such as in the third direction (③).

[0108] The touch sensor (413) can recognize a touch input using at least one of, for example, a capacitive, pressure-sensitive, infrared, or ultrasonic method. For example, the capacitive touch sensor (413) can recognize a physical touch (or contact) input or a hovering input (or proximity) of an external object. According to some embodiments, the electronic device (401) may utilize a proximity sensor (not shown) to recognize proximity of an external object.

[0109] According to one embodiment, the touch sensor (413) has a two-dimensional surface and can transmit touch data (e.g., touch coordinates) of an external object (e.g., a user's finger) that comes into contact with the touch sensor (413) to a processor (e.g., the processor (120) of FIG. 1). The touch sensor (413) can detect a hovering input for an external object (e.g., a user's finger) that approaches within a first distance from the touch sensor (413), or detect a touch input that touches the touch sensor (413).

[0110] According to one embodiment, the touch sensor (413) may provide two-dimensional information about the point of contact as “touch data” to the processor (120) when an external object touches the touch sensor (413). The touch data may be described as a “touch mode.” The touch sensor (413) may provide hovering data about the time or location of hovering around the touch sensor (413) to the processor (120) when an external object is located within a first distance from the touch sensor (or in proximity, hovering above the touch sensor). The hovering data may be described as a “hovering mode / proximity mode.”

[0111] According to one embodiment, the electronic device (401) may obtain hovering data using at least one of a touch sensor (413), a proximity sensor (not shown), or / and a depth sensor (417) to generate information about a distance, location, or time point between the touch sensor (413) and an external object.

[0112] According to one embodiment, the interior of the main body (410) may include a processor (e.g., processor (120) of FIG. 1) and memory (e.g., memory (130) of FIG. 1).

[0113] Memory can store various instructions that can be executed by the processor. Instructions can include arithmetic and logical operations, data transfer, or control commands such as input / output that can be recognized by the processor. Memory can temporarily or permanently store various data, including volatile memory (e.g., volatile memory (132) of FIG. 1) and non-volatile memory (e.g., non-volatile memory (134) of FIG. 1).

[0114] The processor may be a configuration that is operatively, functionally, and / or electrically connected to each component of the electronic device (401) and can perform calculations or data processing related to control and / or communication of each component. The operations performed by the processor may be stored in memory and, when executed, executed by instructions that cause the processor to operate.

[0115] Hereinafter, the computational and data processing functions that the processor can implement on the electronic device (401) are not limited, but a series of operations related to the XR content service function will be described. The operations of the processor described below can be performed by executing instructions stored in memory.

[0116] According to one embodiment, the processor may generate a virtual object based on virtual information based on image information. The processor may output a virtual object related to an XR service together with background space information through the display (421). For example, the processor may capture an image related to a real space corresponding to the field of view of a user wearing the electronic device (401) through a second function camera (411, 412) to obtain image information or generate a virtual space for a virtual environment. For example, the processor may control the display (421) to display XR content (hereinafter referred to as an XR content screen) in which at least one virtual object is output so as to be overlapped in an area determined to be a field of view or a field of view (FoV) of the user.

[0117] According to one embodiment, the electronic device (401) may have a form factor for being worn on a user's head. The electronic device (401) may further include a strap and / or a wearable member for being secured on a body part of the user. The electronic device (401) may provide a user experience based on augmented reality, virtual reality, and / or mixed reality while being worn on the user's head.

[0118] FIG. 5 illustrates examples of construction of a virtual space, input from a user within the virtual space, and output to the user according to various embodiments.

[0119] An electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (201) of FIG. 2, the electronic device (301) of FIG. 3, and the electronic device (401) of FIG. 4) can obtain spatial information about a physical space in which the sensor is located using a sensor. The spatial information can include a geographical location of the physical space in which the sensor is located, a size of the space, an appearance of the space, a location of a physical object (551) arranged in the space, a size of the physical object (551), an appearance of the physical object (551), and illuminant information. The appearance of the space and the physical object (551) can include at least one of a shape, a texture, or a color of the space and the physical object (551). The illuminant information is information about a light source that emits light acting within the physical space, and can include at least one of an intensity, a direction, or a color of the illumination. The aforementioned sensor can collect information for providing augmented reality. For example, referring to the augmented reality device illustrated in FIGS. 2 to 4, the sensor may include a camera and a depth sensor. However, the present invention is not limited thereto, and the sensor may further include at least one of an infrared sensor, a depth sensor (e.g., a lidar sensor, a radar sensor, or a stereo camera), a gyro sensor, an acceleration sensor, or a geomagnetic sensor.

[0120] The electronic device (501) can collect spatial information over multiple time frames. For example, in each time frame, the electronic device (501) can collect spatial information about a portion of a scene within a sensing range (e.g., field of view (FOV)) of a sensor at the location of the electronic device (501) in physical space. By analyzing the spatial information of multiple time frames, the electronic device (501) can track changes in an object (e.g., movement of position or change of state) over time. The electronic device (501) can also obtain integrated spatial information (e.g., an image that spatially stitches scenes around the electronic device (501) in physical space) for the integrated sensing range of the multiple sensors by comprehensively analyzing the spatial information collected through the multiple sensors.

[0121] An electronic device (501) according to one embodiment can analyze a physical space into three-dimensional information by utilizing various input signals of sensors (e.g., sensing data from an RGB camera, an infrared sensor, a depth sensor, or a stereo camera). For example, the electronic device (501) can analyze at least one of the shape, size, and position of a physical space, and the shape, size, or position of a physical object (551).

[0122] For example, the electronic device (501) can detect an object captured within a scene corresponding to the camera's field of view using the camera's sensing data (e.g., a captured image). The electronic device (501) can determine a label of a physical object (551) from the camera's two-dimensional scene image (e.g., information indicating the classification of the object, including a value indicating a chair, a monitor, or a plant) and an area (e.g., a bounding box) occupied by the physical object (551) within the two-dimensional scene. Accordingly, the electronic device (501) can obtain two-dimensional scene information at a position viewed by the user (590). In addition, the electronic device (501) can also calculate the physical space location of the electronic device (501) based on the camera's sensing data.

[0123] The electronic device (501) can obtain location information of the user (590) and depth information of the actual space in the viewing direction using sensing data (e.g., depth data) of the depth sensor. The depth information is information indicating the distance from the depth sensor to each point and can be expressed in the form of a depth map. The electronic device (501) can analyze the distance of each pixel in the three-dimensional position viewed by the user (590).

[0124] The electronic device (501) can acquire information including a 3D point cloud and mesh using various sensing data. The electronic device (501) can analyze a physical space to acquire a surface, mesh, or 3D coordinate point cluster that constitutes the space. Based on the information acquired as described above, the electronic device (501) can acquire a 3D point cloud representing physical objects.

[0125] The electronic device (501) can analyze a physical space to obtain information including at least one of a three-dimensional position coordinate, a three-dimensional shape, or a three-dimensional size (e.g., a three-dimensional bounding box) of physical objects placed in the physical space.

[0126] Accordingly, the electronic device (501) can obtain information on a physical object detected in a three-dimensional space and semantic segmentation information on the three-dimensional space. The physical object information may include at least one of the position, appearance (e.g., shape, texture, and color), or size of the physical object (551) in the three-dimensional space. The semantic segmentation information is information that semantically divides the three-dimensional space into subspaces, and may include, for example, information indicating that the three-dimensional space is divided into an object and a background, and information indicating that the background is divided into a wall, a floor, and a ceiling. The electronic device (501) can obtain and store three-dimensional information (e.g., spatial information) on the physical object (551) and the physical space as described above. The electronic device (501) can store three-dimensional position information of the user (590) in the space together with the spatial information.

[0127] An electronic device (501) according to one embodiment can construct a virtual space (500) based on the physical location of the electronic device (501) and / or the user (590). The electronic device (501) can create the virtual space (500) by referring to the spatial information described above. The electronic device (501) can create a virtual space (500) of the same scale as the physical space based on the spatial information and place objects within the created virtual space (500). The electronic device (501) can provide a complete virtual reality to the user (590) by outputting an image that represents the entire physical space. The electronic device (501) can provide mixed reality (MR) or augmented reality (AR) by outputting an image that represents a part of the physical space. However, although the construction of a virtual space (500) based on spatial information acquired through analysis of the aforementioned physical space is described, the electronic device (501) may also construct a virtual space (500) regardless of the physical location of the user (590). In this specification, the virtual space (500) is a space corresponding to augmented reality or virtual reality, and may also be referred to as a metaverse space.

[0128] For example, the electronic device (501) may provide a virtual graphic representation that replaces at least a portion of a physical space. An electronic device (501) based on optical see-through may output a virtual graphic representation by overlaying the virtual graphic representation on a screen area corresponding to at least a portion of the space on a screen display unit. An electronic device (501) based on video see-through may output an image generated by replacing an image area corresponding to at least a portion of a space image corresponding to a physical space rendered based on spatial information with a virtual graphic representation. The electronic device (501) may replace at least a portion of a background in a physical space with a virtual graphic representation, but is not limited thereto. The electronic device (501) may also perform only additional placement of a virtual object (552) within a virtual space (500) based on spatial information without changing the background.

[0129] The electronic device (501) can place and output a virtual object (552) in a virtual space (500). The electronic device (501) can set an operation area of ​​the virtual object (552) in a space occupied by the virtual object (552) (e.g., a volume corresponding to the appearance of the virtual object (552). The operation area may indicate an area where manipulation of the virtual object (552) occurs. In addition, the electronic device (501) can output a physical object (551) by replacing it with the virtual object (552). The virtual object (552) corresponding to the physical object (551) may have a shape identical to or similar to that of the physical object (551). However, the present invention is not limited thereto, and the electronic device (501) may also set only an operation area in a space occupied by the physical object (551) or in a location corresponding to the physical object (551) without outputting a virtual object (552) that replaces the physical object (551). In other words, the electronic device (501) can transmit visual information representing a physical object (551) (e.g., light reflected from the physical object (551) or an image captured of the physical object (551)) to the user (590) without change, and set a manipulation area on the physical object (551). The manipulation area can be set to the same shape and volume as the space occupied by the virtual object (552) or the physical object (551), but is not limited thereto. The electronic device (501) can also set a manipulation area smaller than the space occupied by the virtual object (552) or the space occupied by the physical object (551).

[0130] According to one embodiment, the electronic device (501) may place a virtual object (552) representing a user (590) (e.g., an avatar object) within the virtual space (500). When the avatar object is provided in a first-person view, the electronic device (501) may visualize a graphical representation corresponding to a part (e.g., a hand, a torso, or a leg) of the avatar object to the user (590) through the aforementioned display (e.g., an optical see-through display or a video see-through display). However, the present invention is not limited thereto, and when the avatar object is provided in a third-person view, the electronic device (501) may also visualize a graphical representation corresponding to the entire shape (e.g., a back view) of the avatar object to the user (590) through the aforementioned display. The electronic device (501) may provide the user (590) with an experience integrated with the avatar object.

[0131] In addition, the electronic device (501) can provide an avatar object of another user who has entered the same virtual space (500). The electronic device (501) can receive feedback information that is the same or similar to feedback information (e.g., information based on at least one of visual, auditory, or tactile senses) provided to another electronic device (501) who has entered the same virtual space (500). For example, when an object is placed in a certain virtual space (500) and multiple users access the virtual space (500), the electronic devices (501) of the multiple users can receive feedback information (e.g., graphical representation, sound signal, or haptic feedback) of the same object placed in the virtual space (500) and provide the feedback information to each user (590).

[0132] The electronic device (501) can detect input to an avatar object of another electronic device (501) and can also receive feedback information from the avatar object of the other electronic device (501). The exchange of input and feedback for each virtual space (500) can be performed by a server (e.g., server (108) of FIG. 1). For example, a server (e.g., a server providing a metaverse space) can transmit input and feedback between an avatar object of a user (590) and an avatar object of another user between users (590). However, the present invention is not limited thereto, and the electronic device (501) can establish direct communication with another electronic device (501) without going through a server to provide input or receive feedback based on an avatar object.

[0133] For example, the electronic device (501) may determine that a physical object (551) corresponding to the selected manipulation area has been selected by the user (590) based on detecting a user input selecting an manipulation area. The user's (590) input may include at least one of a gesture input using a part of the body (e.g., a hand or an eye), an input using a separate virtual reality accessory device, or a user's voice input.

[0134] A gesture input is an input corresponding to a gesture identified based on tracking a body part (510) of a user (590), and may include, for example, an input for pointing or selecting an object. The gesture input may include at least one of a gesture in which a part of the body (e.g., a hand) faces an object for a predetermined period of time or longer, a gesture in which a part of the body (e.g., a finger, an eye, a head) points to an object, or a gesture in which a part of the body and an object make spatial contact. A gesture in which the eye points to an object can be identified based on eye tracking. A gesture in which the head points to an object can be identified based on head tracking.

[0135] Tracking of a body part (510) of a user (590) may be performed primarily based on a camera of the electronic device (501), but is not limited thereto. The electronic device (501) may also track the body part (510) based on the cooperation of sensing data of a vision sensor (e.g., image data of a camera and depth data of a depth sensor) and information collected by an accessory device described below (e.g., controller tracking, finger tracking within the controller). Finger tracking may be performed by sensing the distance or contact between an individual finger and the controller based on a sensor built into the controller (e.g., an infrared sensor).

[0136] The virtual reality accessory device may include a ride-on device, a wearable device, a controller device (520), or other sensor-based device. The ride-on device is a device that a user (590) rides on and operates, and may include, for example, at least one of a treadmill-type device or a chair-type device. The wearable device is a manipulation device that is worn on at least a part of the user's (590) body, and may include, for example, at least one of a full-body and half-body suit-type controller, a vest-type controller, a shoe-type controller, a bag-type controller, a glove-type controller (e.g., a haptic glove), or a face mask-type controller. The controller device (520) may include, for example, an input device (e.g., a stick-type controller or a gun) that is operated by a hand, foot, toe, or other body part (510).

[0137] The electronic device (501) may establish direct communication with the accessory device to track at least one of the location or motion of the accessory device, but is not limited thereto. The electronic device (501) may also communicate with the accessory device via a base station for virtual reality.

[0138] For example, the electronic device (501) may determine that the virtual object (552) has been selected based on detecting an action of gazing at the virtual object (552) for a predetermined period of time or longer through the aforementioned eye gaze tracking technology. As another example, the electronic device (501) may recognize a gesture indicating the virtual object (552) through hand tracking technology. The electronic device (501) may determine that the virtual object (552) has been selected based on whether the direction pointed by the tracked hand indicates the virtual object (552) for a predetermined period of time or longer, or whether the hand of the user (590) contacts or enters an area occupied by the virtual object (552) within the virtual space (500).

[0139] The user's voice input is an input corresponding to the user's voice acquired by the electronic device (501), and may include, for example, voice data sensed by an input module (e.g., a microphone) of the electronic device (501) or received from an external electronic device of the electronic device (501). The electronic device (501) may determine that a physical object (551) or a virtual object (552) has been selected by analyzing the user's voice input. For example, the electronic device (501) may determine that at least one of the physical object (551) or the virtual object (552) corresponding to the detected keyword has been selected based on detecting a keyword indicating at least one of the physical object (551) or the virtual object (552) from the user's voice input.

[0140] The electronic device (501) may provide feedback as described below in response to the user (590) input described above.

[0141] Feedback may include visual feedback, auditory feedback, tactile feedback, olfactory feedback, or gustatory feedback. The feedback may be rendered by a server (108), an electronic device (101), or an external electronic device (102), as described above in FIG. 1.

[0142] Visual feedback may include outputting an image through a display (e.g., a transparent display or an opaque display) of the electronic device (501).

[0143] Auditory feedback may include outputting sound through a speaker of the electronic device (501).

[0144] Tactile feedback may include force feedback that simulates weight, shape, texture, dimension, and dynamics. For example, a haptic glove may include haptic elements (e.g., electrical muscles) that can simulate the sensation of touch by tensing and relaxing the body of a user (590). The haptic elements within the haptic glove may act as tendons. The haptic glove may provide haptic feedback to the entire hand of the user (590). The electronic device (501) may provide feedback indicating the shape, size, and stiffness of an object through the haptic glove. For example, the haptic glove may generate forces that mimic the shape, size, and stiffness of an object. The exoskeleton of the haptic glove (or suit-like device) may include sensors and finger motion measurement devices, and may transmit tactile information to the body by transmitting a force (e.g., an electromagnetic, DC motor, or pneumatic-based force) that pulls a cable to the fingers of the user (590). Hardware providing tactile feedback may include sensors, actuators, power sources, and wireless transmission circuitry. Haptic gloves may operate by inflating and deflating inflatable air bladders on the glove surface.

[0145] The electronic device (501) may provide feedback to the user (590) based on the selection of an object within the virtual space (500). For example, the electronic device (501) may output a graphical representation indicating the selected object (e.g., an expression highlighting the selected object) through the display. As another example, the electronic device (501) may output a sound (e.g., a voice) guiding the selected object through the speaker. As another example, the electronic device (501) may provide the user (590) with a haptic movement that simulates the tactile sensation of the corresponding object by transmitting an electrical signal to a haptic-enabled accessory device (e.g., a haptic glove).

[0146] FIG. 6 is a diagram illustrating an example of a method of bookmarking a page performed by an electronic device according to various embodiments.

[0147] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may, when obtaining a bookmark input for a page, store bookmark information in a memory, including information for accessing the page and information for accessing other pages related to the page.

[0148] In various embodiments of the present disclosure, a page may include at least one of a web page or an application page.

[0149] In one embodiment, the page may include a web page. The web page may be provided as a wired and / or wireless Internet page via an Internet Protocol-based network (e.g., network (199) or server (108) of FIG. 1). The web page may have a uniform resource locator (URL) address for defining a connection path. The web page may be implemented in the form of, for example, a hypertext markup language (HTML) document or an extensible markup language (XML) document. The web page may be part of a website, which is a collection of interconnected web pages.

[0150] In one embodiment, a page may correspond to a space (e.g., a virtual space) provided by an application (e.g., application (146) of FIG. 1). For example, a page may include a page corresponding to a chat room provided by a messaging application. For example, a page may include a page corresponding to a shopping cart provided by a shopping application.

[0151] In operation (610), the electronic device may display a preceding page.

[0152] In operation (620), the electronic device may display a target page after displaying a preceding page. According to one embodiment, the electronic device may display the preceding page and the target page through a display (e.g., a display module (160) of FIG. 1, a first display (205) of FIG. 2, a second display (210) of FIG. 2, a display (320) of FIG. 3, and a display (421) of FIG. 4).

[0153] The target page may refer to a page selected by the user's bookmark input. Bookmark input is described in more detail later in operation (630).

[0154] In one embodiment, a preceding page may refer to a page displayed before a target page. In various embodiments of the present disclosure, "the preceding page being displayed before the target page" may include the point in time when the display of the preceding page begins before the point in time when the display of the target page begins. Similarly, "the target page being displayed after the display of the preceding page" may mean the point in time when the display of the target page begins after the point in time when the display of the preceding page begins.

[0155] For example, the electronic device may display a preceding page from a first point in time, and then stop displaying the preceding page and display a target page from a second point in time that is the same as or subsequent to the first point in time.

[0156] For example, an electronic device may display a preceding page from a first point in time, and then display a target page from a second point in time following the first point in time. The electronic device may display the preceding page and the target page together at a third point in time following the second point in time. For example, when operating in multi-window mode at the third point in time, the electronic device may display the preceding page and the target page individually through corresponding windows.

[0157] In one embodiment, a preceding page may refer to a page corresponding to an operation preceding an operation corresponding to a target page. Each of the preceding page and the target page may correspond to at least some of a plurality of operations constituting a single function. For example, the preceding page may be a page for performing a first operation among the plurality of operations. The target page may be a page for performing a second operation following the first operation among the plurality of operations.

[0158] According to one embodiment, a preceding page may refer to a page where a user's input is acquired before a target page. For example, a user's input acquired through a preceding page may precede a user's input acquired through a target page. In various embodiments of the present disclosure, "a preceding page where a user's input acquired through a target page precedes a user's input acquired through a target page" may include that the acquisition time of the user's input acquired through a preceding page (e.g., an area corresponding to the preceding page) precedes the acquisition time of the user's input acquired through the target page. When a plurality of user inputs are acquired through at least one of the preceding page and the target page, the electronic device may determine the preceding page based on a reference time corresponding to each page. For example, the reference time of the preceding page may precede the reference time of the target page. The reference time may represent the acquisition time(s) of at least one user's input acquired through the corresponding page. For example, the reference time may include at least one of the first time of acquiring a user's input on a corresponding page or the last time of acquiring a user's input on a corresponding page.

[0159] In operation (630), the electronic device may obtain a user's bookmark input for bookmarking a target page.

[0160] In operation (640), the electronic device may store bookmark information including information for accessing a target page in a memory (e.g., memory (130) of FIG. 1).

[0161] A bookmark input may refer to a user input requesting to bookmark a target page. In various embodiments of the present disclosure, "bookmarking a page" may refer to storing information about the page.

[0162] According to one embodiment, the user's input (e.g., bookmark input) may include at least one of a button input, a motion input, a gesture input, or a voice input.

[0163] A button input may refer to an input detected through a physical button included in an electronic device or an external device (e.g., an input module (150) of FIG. 1, a controller device (520) of FIG. 5) that is connected to the electronic device through communication. For example, the electronic device or the external device may include a button assigned to a bookmark function, and may detect a bookmark input when it detects that the button has been pressed.

[0164] Motion input may refer to a motion applied to an electronic device or an external device (e.g., input module (150) of FIG. 1, controller device (520) of FIG. 5) that is connected to the electronic device through communication. The motion input may include an input detected when a user rotates, tilts, and / or moves the electronic device or the external device. For example, a specific motion may be pre-assigned to a bookmark function, and a bookmark input may be detected when the electronic device or the external device detects that the specific motion has occurred. For example, the specific motion may be pre-determined as a motion that moves in a predetermined direction (e.g., from bottom to top based on the posture of the electronic device) from an area corresponding to a target page.

[0165] Gesture input may refer to an input corresponding to a gesture identified based on tracking of a user's body part (e.g., fingers, eyes, head). Gesture input may include a gesture input corresponding to a gesture of pointing and / or dragging an object with a user's body part. For example, a specific gesture may be pre-assigned to a bookmark function, and a bookmark input may be detected when the occurrence of the specific gesture is detected on an electronic device or an external device. For example, a specific gesture may be pre-determined as a gesture of dragging from a point included in an area corresponding to a target page in a predetermined direction (e.g., from bottom to top based on the posture of the electronic device) with the user's eye or finger.

[0166] Voice input can refer to input corresponding to a user's voice acquired through an electronic device or an external device connected to the electronic device through communication. The electronic device can analyze the voice input to determine whether the user's voice input requests a bookmark for a target page. If the user's intent, as determined based on the voice input, is to bookmark the target page, the electronic device can acquire the voice input as a bookmark input.

[0167] According to one embodiment, an electronic device can obtain voice data corresponding to a user's voice. The electronic device can analyze the voice data using a natural language platform. The natural language platform may include an automatic speech recognition module (ASR module) and / or a natural language understanding module (NLU module).

[0168] For example, an automatic speech recognition module can convert a user's speech input into text data. A natural language understanding module can use the text data of the speech input to determine the user's intent. For example, the natural language understanding module can perform syntactic analysis or semantic analysis to determine the user's intent. In one embodiment, the natural language understanding module can use linguistic features (e.g., grammatical elements) of morphemes or phrases to determine the meaning of words extracted from the speech input, and can match the meaning of the determined words to the intent to determine the user's intent.

[0169] Bookmark input is not limited to being composed of one of button input, motion input, gesture input, or voice input, and may be a user input that combines two or more of button input, motion input, gesture input, or voice input. User input that combines two or more of button input, motion input, gesture input, or voice input may also be expressed as 'multimodal input'.

[0170] For example, a bookmark input may be a combined input of a gesture input (e.g., gaze input) in which the user points the eye at an area corresponding to a target page and a button input in which a physical button assigned to the bookmark function among physical buttons included in an electronic device and / or an external device (e.g., a controller) is pressed.

[0171] For example, a bookmark input may include a target page designation input including a gaze input pointing the user's gaze at an area corresponding to a target page, and a bookmark trigger input for bookmarking a target page specified by a predetermined gesture (e.g., a swipe gesture in a predetermined direction) by the user's body (e.g., a finger).

[0172] Bookmark information may include information about the target page. Information about the page (e.g., the target page, the preceding page) may include information for accessing the page. For example, information for accessing the page may include information for defining the access path to the page. Bookmark information may include information about user input obtained through the target page.

[0173] For example, if the page is a web page, the bookmark information may include the URL address of the page. Information about the page may include at least one of the following: the page's domain name, a thumbnail, the page's title, or an icon (e.g., a favicon) for the page.

[0174] For example, if the page is an application page, the bookmark information may include information for displaying the page. Information about the page may include at least one of information about the application or a server corresponding to the application, information about a user account, a thumbnail of the page, a title of the page, or an icon for the page. A user account is created by signing up for a service corresponding to the application, and the server may act as a service platform that provides services to user accounts signed up for the service.

[0175] In operation (650), the electronic device may add information for accessing the preceding page to the bookmark information based on the relationship between the preceding page and the target page. The relationship between the preceding page and the target page may include whether the preceding page contributed to the display of the target page and / or the extent to which the display of the preceding page contributed.

[0176] For example, an electronic device may determine (e.g., analyze) the relevance between a preceding page and a target page. If the electronic device determines that the preceding page is related to the target page, it may add information for accessing the preceding page to the bookmark information on the target page.

[0177] An electronic device according to one embodiment may determine that a preceding page is related to a target page based on similarity between content included in the preceding page and content included in the target page. A page (e.g., the preceding page, the target page) may include multiple contents. The contents may include at least one of text, an image, a video (e.g., multiple image frames), or an interfacing object. The interfacing object may include a component implemented to interact with a user, such as a button or icon implemented to display other contents and / or other pages in response to a user input.

[0178] An electronic device according to one embodiment may determine that a preceding page is related to a target page based on whether the target page is accessible through the preceding page. For example, the electronic device may determine that the preceding page is related to the target page if the target page is linked to content (e.g., an interfacing object) included in the preceding page. Linking the target page to content included in the preceding page may include the target page being mapped to content included in the preceding page, and information defining an access path to the target page being accessible when an event related to the content included in the preceding page is detected (e.g., a user input is detected). For example, the preceding page may include a button linking to the target page. If the electronic device detects a user input on the button included in the preceding page, the electronic device may display the target page.

[0179] The electronic device may determine that a preceding page that is directly accessible to the target page, as well as a preceding page that is indirectly accessible to the target page by going through another preceding page, are related to the target page. In various embodiments of the present disclosure, "being directly accessible to the target page through the preceding page" may mean that content included in the preceding page is directly linked to the target page. Furthermore, in various embodiments of the present disclosure, "being indirectly accessible to the target page through the preceding page" may mean that content included in the preceding page is linked to another preceding page, and that content included in another preceding page is linked to the target page.

[0180] An electronic device according to one embodiment can determine a relevance between a target page and a preceding page based on information received from a server providing the target page (e.g., server 108 of FIG. 1). The electronic device can display the target page by transmitting information for accessing the target page to the server and receiving information for displaying the target page from the server. The electronic device can further receive, from the server, information about a preceding page related to the target page along with the information for displaying the target page. The electronic device can determine that the preceding page is related to the target page based on the information about the preceding page received from the server.

[0181] An electronic device according to one embodiment may receive a display command for a target page from an external device. When receiving a display command for a target page from an external device, the electronic device may determine a relationship between a preceding page and a target page based on at least one of information received from the external device, information determined by the electronic device, or information received from a server providing the page. An embodiment in which the electronic device receives a display command for a target page from an external device is described in more detail below with reference to FIG. 7.

[0182] An electronic device according to one embodiment may determine a relevance between a preceding page and a target page based on whether the preceding page and the target page correspond to at least some of a plurality of operations constituting a single function. An example of an operation for determining a relevance between pages based on a plurality of operations constituting a single function is described in more detail below with reference to FIG. 10.

[0183] An electronic device according to one embodiment may determine the relevance between a preceding page and a target page based on the relevance between search requests corresponding to the preceding page and the target page, if each of the preceding page and the target page includes search results for corresponding search requests. An example of the operation of determining the relevance between pages based on the relevance between search requests is described in more detail below in FIG. 12.

[0184] An electronic device according to one embodiment may display a bookmark object indicating that a target page has been bookmarked based on stored bookmark information. The bookmark object may include at least a portion of the contents included in the target page. The bookmark object may include representative content representing the contents included in the target page. The representative content may be content selected from among the contents included in the target page, or content generated to summarize and / or describe the contents included in the target page.

[0185] An electronic device may display a bookmark object in a portion of a display area of ​​a display. The display area may refer to an area displayed through the display of the electronic device. According to one embodiment, the display area may include an area determined to be a user's field of view (FOV). The electronic device may determine an area determined to be a user's field of view as a display area among images acquired through a photographing camera (e.g., the third camera (245) of FIG. 2, the second functional cameras (411, 412) of FIGS. 4A and 4B), and display the determined display area through the display.

[0186] Some areas may be predetermined as areas for displaying bookmark objects and / or may be determined by the electronic device as areas for displaying bookmark objects based on bookmark input.

[0187] An electronic device may display a preceding page and a target page based on obtaining a user input for a displayed bookmark object. For example, the electronic device may display the preceding page and the target page in response to a user input for selecting a displayed bookmark object. In the display area, at least a portion of a window corresponding to the preceding page may be arranged to overlap at least a portion of a window corresponding to the target page. According to one embodiment, in the display area, in an overlapping area between the window corresponding to the preceding page and the window corresponding to the target page, the window corresponding to the target page may occlude the window corresponding to the preceding page.

[0188] According to various embodiments, an electronic device may include a first electronic device and a second electronic device. The first electronic device and the second electronic device may establish wired communication and / or wireless communication. The first electronic device may include a wearable device (e.g., an AR device, a VR device, an XR device, a VST device) worn by a user, and the second electronic device (e.g., the electronic device (101) of FIG. 1) may include a mobile terminal (e.g., a tablet or a smartphone) of the user. At least a portion (e.g., operation (610), operation (620), operation (630)) of the above-described operations (610, 620, 630, 640, 650) may be performed by the first electronic device, and the remaining portion (e.g., operation (640), operation (650)) may be performed by the second electronic device. From the perspective of the first electronic device, the second electronic device may be represented as an external device, and similarly, from the perspective of the second electronic device, the first electronic device may be represented as an external device.

[0189] FIG. 7 is a diagram illustrating an example of an operation of an electronic device providing a display history of a page according to various embodiments.

[0190] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may display at least a portion of the preceding page together with the target page when displaying a target page after displaying a preceding page.

[0191] The electronic device can display a first page through a window (710). The electronic device can further display a display history object (e.g., a display history object having a first graphical representation (711)) together with the first page through the window (710). The display history object may refer to a graphical object (e.g., an interfacing object) for switching whether a display history view is enabled.

[0192] A display history view may refer to an interface that visually represents the history of pages displayed by an electronic device. In one embodiment, a display history view may be configured for a window, and the electronic device may provide an interface that visually represents pages displayed through a window in which a display history view is enabled.

[0193] For example, an electronic device may display a preceding page through a primary window and then display a target page through the primary window. If the display history view is disabled, the electronic device may not create an additional window for the preceding page and / or may not display at least a portion of the preceding page. If the display history view is enabled, the electronic device may create an additional window for the preceding page and display at least a portion of the preceding page through the created additional window. The additional window and / or the preceding page displayed through the additional window may overlap with the primary window and / or the target page displayed through the primary window. In the overlapping area, the preceding page displayed through the additional window and / or the additional window may be obscured by the primary window and / or the target page displayed through the primary window.

[0194] According to one embodiment, the display history object may have at least one of a first graphical representation (711) or a second graphical representation (721). For example, the first graphical representation (711) may indicate disabling the display history view, and the second graphical representation (721) may indicate enabling the display history view. The electronic device may perform a switch between enabling and disabling the display history view based on obtaining user input for the display history object.

[0195] For example, as shown in FIG. 7, the electronic device may display a first page through a first window (710) (e.g., a basic window). If the display history view is disabled, the electronic device may display a display history object having a first graphical representation (711). The electronic device may obtain a user input for the display history object having the first graphical representation (711).

[0196] An electronic device may enable a display history view based on a user input (712) for a display history object. The electronic device may change the display of a display history object included in a first window (720) (e.g., a basic window) from a first graphical representation (711) to a second graphical representation (721). The electronic device may obtain the user input through a first page corresponding to the first window (720). The user input may request the display of a page different from the first page. In FIG. 7, for example, the first page may be a page providing a search service, and the user input may include a search request including a search term.

[0197] Although not explicitly illustrated in FIG. 7, the electronic device may display a display history object of a second graphical representation (721) when display history viewing is enabled. When the electronic device obtains a user input (722) for a display history object having a second graphical representation (721), the electronic device may disable the display history viewing and change the display of the display history object from the second graphical representation (721) to the first graphical representation (711).

[0198] The electronic device may display a second page corresponding to the user's input (722) through a first window (730) (e.g., a basic window). As an example in FIG. 7, the second page may include search results for a search request included in the user's input (722). The electronic device may display at least a portion of the first page together with the second page through a second window (731) (e.g., a first additional window) based on the display history view being enabled for the first window (730). As illustrated in FIG. 7, as an example, the electronic device may display at least a portion of the first page through the second window (731), and at least a portion of the second window (731) may be obscured by at least a portion of the first window (730). The size of the second window (731) may be smaller than that of the first window (730). The electronic device may obtain the user's input including the additional search request through the second page corresponding to the first window (730).

[0199] The electronic device may display a third page corresponding to the user's input through the first window (740) (e.g., the basic window). As an example in FIG. 7, the third page may include additional search results for an additional search request included in the user's input. Based on the display history view being enabled for the first window (740), the electronic device may display at least a portion of the second page together with the third page through the third window (742) (e.g., the second additional window). As an example, as shown in FIG. 7, based on the display history view being enabled, the electronic device may display at least a portion of the first page and at least a portion of the second page that were displayed through the first window (740) (e.g., the basic window) through the second window (741) (e.g., the first additional window) and the third window (742) (e.g., the second additional window), respectively.

[0200] An electronic device according to one embodiment may determine the sizes of windows corresponding to pages based on the order in which they are displayed through the first window (740). For example, the electronic device may determine the size of a window (e.g., a second window (741)) corresponding to a page (e.g., a first page) displayed first through the first window (740) to be smaller than or equal to the size of a window (e.g., a third window (742)) corresponding to a page (e.g., a second page) displayed later through the first window (740).

[0201] When the electronic device displays a third page through the first window (740), the electronic device can obtain a bookmark input (743) for the third page. In FIG. 7, the bookmark input (743) can be obtained through a motion or gesture of dragging from a point included in an area corresponding to the third page or the first window (740) to a point outside the area corresponding to the third page or the first window (740). When the electronic device obtains the bookmark input (743), the electronic device can process the third page specified by the bookmark input (743) as the target page.

[0202] The electronic device may store bookmark information including information for accessing a third page in its memory based on the bookmark input (743). The electronic device may add information for accessing other pages (e.g., information for accessing the first page, information for accessing the second page) to the bookmark information based on the relationship between the third page and other pages (e.g., the first page, the second page).

[0203] An electronic device according to one embodiment may determine that a preceding page is related to a target page based on a display history view. For example, the electronic device may obtain a bookmark input (743) for a target page displayed through a main window. If the electronic device displays at least a portion of the preceding page along with the target page based on the display history view enabled for the main window, the electronic device may determine that the preceding page is related to the target page. The electronic device may add information for accessing the preceding page to the bookmark information of the target page.

[0204] The electronic device may display a bookmark object (751) for a third page based on the stored bookmark information. According to one embodiment, the bookmark object (751) may have a first bookmark representation (e.g., a graphical representation of the bookmark object (751) illustrated in FIG. 7) if the bookmark information further includes information for accessing a preceding page in addition to information for accessing a target page. The bookmark object (751) may have a second bookmark representation if the bookmark information includes information for accessing the target page and does not include information for a preceding page. The first bookmark representation and the second bookmark representation are each different graphical representations that the bookmark object (751) may have, wherein the first bookmark representation may include a form in which multiple windows are stacked, and the second bookmark representation may include a form in which a single window is formed.

[0205] The electronic device may change the page displayed through the first window (750) (e.g., the basic window) to a preceding page based on the stored bookmark information. The electronic device may display the second page that was last displayed before the third page was displayed through the first window (750). According to one embodiment, if the change of the page displayed through the first window (750) is performed based on a bookmark function (e.g., acquisition of bookmark input, storage of bookmark information), the electronic device may apply the bookmarked page to the display history view as a preceding page, thereby limiting the display of at least a portion of the bookmarked page.

[0206] For example, if the electronic device stores bookmark information for a third page and changes the page displayed through the first window (750) from the third page to the second page, the electronic device may skip the creation of an additional window for the third page and the display of at least a portion of the third page. The electronic device may stop displaying an additional window (e.g., the third window (742)) for a preceding page (e.g., the second page) to be displayed through the first window (750) and display the preceding page through the first window (750). If there is a preceding page (e.g., the first page) different from the preceding page to be displayed through the first window (750), the electronic device may maintain displaying at least a portion of the other preceding page through an additional window (e.g., the second window (751)) for the other preceding page. For example, as shown in FIG. 7, the electronic device may display the second page through the first window (750) and display at least a portion of the first page through the second window (751). The electronic device may display an interface prior to displaying the third page (e.g., an interface displaying the second page through the first window (730) and displaying at least a portion of the first page through the second window (731)) based on the third page being bookmarked.

[0207] However, and not limited thereto, even if the third page is bookmarked, the electronic device may create an additional window for the third page and display at least a portion of the third page together with the current page (e.g., the second page displayed through the first window (750) in FIG. 7) based on the display history view for the first window (750) being enabled.

[0208] The electronic device according to various embodiments of the present disclosure is not limited to changing the page displayed through the default window to the preceding page based on the stored bookmark information. In one embodiment, the electronic device may maintain displaying the third page through the first window (750) after storing the bookmark information, or may display a page other than the first page, the second page, and the third page (e.g., a page set as the default page).

[0209] FIG. 8 is a drawing illustrating an example of an operation of displaying a target page when an electronic device receives a display command for a target page from an external device according to various embodiments.

[0210] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may receive a display command for a target page from an external device (e.g., electronic device (102) of FIG. 1, electronic device (104) of FIG. 1). The external device may be connected to the electronic device via wired and / or wireless communication. For example, the external device may include at least a part of the configuration of the electronic device (101) of FIG. 1.

[0211] When an electronic device receives a display command for a target page from an external device, the electronic device may determine another page (e.g., a preceding page) to be displayed together with the target page. According to one embodiment, the display of the electronic device may include a first display (e.g., the first display (205) of FIG. 2) that outputs a right image corresponding to the user's right eye and a second display (e.g., the second display (210) of FIG. 2) that outputs a left image corresponding to the user's left eye. The electronic device may display the page through a window arranged in a virtual space (e.g., the virtual space (500) of FIG. 5). When the electronic device displays the target page together with the target page, the target page and the preceding page may be displayed through windows aligned within the virtual space. For example, the window corresponding to the target page and the window corresponding to the preceding page within the virtual space may be aligned to have the same height and / or size.

[0212] However, the electronic device according to one embodiment is not limited to providing the user with an image rendering a virtual space, and the electronic device may also display one or more pages through a single display.

[0213] An electronic device according to one embodiment may further display a target page together with a preceding page based on a relationship between the target page and a preceding page displayed before the target page on an external device.

[0214] For example, an electronic device may receive a display command for a target page from an external device that displays a target page after displaying a preceding page. The electronic device may display the target page based on the display command. The electronic device may further display the preceding page along with the target page based on a relationship between the preceding page and the target page. The preceding page may include a page that was displayed by the external device, even if it was not displayed by the electronic device. The electronic device may determine a relationship between the preceding page and the target page.

[0215] As described above in FIG. 6, the electronic device may determine the relevance between the preceding page and the target page based on at least one of the following: similarity between contents included in the preceding page and the target page, whether the target page is accessible through the preceding page, whether the preceding page and the target page correspond to at least some of a plurality of operations constituting one function, or relevance between search requests.

[0216] An electronic device according to one embodiment can determine a relevance between a target page and a preceding page based on information received from a server providing the target page.

[0217] For example, an electronic device may receive a display command for a target page from an external device that displays the target page. The electronic device may display the target page based on the display command. The electronic device may display the target page through a window arranged in a virtual space. The electronic device may receive information about a preceding page as a page related to the target page from a server that provides the target page (e.g., server 108 of FIG. 1). Based on receiving the information about the preceding page, the electronic device may further display the preceding page. Based on receiving the information about the preceding page, the electronic device may further display the preceding page together with the target page through windows arranged in a virtual space. The information about the preceding page may include information for accessing the preceding page. For example, the server may transmit a display command for further displaying the preceding page together with the target page to the electronic device. If the electronic device receives a display command for further displaying the preceding page together with the target page from the server, the electronic device may determine that the preceding page is related to the target page and further display the preceding page.

[0218] According to one embodiment, when an electronic device obtains a bookmark input for a target page, even if the preceding page is not directly displayed by the electronic device, if the preceding page is displayed by an external device, the electronic device can determine the relevance between the preceding page and the target page. If the electronic device determines that the preceding page is related to the target page, the electronic device can add information for accessing the preceding page to the bookmark information of the target page.

[0219] An electronic device according to one embodiment can determine a relevance between a target page and a preceding page based on an external device simultaneously displaying the target page and the preceding page through a split screen.

[0220] For example, an electronic device may receive a display command for a preceding page and a target page from an external device that simultaneously displays the preceding page and the target page through individually corresponding windows. The electronic device may display the preceding page and the target page through windows arranged in a virtual space. The electronic device may determine whether a preceding page that was displayed simultaneously with the target page exists based on obtaining a bookmark input for the target page. The electronic device may determine whether each of the preceding page and the target page includes a search result for a search request corresponding to the corresponding page based on the existence of a preceding page that was displayed simultaneously with the target page by the external device. For example, the electronic device may determine whether the preceding page includes a search result for a search request corresponding to the preceding page and whether the target page includes a search result for a search request corresponding to the target page. The electronic device may determine that the preceding page is related to the target page based on the presence of the search result for the preceding page and the target page. For example, when obtaining a bookmark input for the target page, the electronic device may store bookmark information that includes information for accessing the target page. The electronic device may add information for accessing the preceding page to the bookmark information based on the existence of a preceding page displayed together with the target page by an external device, and the preceding page and the target page including search results.

[0221] Referring to FIG. 8, the external device (801) may display a first page (811) and then display a second page (812). According to one embodiment, multiple pages may correspond to at least some of multiple operations constituting one function. For example, a first operation (represented as STEP 1 in FIG. 8), a second operation (represented as STEP 2 in FIG. 8), a third operation (represented as STEP 3 in FIG. 8), and a fourth operation (represented as STEP 4 in FIG. 8) may constitute one function. The first operation may correspond to the first page (811). The second operation may correspond to the second page (812). The third operation may correspond to the third page (813). The fourth operation may correspond to the fourth page (814).

[0222] An external device (810) can transmit a display command for a second page (812) to an electronic device (not shown). The electronic device can receive a display command for a second page (822). The electronic device can display the second page (822) through a screen (820). For example, the electronic device can display the second page (822) through a window arranged in a virtual space. The electronic device can further display the first page (821) together with the second page (822) through windows arranged in a virtual space based on a relationship between the first page (821) and the second page (822) through the screen (820).

[0223] In various embodiments of the present disclosure, the electronic device primarily describes, but is not limited to, displaying and / or bookmarking a preceding page of the target page along with the target page. The electronic device determines a relevance between the target page and a subsequent page based on information determined by the electronic device or information received from a server, and if the subsequent page is related to the target page, further displays and / or bookmarks the subsequent page of the target page. The subsequent page may refer to a page displayed later than the target page. The subsequent page may refer to a page corresponding to an operation subsequent to an operation corresponding to the target page. The operation of determining the relevance between the target page and the subsequent page may be performed similarly to the operation of determining the relevance between the target page and the preceding page. For example, the relevance between the target page and the subsequent page may be determined based on at least one of the following: similarity between contents included in the target page and the subsequent page, whether the subsequent page is accessible through the target page, whether the target page and the subsequent page correspond to at least a portion of a plurality of operations constituting a single function, relevance between search requests, or information received from a server providing the target page and / or the subsequent page.

[0224] As shown in FIG. 8, by way of example, through the screen (820), the electronic device can further display subsequent pages (e.g., a third page (823), a fourth page (824)) related to the second page (822) along with the second page (822).

[0225] FIG. 9 illustrates an example of an operation in which an electronic device, according to various embodiments, bookmarks a page for at least some of a plurality of operations constituting a function and then re-performs the function.

[0226] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may display pages corresponding to a plurality of operations constituting one function.

[0227] For example, a first action (represented as STEP 1 in FIG. 9), a second action (represented as STEP 2 in FIG. 9), a third action (represented as STEP 3 in FIG. 9), a fourth action (represented as STEP 4 in FIG. 9), and a fifth action (represented as STEP 5 in FIG. 9) may constitute one function. The first action may correspond to the first page (911-1). The second action may correspond to the second page (911-2). The third action may correspond to the third page (911-3). The fourth action may correspond to the fourth page (911-4). The fifth action may correspond to the fifth page (911-5).

[0228] An electronic device according to one embodiment may display a page (e.g., a target page, a preceding page) through a window arranged in a virtual space (e.g., a virtual space (500) of FIG. 5). The electronic device may arrange a window corresponding to the page within a board area (911) arranged in the virtual space (hereinafter, also referred to as “board (911)”). The board (911) may refer to a space where main content to be provided to a user is arranged. The board (911) may be set as a two-dimensional flat area within the virtual space, for example. The board (911) may be maintained even if the user’s line of sight and / or the heading direction of the user (or the electronic device) changes. However, the board (911) is not limited to being arranged in a fixed position and / or posture within the virtual space, and the board (911) may change its position and / or posture within the virtual space based on a user’s input.

[0229] According to one embodiment, the display area (910) of the electronic device can be divided into a first area and a second area.

[0230] The first region corresponds to the board (911) of the virtual space and may refer to an area of ​​the display where the board (911) is displayed. The second region may refer to an area of ​​the display where the board (911) is displayed. For example, the second region may be a remaining area of ​​the first region of the display. An electronic device according to one embodiment may display primary content to be provided to a user in the first region and supplementary content to be provided to the user in the second region. The distinction between primary content and supplementary content may be performed by the user and / or the electronic device. In various embodiments of the present disclosure, the primary content may include a preceding page and / or a target page, and the supplementary content may include a bookmark object (912).

[0231] However, electronic devices are not limited to displaying pages through windows arranged in a virtual space. For example, the electronic device may include a display, and may determine a portion of the display area (e.g., display area (910), display area (920)) of the display as a board area (911).

[0232] Referring to FIG. 9, the electronic device can display a plurality of pages (e.g., a first page (911-1), a second page (911-2), a third page (911-3), a fourth page (911-4), and a fifth page (911-5)) in a first area (911).

[0233] When an electronic device displays pages corresponding to multiple operations constituting a single function, the electronic device may apply a visual effect to the current page corresponding to the currently performed operation to distinguish it from other pages. For example, in FIG. 9, the current page may be the fifth page (911-5), which may be displayed through a window larger than other pages (e.g., the first page (911-1), the second page (911-2), the third page (911-3), and the fourth page (911-4)). The border of the current page may be displayed with a different type of line than the borders of other pages. For example, the border of the current page may be displayed with a thicker line than the borders of other pages.

[0234] An electronic device may obtain a bookmark input for a fifth page (911-5). The electronic device may store bookmark information including information for accessing the fifth page (911-5) as a target page. The electronic device may add information for accessing the preceding page to the bookmark information based on a relationship between the target page (e.g., the fifth page (911-5)) and a preceding page (e.g., the first page (911-1), the second page (911-2), the third page (911-3), and the fourth page (911-4)). For example, the electronic device may determine that the first page (911-1), the second page (911-2), the third page (911-3), the fourth page (911-4), and the fifth page (911-5) are related to the fifth page (911-5) based on the fact that the first page (911-1), the second page (911-2), the third page (911-3), and the fourth page (911-4) individually correspond to multiple operations that constitute a single function. The electronic device may display a bookmark object (912) in an area different from the first area (911) based on the stored bookmark information. The bookmark object (912) may include a graphical representation ('STEP 5' in FIG. 9) indicating that the fifth page (911-5) corresponds to the fifth operation.

[0235] An electronic device may re-perform a function based on bookmarking pages corresponding to at least some of the multiple operations constituting a single function. For example, in FIG. 9, based on bookmarking the fifth page (911-5), the electronic device may display the first page (921-1), the second page (921-2), the third page (921-3), the fourth page (921-4), and the fifth page (921-5) to re-perform the function. The electronic device may perform the first operation corresponding to the first page (921-1). The electronic device may display the first page (921-1) as the current page by applying a visual effect that distinguishes it from other pages. As shown in FIG. 9, the electronic device can display a first page (921-1), a second page (921-2), a third page (921-3), a fourth page (921-4), and a fifth page (921-5) in a first area (921) among the display areas (910), and can display a bookmark object (912) in an area other than the first area (921).

[0236] An electronic device can store bookmark information for multiple pages and display multiple bookmark objects. For example, as shown in FIG. 9, after re-performing a function, the electronic device can obtain a bookmark input for a fifth page (921-5). The electronic device can store bookmark information including information for accessing the fifth page (921-5) as a target page. The electronic device can add information for accessing a previous page to the bookmark information. For example, the electronic device can add information for accessing the fifth page (921-5) to the bookmark information for the first page (921-1), the second page (921-2), the third page (921-3), and the fourth page (921-4). Based on the stored bookmark information, the electronic device can display a bookmark object (922) in an area different from the first area (911). The bookmark object (922) may include a graphical representation ('STEP 5' in FIG. 9) indicating a fifth page (921-5) corresponding to the fifth action.

[0237] FIG. 10 is a drawing illustrating an example of an operation in which an electronic device according to various embodiments performs multiple operations constituting one function.

[0238] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) can perform a plurality of operations constituting one function.

[0239] For example, in FIG. 10, the first action (e.g., schedule selection action), the second action (e.g., passenger information input action), the third action (e.g., convenience facility selection action), the fourth action (e.g., input information confirmation action), and the fifth action (e.g., payment information input action) may constitute a train reservation function.

[0240] Multiple actions may individually correspond to multiple pages. For example, a first action may correspond to a first page (e.g., page 1 (1011)), a second action may correspond to a second page (e.g., page 1 (1012)), a third action may correspond to a third page (e.g., page 1 (1013)), a fourth action may correspond to a fourth page (e.g., page 1 (1014)), and a fifth action may correspond to a fifth page (e.g., page 1 (1015)).

[0241] According to one embodiment, an electronic device may receive information from a server providing a plurality of pages, indicating that the plurality of pages correspond to at least some of the plurality of operations constituting a single function. Based on the information received from the server, the electronic device may determine that the plurality of pages perform at least some of the plurality of operations constituting a single function. In FIG. 10, as an example, the electronic device may receive information from a server providing a train reservation function, indicating that the first to fifth pages correspond to at least some of the plurality of operations constituting the train reservation function.

[0242] According to one embodiment, the electronic device may determine that one page and another page correspond to at least some of a plurality of operations constituting a function, based on the content included in another page being determined based on a user input entered on one page. In FIG. 10, as an example, the electronic device may determine that pages of the first to fifth pages correspond to at least some of a plurality of operations constituting a function, based on the user input entered on the first page (e.g., a user input specifying a starting point, a user input specifying a destination) being necessary for determining the content of the fifth page (e.g., a payment amount).

[0243] According to one embodiment, when an electronic device obtains a bookmark input for a target page among pages corresponding to a plurality of operations constituting one function, the electronic device can determine a relevance between the pages based on the operations performed on the pages.

[0244] For example, an electronic device may display a preceding page. The electronic device may obtain a user input from the preceding page. The electronic device may perform a first operation among a plurality of operations constituting a function based on the user input obtained from the preceding page. The electronic device may display a target page. Following the first operation, the electronic device may perform a second operation based on the user input obtained from the target page. The electronic device may obtain a bookmark input for the target page. The electronic device may determine that the preceding page is related to the target page based on the target page including content determined based on the result of performing the first operation and the user input obtained through the target page performing the second operation among the plurality of operations. The electronic device may add information for accessing the preceding page to the bookmark information.

[0245] Referring to FIG. 10, for example, an electronic device may display pages corresponding to a plurality of operations constituting a function in order to provide a function (e.g., a train reservation function). The electronic device may display a first page (1011), a second page (1012), a third page (1013), a fourth page (1014), and a fifth page (1015) in a board area (1010). The electronic device may perform a first operation (e.g., a schedule selection operation) that is the most preceding among the plurality of operations. The electronic device may display the first page (1011) in the board area (1010) as a current page corresponding to the currently performed operation (e.g., the first operation) by applying a visual effect that distinguishes it from the remaining pages.

[0246] After completing a first action, the electronic device may perform a second action (e.g., a passenger information input action) subsequent to the first action. The electronic device may display a second page (1022) in the board area (1010) as the current page corresponding to the action currently being performed (e.g., the second action) by applying a visual effect that distinguishes it from the remaining pages.

[0247] The electronic device may perform a fifth action (e.g., a payment information input action) after completing the first action, the second action, the third action, and the fourth action. The electronic device may display a fifth page (1035) in the board area (1010) as the current page corresponding to the currently performed action (e.g., the fifth action) by applying a visual effect that distinguishes it from the remaining pages. By displaying the first to fifth pages (1035) together in the board area (1010), the electronic device may provide not only a user input entered in relation to the fifth action, but also a user input obtained in the first, second, third, and / or fourth actions.

[0248] An electronic device may obtain a bookmark input for a fifth page (1045) displayed in a board area (1010). The electronic device may store bookmark information including information for accessing the fifth page (1045) and / or information regarding a user input obtained on the fifth page, and may display a bookmark object (1046) in an area other than the board area (1010). The electronic device may add information for accessing other pages to the bookmark information based on determining that other pages (e.g., a first page, a second page, a third page, a fourth page) are related to the fifth page (1045).

[0249] The electronic device can re-perform the function in the board area (1010) after bookmarking the fifth page. The electronic device can display the first to fifth pages for re-performing the function while displaying the bookmark object (1056). When re-performing the function, the electronic device can display at least a portion of the bookmark information if it detects an input to the bookmark object (1056). For example, if the electronic device detects a user input to the bookmark object (1056), it can display the results of performing at least a portion of the first to fifth operations performed through at least a portion of the first to fifth pages.

[0250] FIGS. 11A and 11B illustrate example interfaces for displaying multiple pages together according to various embodiments.

[0251] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may display a plurality of pages corresponding to a plurality of operations constituting one function.

[0252] An electronic device may display multiple pages (e.g., a first page, a second page, a third page, a fourth page, and a fifth page). The multiple pages may individually correspond to multiple actions, and the multiple actions may constitute a single function.

[0253] An electronic device may display a page corresponding to the currently performed action (e.g., the current page) by applying visual effects that distinguish it from other pages.

[0254] For example, the window corresponding to the current page may have a larger size than the windows corresponding to other pages.

[0255] For example, the border of a window corresponding to the current page may be displayed with a different type of line than the border of a window corresponding to another page. For example, the border of a window corresponding to another page may be displayed with a dotted line, while the border of a window corresponding to the current page may be displayed with a solid line. For example, the border of a window corresponding to another page may be displayed with a thinner line than the border of a window corresponding to the current page.

[0256] For example, a window corresponding to the current page can be positioned at a preset location. The preset location may be a location determined to be highly accessible to the user. For example, the preset location may be at or near the center of the display's display area (or board area).

[0257] For example, as shown in FIG. 11a, the electronic device may display a third page (1113a) corresponding to a currently performed operation (e.g., a third operation) among a plurality of operations in a board area (1110a) by applying a visual effect that distinguishes the third page (1113a) from other pages (e.g., a first page (1111a), a second page (1112a), a fourth page (1114a), and a fifth page (1115a)).

[0258] For example, as shown in FIG. 11b, the electronic device may display a third page (1113b) corresponding to a currently performed operation (e.g., a third operation) among a plurality of operations in a board area (1110b) by applying a visual effect that distinguishes the third page (1113b) from other pages (e.g., a first page (1111b), a second page (1112b), a fourth page (1114b), and a fifth page (1115b)). The electronic device may utilize a sense of space to arrange the current page among the plurality of pages so that it is emphasized.

[0259] FIG. 12 is a diagram illustrating an example of an operation of an electronic device according to various embodiments to determine relevance between search screens based on relevance between search requests.

[0260] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may display a page including search results for a search request. The electronic device may determine a relevance between pages based on a relevance between the search requests, if each of the preceding page and the target page includes a search result for the search request.

[0261] A search request may be generated based on user input. The search request may include text (e.g., search terms), images (e.g., search images), or search options. The search options may refer to options selected by the user from among a plurality of predetermined candidate options. For example, the plurality of candidate options may individually correspond to "registered within 1 day," "registered within 7 days," and "registered within 30 days," and the search option may be determined as "registered within 30 days" based on the user input.

[0262] For example, an electronic device may obtain user input. The electronic device may display a preliminary page containing search results for a preliminary search request generated based on the user input.

[0263] The electronic device may obtain further user input after displaying a preceding page. The electronic device may then display a target page containing search results for a target search request generated based on the user's further input.

[0264] An electronic device can obtain a bookmark input for a target page. The electronic device can determine the relevance between the target page and a preceding page. For example, the electronic device can determine whether to add information about the preceding page based on the relevance between the preceding search request and the target search request. The electronic device can determine the relevance between the target page and the preceding page based on the relevance between the preceding search request and the target search request. If the electronic device determines that the preceding page is related to the target page, it can add information for accessing the preceding page to the bookmark information of the target page.

[0265] The electronic device can determine a relevance between a prior search request and a target search request.

[0266] An electronic device may determine a similarity score between search terms when each of the preceding search request and the target search request includes a search term. For example, the electronic device may determine the similarity score between the search requests based on words commonly included in the search terms. For example, the electronic device may determine a higher similarity score between the preceding search request and the target search request as the number of words commonly included in the preceding search request and the target search request increases. For example, the electronic device may determine the similarity score between the search terms using a search term similarity determination model. The search term similarity determination model may be implemented based on a machine learning model (e.g., a neural network). The search term similarity determination model may include a model generated and / or trained to output output data including a similarity score indicating the similarity between the search terms by being applied to input data based on the search terms.

[0267] An electronic device may determine a similarity score between search images when each of the preceding search request and the target search request includes a search image. For example, the electronic device may determine a similarity score between the search images using a search image similarity determination model. The search image similarity determination model may be implemented based on a machine learning model (e.g., a neural network). The search image similarity determination model may include a model generated and / or trained to output output data including a similarity score indicating similarity between the search images by being applied to input data based on the search images.

[0268] For example, referring to FIG. 12, an electronic device may display a first page (1210) providing a search service. The electronic device may obtain user input through the first page (1210). Based on the user input obtained from the first page (1210), the electronic device may generate a search request (1221) containing a search term.

[0269] The electronic device may display a second page (1220) containing search results for a search request (1221). The electronic device may obtain user input when displaying the second page (1220). The electronic device may generate a search request (1231) based on the user input obtained when displaying the second page (1220).

[0270] The electronic device may display a third page (1230) containing search results for a search request (1231). The electronic device may obtain user input when displaying the third page (1230). The electronic device may generate a search request (1241) based on the user input obtained when displaying the third page (1230).

[0271] The electronic device may display a fourth page (1240) including search results for the search request (1241). The electronic device may obtain a bookmark input (1242) for the fourth page (1240). Based on the fact that the fourth page (1240) includes search results for the search request (1241), the electronic device may determine a relevance between another page (e.g., the second page (1220), the third page (1230)) including search results and the fourth page (1240) using the relevance between the search requests. For example, the electronic device may determine a relevance between the second page (1220) and the fourth page (1240) based on the relevance between the search request (1221) and the search request (1241). For example, the electronic device may determine a relevance between the third page (1230) and the fourth page (1240) based on the relevance between the search request (1231) and the search request (1241).

[0272] FIG. 13 is a diagram illustrating an example of an operation of an electronic device providing an interface for comparing multiple pages according to various embodiments.

[0273] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may provide an interface for comparing a plurality of pages. At least one of the plurality of pages may be a bookmarked page.

[0274] Referring to FIG. 13, an electronic device may display a search page including a search screen on a screen (1310). A search page may refer to a page that provides a search service. According to one embodiment, the electronic device may determine a page as a search page by receiving information indicating that the page is a search page from a server that provides the search page. The electronic device may store a list of search pages classified and / or registered as search pages, and if the page is included in the search page list, the electronic device may determine the page as a search page.

[0275] An electronic device can obtain user input through a search page. For example, the user input may include text input obtained using an input module of the electronic device (e.g., the input module (150) of FIG. 1 , a keyboard, a mouse, or a display). For example, the user input may include the user's voice input. As described above in FIG. 6 , the user's voice input may be converted into text data by being analyzed using a natural language platform, and the user's intent may be determined based on the text data. For example, as shown in FIG. 13 , the electronic device may obtain the user's input "Galaxy Z Flip 5 specifications" on the screen (1310).

[0276] The electronic device may generate a search request based on the acquired user input "Galaxy Z Flip 5 specifications." The electronic device may display a first result page (result page_1) containing search results for the search request on the screen (1320). The result page may refer to a page containing search results for the search request. The electronic device may display at least a portion of the search page (search screen) on the screen (1320).

[0277] The electronic device can detect a user's input for a search page on the screen (1330). For example, the electronic device can detect a user's gesture input of pointing a portion of the search page displayed on the screen (1330) with a body part of the user (e.g., an eye or a finger). The electronic device can obtain a bookmark input for the first result page (result page_1) based on detecting the user's input for the search page when displaying the first result page (result page_1) including the search results. For example, when the electronic device detects an input requesting display of the search page again while displaying the search results, the electronic device can obtain a bookmark input for the page including the currently displayed search results (e.g., the first result page (result page_1)).

[0278] The electronic device may display a search page on the screen (1340). The electronic device may further display at least a portion of the first result page (result page_1) along with the search page. If the electronic device obtains a bookmark input for the first result page (result page_1) on the screen (1330), the electronic device may display at least a portion of the first result page (result page_1) as a bookmark object on the screen (1340).

[0279] The electronic device can obtain the user's input "Galaxy Z Flip 5 specifications details" on the screen (1340).

[0280] The electronic device may generate a search request based on the acquired user input of "Galaxy Z Flip 5 Specification Details." The electronic device may display a second result page (result page_2) containing search results for the search request on the screen (1350). The electronic device may display at least a portion of the search page (search screen) and at least a portion of the first result page (result page_1) on the screen (1350).

[0281] When the electronic device displays a second result page (result page_2) on the screen (1350), the electronic device can detect a user input for the first result page (result page_1). For example, the electronic device can detect a user gesture input of pointing a portion of the first result page (result page_1) displayed through the screen (1350) with a body part (e.g., an eye or a finger). When the electronic device displays a second result page (result page_2) including search results, the electronic device can provide a comparison interface for the first result page (result page_1) and the second result page (result page_2) based on detecting a user input for the first result page (result page_1) including other search results.

[0282] When the electronic device detects a user input for another search result while displaying the search result, the electronic device may provide a comparison interface of a page including the search result (e.g., a first result page (result page_1)) and a page including another search result (e.g., a second result page (result page_2)) on the screen (1360). The electronic device may display the first result page (result page_1) and the second result page (result page_2) side by side on the screen (1360).

[0283] FIGS. 14a, 14b, 14c, and 14d are diagrams illustrating examples of screen layouts for displaying a results page including search results according to various embodiments.

[0284] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may provide an interface that displays one or more pages (e.g., one or more result pages).

[0285] In FIG. 14a, the electronic device may display a first result page (result page_1) obtained from a search page on a screen (1410a). While displaying the first result page (result page_1), the electronic device may display at least a portion of the search page.

[0286] In FIG. 14b, the electronic device may display a first result page (result page_1) obtained from a search page on a screen (1410b). The first result page (result page_1) may be a bookmarked page. The bookmark information of the first result page (result page_1) may include information for accessing another page related to the first result page (result page_1) (e.g., a preceding page of the first result page (result page_1)). Based on the fact that the bookmark information of the first result page (result page_1) includes information about another page, the electronic device may display the window of the first result page (result page_1) in a stacked form with the windows of the other pages.

[0287] In FIG. 14c, the electronic device may display a first result page (result page_1) obtained from a search page and a second result page (result page_2) obtained from the search page. The electronic device may display a comparison interface of the first result page (result page_1) and the second result page (result page_2) on the screen (1410c) based on a user input. The electronic device may display the first result page (result page_1) and the second result page (result page_2) side by side.

[0288] In FIG. 14d, the electronic device can display a first result page (result page_1), a second result page (result page_2), and a third result page (result page_3) from the search page. The electronic device can display a comparison interface of the first result page (result page_1), the second result page (result page_2), and the third result page (result page_3) on the screen (1410d) based on a user's input. As shown in FIG. 14, the electronic device can display the search page together with the first result page (result page_1), the second result page (result page_2), and the third result page (result page_3).

[0289] FIG. 15 is a diagram illustrating an example of an operation of an electronic device according to various embodiments to provide a comparison interface between a plurality of target pages and a plurality of preceding pages.

[0290] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may provide a comparison interface between a plurality of bookmarked target pages.

[0291] An electronic device may sequentially store bookmark information for a first target page (target page_1) and a second target page (target page_2). The first bookmark information (also referred to as 'bookmark information') of the first target page (target page_1) may include information for accessing a first preceding page (preceding page_1) related to the first target page (target page_1). The second bookmark information (also referred to as 'other bookmark information') of the second target page (target page_2) may include information for accessing a second preceding page (preceding page_2) related to the second target page (target page_2).

[0292] For example, an electronic device may display a first preceding page (preceding page_1). After displaying the first preceding page (preceding page_1), the electronic device may display a first target page (target page_1). Based on obtaining a bookmark input for the first target page (target page_1), the electronic device may store first bookmark information including access information for the first target page (target page_1). Based on a relationship between the first target page (target page_1) and the first preceding page (preceding page_1), the electronic device may add information for accessing the first preceding page (preceding page_1) to the first bookmark information.

[0293] An electronic device may display a second preceding page (preceding page_2). After displaying the second preceding page (preceding page_2), the electronic device may display a second target page (target page_2). The electronic device may obtain a user's bookmark input for bookmarking the second target page (target page_2). Based on obtaining the bookmark input for the second target page (target page_2), the electronic device may store second bookmark information including information for accessing the second target page (target page_2). Based on a relationship between the second preceding page (preceding page_2) and the second target page (target page_2), the electronic device may add information for accessing the second preceding page (preceding page_2) to the second bookmark information. For example, the electronic device may store other bookmark information including information for accessing a second preceding page (preceding page_2) and information for accessing a second target page (target page_2) based on a relationship between the second preceding page (preceding page_2) and the second target page (target page_2).

[0294] The electronic device may display a comparison interface between the first target page (target page_1) and the second target page (target page_2) based on obtaining a user input for comparing the first target page (target page_1) with the second target page (target page_2). For example, as described above in FIG. 14b, the electronic device may display the first target page (target page_1) and the second target page (target page_2) side by side.

[0295] Referring to FIG. 15, an electronic device can bookmark a first target page (target page_1) together with a first preceding page (preceding page_1), and can bookmark a second target page (target page_2) together with a second preceding page (preceding page_2). The electronic device can obtain a user input requesting a comparison between the bookmarked first target page (target page_1) and the second target page (target page_2). The electronic device can display a comparison interface including the first target page (target page_1) and the second target page (target page_2) on a screen (1510). The electronic device can display the first target page (target page_1) through a first window corresponding to the first target page (target page_1) on the screen (1510), and can display the second target page (target page_2) through a second window corresponding to the second target page (target page_2).

[0296] For example, the electronic device may display a window corresponding to a first target page (target page_1) in a stacked form with multiple windows based on the fact that the bookmark information of the first target page (target page_1) includes information about the first preceding page (preceding page_1) on the screen (1510). The electronic device may display a window corresponding to a second target page (target page_2) in a stacked form with multiple windows based on the fact that the bookmark information of the second target page (target page_2) includes information about the second preceding page (preceding page_2).

[0297] The electronic device may obtain an additional user input (1511) requesting display of a preceding page on the screen (1510). For example, if the electronic device detects a user input pointing to a window corresponding to a second target page (target page_2), the electronic device may obtain an additional user input (1511) requesting display of a second preceding page (preceding page_2) related to the second target page (target page_2).

[0298] The electronic device may change the comparison interface to display at least one of a first preceding page (preceding page_1) or a second preceding page (preceding page_2) based on obtaining an additional input (1511) from a user requesting display of a preceding page on the screen (1520). The electronic device may display a second preceding page (preceding page_2) related to a second target page (target page_2) based on the additional input (1511) from the user regarding the second target page (target page_2) on the screen (1520).

[0299] The electronic device may obtain an additional user input (1521) requesting display of a preceding page on the screen (1520). For example, if the electronic device detects a user input pointing to a window corresponding to a first target page (target page_1), the electronic device may obtain an additional user input (1521) requesting display of a first preceding page (preceding page_1) related to the first target page (target page_1).

[0300] The electronic device may display, on the screen (1530), a first preceding page (preceding page_1) related to a first target page (target page_1) based on an additional input (1521) of the user regarding the first target page (target page_1). When the electronic device displays a comparison interface including the first preceding page (preceding page_1) and the second preceding page (preceding page_2) on the screen (1530), the electronic device may display the first target page (target page_1) and the second target page (target page_2) side by side, and display the first preceding page (preceding page_1) and the second preceding page (preceding page_2) side by side, thereby providing a comparison interface for easy comparison between target pages and between preceding pages.

[0301] In FIG. 15, the additional input from the user requesting display of the preceding page is primarily illustrated as specifying a specific target page, but is not limited thereto. For example, when displaying a comparison interface between a first target page (target page_1) and a second target page (target page_2), if the electronic device detects an additional input from the user requesting display of the preceding page for the comparison interface, the electronic device may display both the first preceding page (preceding page_1) and the second preceding page (preceding page_2).

[0302] FIG. 16 is a diagram illustrating various embodiments of an operation of an electronic device to obtain user input after bookmarking a target page.

[0303] An electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, electronic device (201) of FIG. 2, electronic device (301) of FIG. 3, electronic device (401) of FIG. 4, electronic device (501) of FIG. 5) may perform recommendations for user input for subsequent pages and / or interpretation of user input based on bookmark information.

[0304] A follow-up page may refer to a page displayed by an electronic device after the target page is bookmarked. For example, a follow-up page may refer to a page displayed later after the target page is displayed. For example, a follow-up page may refer to a page where a user input occurs after the target page is bookmarked.

[0305] An electronic device according to one embodiment may generate and / or provide recommended user input for subsequent pages based on bookmark information.

[0306] The electronic device can display a target page after displaying a preceding page, and bookmark the target page based on the bookmark input for the target page. The electronic device can store information about the preceding page related to the target page along with the bookmark information for the target page. After storing the bookmark information for the target page, the electronic device can display the subsequent page.

[0307] An electronic device may generate recommended user inputs for subsequent pages based on bookmark information. For example, the electronic device may generate recommended user inputs based on at least one of information about a preceding page or information about a target page. For example, information about a page (e.g., the preceding page, the target page) may include at least one of: information about a user input obtained through the page; information about a search request corresponding to a search result if the page includes search results; information about content included in the page; or information for accessing the page.

[0308] For example, the electronic device may select a page similar to a subsequent page from among the preceding page or the target page, and generate user input obtained through the selected page as recommended user input for the subsequent page.

[0309] For example, the preceding page and the target page may be pages provided by a server providing a shopping service. The preceding page may be a product detail page containing information about a product (e.g., spinach), and the subsequent page may be a shopping cart page corresponding to a shopping cart. A shopping cart is an example of a space (e.g., virtual space) provided by a shopping service, and may contain information about products registered by a user as items to be purchased. For example, a user may add a specific product to the shopping cart.

[0310] The subsequent page may be a page provided by a server providing the B shopping service. The subsequent page may be a page for searching for products in the B shopping service. Based on the preceding page being a product detail page containing information about spinach, the electronic device may generate recommended user input for entering the search term "spinach" on the subsequent page.

[0311] An electronic device may display a graphical object (e.g., a recommended user input object) for a recommended user input. The electronic device may obtain a user input specified by the recommended user input based on a simple user input to the recommended user input object (e.g., a user input pointing to the recommended user input object).

[0312] According to one embodiment, when a user searches for and compares identical or similar products from multiple shopping services, searches for products through shopping service A, registers them in a shopping cart, and bookmarks the shopping cart page, and then searches for products through shopping service B, the user can search for and / or register products in shopping service B in a simple operation using recommended user input.

[0313] An electronic device according to one embodiment can determine a user's intention corresponding to a user's input entered through a subsequent page based on bookmark information.

[0314] The electronic device can store bookmark information and then display subsequent pages. The electronic device can obtain user input (e.g., voice input) when displaying subsequent pages. The electronic device can determine the user's intent based on the bookmark information and the user's input.

[0315] For example, an electronic device may acquire a user's voice input when displaying a subsequent page. The electronic device may determine the user's intent from the user's voice input. As described above in FIG. 6, the electronic device may analyze the user's voice input using a natural language platform, and the natural language platform may include an automatic speech recognition module and / or a natural language understanding module. The electronic device may convert the user's voice input into text data through the automatic speech recognition module, and may determine the user's intent from the text data through the natural language understanding module. When determining the user's intent from the text data, the electronic device may further utilize bookmark information.

[0316] Referring to FIG. 16, as an example, an electronic device may display a first search page (search page_1) provided by A Shopping Service on a screen (1610). When displaying the first search page (search page_1) on the screen (1610), the electronic device may obtain a user's voice input of "lowest price portable fan." The electronic device may determine the user's intention corresponding to the user's voice input as "search for the lowest priced product among portable fans in A Shopping Service." The electronic device may generate a search request based on the determined user's intention. The electronic device may display a product detail page corresponding to the lowest priced product among portable fans searched for in A Shopping Service as a first result page (result page_1) on the screen (1620). The electronic device may bookmark the first result page (result page_1) on the screen (1620).

[0317] An electronic device may display a second search page (search page_2) provided by B Shopping Service and a third search page (search page_3) provided by C Shopping Service on a screen (1630). When displaying the second search page (search page_2) and the third search page (search page_3) on the screen (1630), the electronic device may obtain the user's voice input "Search again for items under 20,000 won". In determining the user's intention corresponding to the user's voice input, the electronic device may further utilize the bookmark information of the first result page (result page_1) together with the user's voice input "Search again for items under 20,000 won". Based on the user's voice input "Search again for items under 20,000 won" and the user's voice input "lowest price portable fan" included in the bookmark information of the first result page (result page_1), the electronic device may determine the user's intention as "Search for a portable fan priced under 20,000 won in each shopping service." The electronic device can generate a search request based on the determined user intent.

[0318] The electronic device may display, on screen (1640), a product detail page corresponding to a portable fan priced at 20,000 won, searched for in the B shopping service, as a second result page (result page_2). The electronic device may display, on screen (1640), a product detail page corresponding to a portable fan priced at 19,000 won, searched for in the C shopping service, as a third result page. The electronic device may provide, on screen (1640), a comparison interface between the first result page (result page_1), the second result page (result page_2), and the third result page (result page_3).

[0319] An electronic device according to one embodiment uses bookmark information to determine a user's intention corresponding to a user's input obtained through a subsequent page, so that when a user intends to perform a comparison between result pages by repeating a similar action, the electronic device can reach a result page (e.g., a second result page (result page_2), a third result page (result page_3)) with a simple input for the subsequent page.

[0320] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0321] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0322] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0323] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0324] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0325] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0326] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and software applications running on the operating system. Furthermore, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0327] Software may include computer programs, codes, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, or computer storage medium or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

[0328] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination, and the program commands recorded on the medium may be those specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0329] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

Claims

1. In an electronic device (101; 201; 301; 401; 501), display(160; 205; 210; 320; 421); At least one processor (120) comprising a processing circuit; and A memory (130) comprising one or more storage media storing instructions, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Display the preceding page, After displaying the preceding page, the target page is displayed. Obtain the user's bookmark input for bookmarking the above target page, Store bookmark information including information for accessing the target page in the memory (130), Based on the relevance between the preceding page and the target page, information for accessing the preceding page is added to the bookmark information. causing to do, Electronic devices (101; 201; 301; 401; 501).

2. In paragraph 1, The above electronic device (101; 201; 301; 401; 501) Further comprising a communication circuit, The above display (160; 205; 210; 320; 421) It includes a first display (205) that outputs a right image corresponding to the user's right eye and a second display (210) that outputs a left image corresponding to the user's left eye. When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Receive a display command for the preceding page and the target page from an external device (102; 104) that simultaneously displays the preceding page and the target page through individually corresponding windows, The preceding page and the target page are displayed through the display (160; 205; 210; 320; 421) through windows arranged within the virtual space (500), Based on obtaining the bookmark input for the target page, determining whether there is a preceding page that was displayed simultaneously with the target page, Based on the existence of the preceding page that was displayed simultaneously with the target page by the external device (102; 104), it is determined whether each of the preceding page and the target page includes a search result for a search request corresponding to the corresponding page. Adding information for accessing the preceding page to the bookmark information based on the preceding page and the target page including search results causing to do, Electronic devices (101; 201; 301; 401; 501).

3. In any one of paragraphs 1 and 2, The above electronic device (101; 201; 301; 401; 501) Further comprising a communication circuit, The above display (160; 205; 210; 320; 421) It includes a first display (205) that outputs a right image corresponding to the user's right eye and a second display (210) that outputs a left image corresponding to the user's left eye. When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: After displaying the preceding page, a display command for the target page is received from an external device (102; 104) that displays the target page, Display the target page through a window placed in the virtual space (500) based on the display command above, Based on the relationship between the preceding page and the target page, the preceding page is further displayed together with the target page through windows arranged within the virtual space (500). causing to do, Electronic devices (101; 201; 301; 401; 501).

4. In any one of paragraphs 1 to 3, The above electronic device (101; 201; 301; 401; 501) Further comprising a communication circuit, The above display (160; 205; 210; 320; 421) It includes a first display (205) that outputs a right image corresponding to the user's right eye and a second display (210) that outputs a left image corresponding to the user's left eye. When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Receive a display command for the target page from an external device (102; 104) that displays the target page, The target page is displayed through a window placed in the virtual space (500) based on the display command above, Receive information about the preceding page as a page related to the target page from the server providing the target page, Based on receiving information about the preceding page, the preceding page is further displayed together with the target page through windows arranged within the virtual space (500). causing to do, Electronic devices (101; 201; 301; 401; 501).

5. In any one of paragraphs 1 to 4, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Obtain user input from the preceding page, Performing a first operation among a plurality of operations constituting a function based on the user's input obtained from the preceding page; The target page includes content determined based on the result of performing the first operation, and the user's input obtained through the target page performs a second operation among the plurality of operations, so that the preceding page is determined to be related to the target page. causing to do, Electronic devices (101; 201; 301; 401; 501).

6. In any one of paragraphs 1 to 5, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Displaying the preceding page containing search results for the preceding search request generated based on the user's input; After displaying the preceding page, obtain other input from the user, Displaying the target page containing search results for the target search request generated based on the other input of the user; Based on the relevance between the preceding search request and the target search request, determine whether to add information for accessing the preceding page to the bookmark information. causing to do, Electronic devices (101; 201; 301; 401; 501).

7. In any one of paragraphs 1 to 6, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Based on storing the above bookmark information, display a bookmark object indicating that the target page is bookmarked, Based on obtaining the user's input for the above displayed bookmark object, displaying the preceding page and the target page. causing to do, Electronic devices (101; 201; 301; 401; 501).

8. In any one of paragraphs 1 to 7, The preceding page is Including the first preceding page, The above target page is, Includes the first target page, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Display the second preceding page, After displaying the second preceding page, display the second target page, Obtaining the user's bookmark input for bookmarking the second target page; Based on the relevance between the second preceding page and the second target page, store other bookmark information including information for accessing the second preceding page and information for accessing the second target page; Based on obtaining a user's input for comparing the first target page with the second target page, displaying a comparison interface between the first target page and the second target page, Based on obtaining additional input from the user requesting display of a preceding page, modifying the comparison interface to display at least one of the first preceding page or the second preceding page. causing to do, Electronic devices (101; 201; 301; 401; 501).

9. In any one of paragraphs 1 to 8, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: After saving the above bookmark information, display the subsequent page, Based on the above bookmark information, generate recommendation user input for the subsequent page. causing to do, Electronic devices (101; 201; 301; 401; 501).

10. In any one of paragraphs 1 to 9, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: After saving the above bookmark information, display the subsequent pages, Obtain the user's input when displaying the above subsequent pages, Based on the above bookmark information and the above user's input, determine the user's intention. causing to do, Electronic devices (101; 201; 301; 401; 501).

11. In any one of paragraphs 1 to 10, When the above instructions are executed by the at least one processor (120), the electronic device (101; 201; 301; 401; 501) causes: Displaying at least a portion of said preceding page together with said target page based on enabling display history view. causing to do, Electronic devices (101; 201; 301; 401; 501).

12. In a method performed by an electronic device, An action to display the preceding page via the display (160; 205; 210; 320; 421); An action of displaying the target page after displaying the preceding page; An action to obtain a user's bookmark input for bookmarking the above target page; An operation of storing bookmark information including information for accessing the target page in memory (130); and An action of adding information for accessing the preceding page to the bookmark information based on the relevance between the preceding page and the target page. A method including:

13. In paragraph 12, The action of displaying the above target page is: An operation of receiving a display command for the preceding page and the target page from an external device (102; 104) that simultaneously displays the preceding page and the target page through individually corresponding windows; and Including an action of displaying the preceding page and the target page through the display (160; 205; 210; 320; 421) through windows arranged in a virtual space (500), The action of adding information for accessing the preceding page to the above bookmark information is: An operation of determining whether there is a preceding page that was displayed simultaneously with the target page based on obtaining a bookmark input for the target page; An operation of determining whether each of the preceding page and the target page includes a search result for a search request corresponding to the corresponding page based on the existence of the preceding page that was displayed simultaneously with the target page by the external device (102; 104); and An operation including adding information for accessing the preceding page to the bookmark information based on the preceding page and the target page including search results. method.

14. In any one of paragraphs 12 to 13, The action of displaying the above target page is: An action of receiving a display command for the target page from an external device (102; 104) that displays the target page after displaying the preceding page; An operation of displaying the target page through a window placed in a virtual space (500) based on the display command; and An operation of further displaying the preceding page together with the target page through arranged windows within the virtual space (500) based on the relevance between the preceding page and the target page, method.

15. A computer-readable recording medium storing one or more computer programs including commands for performing the method of any one of claims 12 to 14.

Images