Electronic device, method, and non-transitory computer-readable storage medium for processing acquired image

Abstract

According to an embodiment, an electronic device may comprise: a display; a camera; a memory including instructions and including one or more storage media; and at least one processor including processing circuitry. The instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to: display a screen; receive an input for capturing the screen to store an image of the screen; recognize the image; acquire information about an address of a location represented by the image; store the information about the address; execute an application for the camera; display a preview image within an execution screen; and on the basis of identifying that the electronic device is positioned within a threshold distance from the address of the location, display information related to the image to be associated with a visual object corresponding to the location in the preview image.

Description

Electronic device, method, and non-transient computer-readable storage medium for processing acquired images

[0001] The following descriptions relate to an electronic device, a method, and a non-transitory computer-readable storage medium for processing an acquired image.

[0002] An electronic device can capture a screen and save an image. The electronic device can obtain an output photograph that captures exactly what is seen on the screen of a computer display or a mobile electronic device display. By capturing the screen, the electronic device can obtain an image file in a bitmap image format (e.g., BMP (bitmap image file), PNG (portable network graphics), GIF (graphics interchange format)). The electronic device can capture said portion of the screen based on receiving user input that specifies a portion of the screen.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] An electronic device is provided. The electronic device may include a display. The electronic device may include a camera. The electronic device may include at least one processor including a processing circuit. The electronic device may include a memory that stores instructions and includes one or more storage media. The instructions may cause the electronic device to display a screen through the display when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to receive an input for capturing the screen when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to store an image of the screen based on the input when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to recognize the image when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to obtain information regarding the address of said place based on identifying that the image represents a place according to said recognition, when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to store said information when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to display a preview image within the execution screen of said software application through the display, based on executing a software application for said camera after said information has been stored, when executed individually or collectively by the at least one processor.The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to identify whether the electronic device is located within a threshold distance from the address using the stored information while displaying the preview image. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to display information related to the image through the display, so as to be associated with a visual object within the preview image corresponding to the location, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image.

[0005] A method is provided by an electronic device having a display and a camera. The method may include an operation of displaying a screen through the display. It may include an operation of receiving an input for capturing the screen. The method may include an operation of storing an image of the screen based on the input. The method may include an operation of recognizing the image. The method may include an operation of obtaining information about the address of the location based on identifying that the image represents a location based on the recognition. The method may include an operation of storing the information. The method may include an operation of displaying a preview image within the execution screen of the software application through the display, based on executing a software application for the camera of the electronic device after the information is stored. The method may include an operation of identifying whether the electronic device is located within a threshold distance from the address using the stored information while displaying the preview image. The method may include an operation of displaying information related to the image through the display so as to be associated with a visual object within the preview image corresponding to the location, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image.

[0006] A non-transient computer-readable storage medium is provided for storing one or more programs. The one or more programs may include instructions that cause the electronic device to display a screen through the display when executed by at least one processor of an electronic device having a display and a camera. The one or more programs may include instructions that cause the electronic device to receive an input for capturing the screen when executed by the at least one processor. The one or more programs may include instructions that cause the electronic device to store an image of the screen based on the input when executed by the at least one processor. The one or more programs may include instructions that cause the electronic device to recognize the image when executed by the at least one processor. The one or more programs may include instructions that cause the electronic device to obtain information about the address of the place based on identifying that the image represents a place according to the recognition when executed by the at least one processor. The above one or more programs may include instructions that cause the electronic device to store the information when executed by the at least one processor. The above one or more programs may include instructions that cause the electronic device to display a preview image within the execution screen of the software application through the display, based on executing a software application for the camera after the information has been stored when executed by the at least one processor.The above one or more programs may include instructions that cause the electronic device to identify, using the stored information, whether the electronic device is located within a threshold distance from the address while displaying the preview image when executed by the at least one processor. The above one or more programs may include instructions that cause the electronic device to display information related to the image through the display, so as to be associated with a visual object within the preview image corresponding to the location, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image when executed by the at least one processor.

[0007] Figure 1 illustrates a block diagram of an electronic device in a network environment.

[0008] Figure 2 illustrates a method of displaying information related to an image within a preview screen.

[0009] Figure 3 illustrates a simplified block diagram of an electronic device.

[0010] FIG. 4 is a flowchart illustrating the operation of displaying image-related information in a camera preview according to information about the address of a place.

[0011] FIG. 5a is a flowchart illustrating the operation of identifying text within an image to obtain information about an address.

[0012] FIG. 5b illustrates a method for obtaining information about an address by identifying text within an image.

[0013] FIG. 6a is a flowchart illustrating the operation of identifying text and / or graphic objects within an image to obtain information about an address.

[0014] FIG. 6b illustrates a method for obtaining information about an address by identifying text and / or graphic objects within an image.

[0015] FIG. 7 is a flowchart illustrating the operation of obtaining information about an address using keywords generated using text and / or graphic objects.

[0016] Figure 8 illustrates a method for storing information about an address.

[0017] FIG. 9 illustrates a method for obtaining information about the address of a place represented by a second image using information from a first image.

[0018] FIG. 10a is a flowchart illustrating the operation of enabling the display of an augmented reality (AR) graphic object.

[0019] FIG. 10b illustrates a method for enabling the display of AR graphic objects.

[0020] Figure 11 illustrates information related to a visually emphasized image.

[0021] FIG. 12a is a flowchart illustrating the operation of displaying an object to guide a path from the location of an electronic device to an address.

[0022] FIG. 12b illustrates a method of displaying an object to guide a path from the location of an electronic device to an address.

[0023] FIG. 13a is a flowchart illustrating the operation of displaying an image of the exterior of a place as a superposition on a visual object.

[0024] FIG. 13b illustrates a method of displaying an image of the appearance of a place by superimposing it onto a visual object.

[0025] FIG. 14a is a flowchart illustrating an operation of displaying information related to an image based on input for a visual indication.

[0026] FIG. 14b illustrates a method of displaying information related to an image based on input for a visual indication.

[0027] FIG. 15 illustrates a method for displaying multiple contents included in image-related information within a user interface (UI) in a camera preview.

[0028] FIG. 16 illustrates a method of displaying information related to an image on a map application execution screen.

[0029] FIG. 17 illustrates a method of displaying an execution screen including a map screen and another execution screen including additional information about an image as a split view.

[0030] FIG. 18 illustrates a method of displaying information related to an image on the screen of a display of an XR (extended reality) device.

[0031] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0032] Figure 1 is a block diagram of an electronic device in a network environment.

[0033] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0034] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0035] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0036] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0037] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0038] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0039] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0040] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0041] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0042] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0043] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0044] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0045] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0046] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0047] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0048] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0049] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0050] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0051] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0052] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0053] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0054] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0055] An electronic device (e.g., the electronic device (200) of FIG. 3) can display a screen. The electronic device can capture the screen to obtain an image. The electronic device can analyze the image to obtain information about the address of a place. Based on the obtained information, the electronic device can identify that a visual object corresponding to the place is displayed. Based on the identification, the electronic device can display information related to the image to be associated with the visual object. These operations are described in more detail with reference to FIGS. 2 through 18.

[0056] Figure 2 illustrates a method of displaying information related to an image within a preview screen.

[0057] Referring to FIG. 2, at least one processor of an electronic device (200) (e.g., processor (210) of FIG. 3) can capture and store an image (201) on a screen, recognize the image (201), and obtain information about the address of the place represented by the image (201). After obtaining information about the address, at least one processor can display an execution screen (202) of a software application for a camera (e.g., camera (220) of FIG. 3). At least one processor can identify whether the electronic device (200) is located within a threshold distance from the address of the place. Based on identifying that the electronic device (200) is located within a threshold distance from the address of the place, at least one processor can display information (204) related to the image (201) through a display (e.g., display (230) of FIG. 3) so as to be associated with a visual object (203) corresponding to the place displayed on the execution screen (202). For example, at least one processor can display information (204) related to the image (201) by displaying a thumbnail image of the image. For example, at least one processor can display information (204) related to the image (201) in association with the visual object (203) by placing the information (204) related to the image (201) next to a visual object (203) corresponding to the location.

[0058] For example, at least one processor may prioritize displaying content with a higher priority (e.g., content set as a favorite by the user) among multiple contents included in information related to an image. As an example, at least one processor may display a graphic object (205) indicating that the content has a higher priority over the content with a graphic object (205) indicating that the content has a higher priority.

[0059] For example, at least one processor may display a scroll (206) within the area of ​​a display (e.g., the display (230) of FIG. 3) where the image-related information (204) is displayed, when some content included in the image-related information is not displayed due to spatial constraints of the screen. For example, at least one processor may display, through the display (230), at least one content that is distinguished from the content among a plurality of contents included in the image-related information (204), based on receiving input for the scroll (206).

[0060] Figure 3 illustrates a simplified block diagram of an electronic device.

[0061] Referring to FIG. 3, the electronic device (200) may include at least some or all of the components of the electronic device (101) of FIG. 1. For example, the electronic device (200) may correspond to the electronic device (101) of FIG. 1. For example, the electronic device (200) may include at least one of a processor (210), a camera (220), a display (230), a memory (240), and / or a communication circuit (250).

[0062] For example, at least one processor (210) may be operatively coupled with or connected with a camera (220), a display (230), a memory (240), and / or a communication circuit (250). That at least one processor (210) is operatively coupled with or connected with a camera (220), a display (230), a memory (240), and / or a communication circuit (250) may mean that at least one processor (210) can control the camera (220), the display (230), the memory (240), and / or the communication circuit (250). For example, the camera (220), the display (230), the memory (240), and / or the communication circuit (250) may be controlled by the processor (210).

[0063] For example, at least one processor (210) may include at least a part of the processor (120) of FIG. 1 or correspond to at least a part of the processor (120). For example, at least one processor (210) may include one or more processors including an application processor (AP) and / or a communication processor (CP). For example, at least one processor (210) may be implemented as a single chip, such as a system on chip (SoC), or as multiple chips. For example, at least one processor (210) may be implemented as a single integrated circuit. For example, at least one processor (210) may include a first integrated circuit and a second integrated circuit. For example, at least one processor (210) may be distributed within an electronic device (200). For example, a part of at least one processor (210) may be physically separated from another part of at least one processor (210). As a non-limiting example, the other part of at least one processor (210) is spaced apart from the part of at least one processor (210), but the other part of at least one processor (210) may be connected to the part of at least one processor (210) through an interface circuit.

[0064] The display (230) may include at least a part of the display module (160) of FIG. 1 or correspond to at least a part of the display module (160). The display (230) of the electronic device (200) may output visualized information (e.g., a screen) to a user. For example, the display (230) may be controlled by a controller, such as a GPU (graphic processing unit), to output visualized information to a user. The display (230) may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LEDs may include organic LEDs (OLEDs). The display (230) may have at least a partially curved shape or a deformable shape. A display (230) having a deformable shape may be referred to as a flexible display.

[0065] The memory (240) of the electronic device (200) may include a circuit and / or a storage medium for storing data and / or instructions input and / or output by at least one processor (210). The memory (240) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). Non-volatile memory may be referred to as storage. Volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). Non-volatile memory may include, for example, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disk, solid state drive (SSD), and embedded multi-media card (eMMC).

[0066] The memory (240) may include at least a portion of the memory (130) of FIG. 1 or correspond to at least a portion of the memory (130) of FIG. 1. For example, the memory (240) may be implemented as a single chip or as a plurality of chips. For example, the memory (240) may be implemented as a single integrated circuit or as a plurality of integrated circuits. For example, the memory (240) may be distributedly arranged within the electronic device (200).

[0067] The camera (220) may include at least a part of the camera module (180) of FIG. 1 or correspond to at least a part of the camera module (180) of FIG. 1. For example, the camera (220) may be placed on the rear of the electronic device to recognize an external object different from the electronic device (200). For example, using the camera (220), the electronic device (200) may acquire an image and / or video.

[0068] The communication circuit (250) may include at least a part of the communication module (190) of FIG. 1 or correspond to at least a part of the communication module (190) of FIG. 1. For example, the communication circuit (250) of the electronic device (200) may include hardware components to support the transmission and / or reception of electrical signals between the electronic device (200) and an external electronic device. The communication circuit (250) may include, for example, at least one of a modem, an antenna, and an optic / electronic converter. The communication circuit (250) may support the transmission and / or reception of electrical signals based on various types of protocols such as Ethernet, LAN (local area network), WAN (wide area network), WiFi (wireless fidelity), Bluetooth, BLE (bluetooth low energy), ZigBee, LTE (long term evolution), 5G NR (new radio), and / or 6G. At least one processor (210) of the electronic device (200) can execute instructions in memory (240) within the electronic device (200) to perform functions and / or operations indicated by said instructions. For example, if the electronic device (200) includes at least one processor (210), said at least one processor (210) may be configured to execute said instructions collectively or individually.

[0069] FIG. 4 is a flowchart illustrating an operation of displaying image-related information in a camera preview according to information about the address of a place. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0070] Referring to FIG. 4, in operation 401, at least one processor (210) may display a screen. For example, the screen may include a representation of a webpage (e.g., a blog). For example, the screen may be an execution screen of an application (e.g., a search engine-based application, a map application, a blog application). For example, the screen may include text objects and / or graphic objects. For example, the text object may include information about an address or a uniform resource identifier (URI). The URI may include a uniform resource locator (URL). For example, the graphic object may include an image representing the appearance of a place or an image of food. For example, the graphic object may represent text. For example, the graphic object may correspond to a signboard and / or placard containing a business name.

[0071] In operation 402, at least one processor (210) may receive input for capturing the screen. For example, at least one processor (210) may receive input for capturing the screen resulting from the pressing of a hardware button while the screen is being displayed. For example, at least one processor (210) may receive touch input for capturing the screen while the screen is being displayed. For example, at least one processor (210) may receive input for capturing the screen by identifying a user's gesture input. Identification of the gesture input may be performed via a camera (220). Identification of the gesture input may also be performed via a display (230) (e.g., a touch-sensitive display).

[0072] In operation 403, at least one processor (210) may store an image of the screen based on receiving an input for capturing the screen. For example, the image of the screen may contain information about the entire area of ​​the screen. For example, the image of the screen may contain information about a partial area of ​​the screen. For example, a partial area of ​​the screen may be described as the remaining area excluding an indicator area of ​​the screen (e.g., containing time information and / or battery level information).

[0073] For example, an image of the screen may be stored in the memory (240) of the electronic device (200). At least one processor (210) may store meta-information of the image of the screen. For example, the meta-information may be described as information stored in an image description. For example, the meta-information may include information about the time at which the image of the screen was captured. For example, the meta-information may be stored to be associated with the image of the screen.

[0074] For example, at least one processor (210) may receive information about an image stored in an electronic device distinct from the electronic device (200) through a communication circuit (250). Based on the reception, at least one processor (210) may store information about the image in memory (240). For example, the electronic device distinct from the electronic device (200) may be described as a personal computer (PC), a mobile device, or an external server. For example, at least one processor (210) may receive information about the image through the communication circuit (250) based on the transmission of information about the image from a mobile device distinct from the electronic device (200). At least one processor (210) may store information about the image in memory (240). In operation 404, at least one processor (210) may recognize the stored image. For example, at least one processor (210) can analyze an image through at least one of a text analysis model, an image analysis model, and / or a generative AI model. The text analysis model, the image analysis model, and / or generative AI model may or may not be an on-device model implemented on an electronic device.

[0075] For example, a text analysis model may include at least one of a logistic regression model, a support vector machine (SVM) model, a recurrent neural network (RNN) model, a long short-term memory (LSTM) network model, a gated recurrent unit (GRU) model, and / or a convolutional neural network (CNN) model. For example, an image analysis model may include at least one of a CNN model, a YOLO (you only look once) model, and / or an R-CNN (regions with convolutional neural network) model.

[0076] For example, a generative AI model may include a generative AI model comprising multiple parameters related to a neural network having a structure based on an encoder and a decoder, such as a transformer. As an example, a generative AI model may include a bidirectional model based on learning about an encoder (e.g., BERT (bidirectional encoder representations from transformers)), or an auto-encoding model (e.g., a diffusion model). As an example, a generative AI model may include an auto-regressor model based on learning about a decoder (e.g., GPT (generative pre-trained transformer)). As an example, a generative model may include a sequence-to-sequence model based on learning about an encoder and a decoder (e.g., stable diffusion, DALL-E 2). For example, a generative AI model may include a large language model (LLM) for processing natural language based on massive parameters, but is not limited thereto. For example, a generative AI model may include parameters for driving neural networks such as a convolutional neural network (CNN), a recurrent neural network (RNN), a feedforward neural network (FNN), and / or a long short-term memory (LSTM).

[0077] For example, at least one processor (210) can identify the type of object within a stored image. For example, at least one processor (210) can identify whether an object within a stored image is a graphic object. Based on identifying that an object within a stored image is a graphic object, at least one processor (210) can identify an external object corresponding to said graphic object.

[0078] In operation 405, at least one processor (210) can identify that the stored image represents a place. For example, based on the recognition according to operation 404, it can identify that the stored image represents a place.

[0079] For example, at least one processor (210) can identify that the image represents a place which is an external object. For example, the place may include at least one of a building, a structure, and / or a space. For example, the place may include at least one of a restaurant, a cafe, a workplace, a factory, a school, an art gallery, a museum, and / or an office. For example, the image may represent the place by including at least one of a visual object representing the appearance of the place, a visual object representing an external object that may be found near the place, and / or a visual object representing an external object that may be found inside the place.

[0080] In operation 406, at least one processor (210) can obtain information about the address of a place represented by an image and can store said obtained information. For example, at least one processor (210) can obtain information about the address of a place represented by an image according to the identification performed in operation 405. At least one processor (210) can store information about the obtained address of a place.

[0081] For example, information regarding the address may include at least one of longitude / latitude regarding the place and / or an address indicating the place (e.g., city name, state code, zip code, street name, floor information or room number information).

[0082] For example, at least one processor (210) may store information about an address as meta-information of an image. For example, information about an address may be stored so as to be associated with an image on a screen. For example, at least one processor (210) may store information about an address so as to be associated with an image on a screen by storing the information about the address in an image description included in the meta-information of the image. An example of storing information about an address in the image description will be described later with reference to FIG. 8.

[0083] In operation 407, at least one processor (210) may execute a software application for the camera (220) after operations 401 through 406 have been performed. For example, the software application for the camera (220) may be executed based on input to an icon (or UI object) (or executable object) of the software application for the camera (220). For example, the software application for the camera (220) may be executed based on input to a physical button included in the electronic device (200). For example, the software application for the camera (220) may be executed based on an event identified according to a routine (or shortcut) set within the electronic device. For example, the software application for the camera (220) may be executed based on a user situation identified through a trained model on the electronic device. However, various embodiments are not limited thereto.

[0084] In operation 408, at least one processor (210) may display, through a display (230), an execution screen of a software application for a camera (220) that includes a preview image, according to the execution performed in operation 407. For example, the preview image may include a screen showing an external scene. For example, at least one processor (210) may display an executable object that changes shooting conditions (e.g., a zoom button, a front camera / rear camera switching button, a brightness adjustment scroll) through the display (230) together with the preview image. For example, the executable object may be displayed superimposed on the preview image. For example, when the executable object is displayed superimposed on the preview image, at least one processor (210) may display the executable object transparently.

[0085] In operation 409, at least one processor (210) can identify whether the electronic device (200) is located within a threshold distance from said address by using information about an address stored in memory (240). For example, the threshold distance may be a parameter used to display information related to an image of a screen to be described through operation 410 so as to overlap on a preview image.

[0086] According to one embodiment, the threshold distance can be adjusted. For example, the threshold distance can be adjusted according to user input. For example, the threshold distance can be adjusted according to input received while a preview image is displayed and for adjusting the magnification of the camera (220). For example, when the magnification of the camera (220) is a first magnification, the threshold distance may correspond to the first distance. For example, when the magnification of the camera (220) is a second magnification higher than the first magnification, the threshold distance may correspond to a second distance shorter than the first distance. For example, when the magnification of the camera (220) is a third magnification lower than the first magnification, the threshold distance may correspond to a third distance longer than the first distance.

[0087] For example, the threshold distance may be described as a distance at which a visual object corresponding to a location can be included within the preview image. For example, the threshold distance may be described as a distance for enabling identification of a visual object corresponding to a location within the preview image. For example, at least one processor (210) may identify whether a visual object corresponding to a location exists within the preview image based on identifying that an electronic device is located within a threshold distance from an address.

[0088] For example, at least one processor (210) can identify which direction the address is located from the location of the electronic device (200) and the distance between the address and the electronic device (200), based on identifying that the electronic device (200) is located within a critical distance from the address. For example, at least one processor (210) can display information indicating which direction the address is located from the location of the electronic device (200) through a display (230). For example, the information indicating which direction the address is located from the location of the electronic device (200) may be a graphic object (e.g., a graphic object having an arrow shape).

[0089] In operation 410, at least one processor (210) may display information related to the image through the display (230) so as to be associated with a visual object within the preview image corresponding to the location, based on identifying that the electronic device (200) is located within a threshold distance from the address while displaying the preview image. For example, at least one processor (210) may display information related to the image superimposed on the visual object. For example, at least one processor (210) may adjust the opacity of information related to the image displayed on the preview image when information related to the image is superimposed on the visual object. For example, at least one processor (210) may adjust the opacity based on receiving user input to adjust the opacity.

[0090] According to one embodiment, at least one processor (210) can visually highlight information related to an image within a preview image. For example, information related to an image can be visually highlighted within a preview image. Information related to an image that is visually highlighted will be described later with reference to FIG. 11.

[0091] According to one embodiment, at least one processor (210) may display an object through a display (230) to guide a path from a location of an electronic device (200) to an address. For example, an object to guide a path from a location of an electronic device (200) to an address may be displayed through the display (230). With reference to FIGS. 12a and 12b, a method for displaying an object to guide a path from a location of an electronic device to an address will be described later.

[0092] According to one embodiment, at least one processor (210) may display an image of the appearance of a place superimposed on a visual object within a preview image. For example, an image of the appearance of a place may be displayed superimposed on a visual object within a preview image. With reference to FIGS. 13a and 13b, a method of displaying an image of the appearance of a place superimposed on a visual object will be described later.

[0093] According to one embodiment, at least one processor (210) may display a visual indication associated with a visual object within a preview image. Based on receiving an input for the visual indication, at least one processor (210) may display information related to the image through a display (230) associated with the visual object. Referring to FIGS. 14a and 14b, a method for displaying information related to the image based on an input for the visual indication will be described later.

[0094] According to one embodiment, at least one processor (210) can display a plurality of contents included in information related to an image within a UI within a camera preview. With reference to FIG. 15, a method for displaying a plurality of contents included in information related to an image within a UI within a camera preview will be described later.

[0095] According to one embodiment, as operations according to operations 401 to 410 are performed, at least one processor (210) can display information related to an image representing a place of interest to a user of the electronic device (200) through a display (230) when executing a software application that utilizes information about an address. For example, the user can search for places of interest and capture a screen representing the places of interest. The user can access a website regarding the places of interest. The user can capture a screen including a representation of the website. For example, at least one processor (210) can save an image of the screen. For example, when a visual object corresponding to the places is displayed through the display (230) in the execution screen of a software application that utilizes information about an address, at least one processor (210) can display information related to the image in association with the visual object without user input. According to the example described above, at least one processor (210) can provide the user with a screen capture history related to the location by displaying information related to the image so as to be associated with the visual object.

[0096] In operation 402, an example is described in which at least one processor (210) stores an image representing a place so as to be associated with information about an address, but the embodiments of the present disclosure are not limited thereto. For example, according to operation 403, at least one processor (210) may store a portion of an image representing a place so as to be associated with information about an address. For example, the portion of the image may be described as a text object and / or graphic object included within the image. For example, the portion of the image may be described as a portion of a text object and / or graphic object included within the image.

[0097] For example, at least one processor (210) may store the image and store a portion of the image so as to be associated with the image. For example, at least one processor (210) may store the image and additionally store coordinate information of the portion of the image. For example, the coordinate information of the portion of the image may be described as the coordinates of the top-left corner of the portion of the image and the coordinates of the bottom-right corner of the portion of the image. After storing the portion of the image, at least one processor (210) may identify the stored portion of the image so as to be associated with information about an address according to operation 410. At least one processor (210) may display the portion of the image through a display (230) so as to be associated with a visual object. The display operation may correspond to operation 410.

[0098] FIG. 5a is a flowchart illustrating the operation of identifying text within an image to obtain information about an address.

[0099] FIG. 5b illustrates a method for obtaining information about an address by identifying text within an image.

[0100] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0101] Referring to FIG. 5a, in operation 511, at least one processor (210) can display a screen through a display (230). At least one processor (210) can identify text objects contained within an image based on recognizing an image on the screen. For example, text objects may include text and / or text contained in a graphic object. For example, at least one processor (210) can identify text objects through optical character recognition (OCR). For example, at least one processor (210) can identify text objects through at least one of a text analysis model, an image analysis model, and / or a generative AI model.

[0102] In operation 512, at least one processor (210) can identify that a text object represents a place. For example, a text object may represent a place by including information about the address of the place. For example, a part of the text object may be information about the address of the place.

[0103] In operation 513, at least one processor (210) may identify that an image represents the place based on identifying that a text object represents the place. For example, according to operation 512, the image may represent the place by a text object within the image representing the place.

[0104] In operation 514, at least one processor (210) can obtain information about the address of a place. For example, at least one processor (210) can obtain information about the address of the place from a text object. For example, information about the address can be stored in an image description. For example, information about the address can be stored in a tag format. Examples of information about the address being stored in an image description and / or information about the address being stored in a tag format will be described later with reference to FIG. 8.

[0105] Referring to FIG. 5b, at least one processor (210) can display a screen containing a representation of a webpage through a display (230) according to operation 401 of FIG. 4. At least one processor (210) can store an image (520) of the screen by receiving an input for capturing the screen.

[0106] For example, an image (520) may include a plurality of text objects (521). For example, the plurality of text objects (521) may include a text object (521-1) and a text object (521-2). At least one processor (210) may identify the text object (521-1) and the text object (521-2) contained within the image (520) based on recognizing the image (520) on the screen. For example, the text object (521-1) may indicate 'China Restaurant'. The text object (521-2) may indicate '123 Main Street Apartment 4B New York, NY 10001 USA'.

[0107] For example, at least one processor (210) can identify that an image (520) represents a place called 'China Restaurant' based on identifying that multiple text objects (521) represent a place. At least one processor (210) can obtain the address of the place represented by the image (520) (e.g., '123 Main Street Apartment 4B New York, NY 10001 USA').

[0108] As described above, according to the operation described in FIG. 5a and the method described in FIG. 5b, at least one processor (210) can store the image based on identifying that the screen contains information about the address of a text object, and can store information about the address in the image description. When storing the image of the captured screen, at least one processor (210) can store information about the address of the place represented by the image together.

[0109] FIG. 6a is a flowchart illustrating the operation of identifying text and / or graphic objects within an image to obtain information about an address.

[0110] FIG. 6b illustrates a method for obtaining information about an address by identifying text and / or graphic objects within an image.

[0111] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0112] Referring to FIG. 6a, in operation 611, at least one processor (210) can identify text objects and / or graphic objects within an image based on recognizing an image according to operation 404 of FIG. 4. For example, at least one processor (210) can identify text objects through optical character recognition (OCR). For example, at least one processor (210) can identify graphic objects through at least one of an image analysis model and / or a generative AI model. For example, at least one processor (210) can identify external objects corresponding to graphic objects within an image. For example, the graphic objects within the image may correspond to the exterior of a building, a signboard containing a business name, a placard, and / or a menu board.

[0113] In operation 612, at least one processor (210) can navigate a webpage using text objects and / or graphic objects. For example, the text objects and / or graphic objects may contain information representing a uniform resource identifier (URI). At least one processor (210) can navigate a webpage based on the information representing the URI. For example, the text objects and / or graphic objects within an image may contain a hyperlink. At least one processor (210) can navigate another webpage connected based on input for the hyperlink.

[0114] In operation 613, at least one processor (210) can obtain information about the address of a place from a searched webpage. For example, at least one processor (210) can identify text objects and / or graphic objects within the searched webpage. If the text objects and / or graphic objects contain information about the address of a place, at least one processor (210) can obtain information about the address. For example, at least one processor (210) can identify multiple webpages. At least one processor (210) can identify all text objects and / or graphic objects within the multiple searched webpages and obtain information about the address. For example, information about the address can be stored in an image description. For example, information about the address can be stored in a tag format. Examples of information about the address being stored in an image description and / or information about the address being stored in a tag format will be described later with reference to FIG. 8.

[0115] Referring to FIG. 6b, at least one processor (210) can recognize an image (620) that has been captured and saved. At least one processor (210) can identify text objects (621-1) and text objects (621-2). For example, text object (621-1) can indicate 'm.place.samsung.com / ChinaRestaurant'. For example, text object (621-2) can indicate 'https: / www.blog.samsung.com / ChinaRestaurant'. At least one processor (210) can identify a menu containing a website address, which is a graphic object (622). At least one processor (210) can navigate a webpage (623) using the text object (621) and / or the graphic object (622). For example, the webpage (623) may include a map webpage (623-1), a blog webpage (623-2), a webpage capable of displaying videos, or a social network service webpage. For example, if there are two or more explored webpages (623), at least one processor (210) can obtain information (624) about the correct address by identifying whether information (624-1) about the address within the webpage (623-1) and information (624-2) about the address within the other webpage (623-2) correspond.

[0116] As described above, according to the operation described in FIG. 6a and the method described in FIG. 6b, at least one processor (210) can obtain information about an address from a webpage explored using text and / or graphic objects within the image. At least one processor (210) can store the image and store information about the address in an image description. If the information about the address cannot be obtained using only text objects within the image, at least one processor (210) can obtain information about the address using graphic objects within the image.

[0117] FIG. 7 is a flowchart illustrating an operation to obtain information about an address using keywords generated using text and / or graphic objects. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0118] Referring to FIG. 7, in operation 701, at least one processor (210) may generate keywords using text objects and / or graphic objects based on identifying text objects and / or graphic objects within an image. For example, some or all of the text objects may be keywords. For example, some or all of the graphic objects may be keywords. For example, when the graphic object is a photograph of food, the photograph of food and / or the name of the food may be keywords.

[0119] For example, at least one processor (210) can generate keywords through an AI model trained to generate keywords using text objects and / or graphic objects. For example, the AI ​​model may include at least one of a text analysis AI model, an image analysis AI model, and a generative AI model. The text analysis AI model, the image analysis AI model, and / or the generative AI model may or may not be on-device models implemented on an electronic device.

[0120] For example, two or more keywords may be generated. For example, the keywords may be generated by combining a text object and a graphic object. For example, at least one processor (210) may generate keywords by combining information within the text object and information within the graphic object. For example, the text object may indicate 'China Restaurant'. For example, the graphic object may indicate 'New York'. For example, at least one processor (210) may generate keywords such as 'China Restaurant in New York' by combining the text object and the graphic object.

[0121] In operation 702, at least one processor (210) can obtain information about the address of a place by using an external search engine based on the generated keyword. For example, the external search engine may be described as a search engine for web pages, a search engine utilizing generative artificial intelligence, or a search engine for an application running on a mobile electronic device (200). For example, at least one processor (210) can perform a search process through an external search engine based on the generated keyword. For example, at least one processor (210) can obtain the information about the address of a place based on identifying that the search screen after the keyword is searched contains information about the address of a place.

[0122] According to one embodiment, at least one processor (210) can identify that the search screen does not contain information about the address of a place. Based on the identification, at least one processor (210) can identify text objects and / or graphic objects included in the search screen according to the method described with reference to FIG. 6a and FIG. 6b. At least one processor (210) can obtain information about the address of a place from a webpage searched using the text objects and / or graphic objects.

[0123] According to one embodiment, the acquired information regarding the address may be stored in an image description. For example, the information regarding the address may be stored in a tag format. Examples of the information regarding the address being stored in an image description and / or examples of the information regarding the address being stored in a tag format will be described later with reference to FIG. 8.

[0124] According to one embodiment, as operations according to operations 701 and 702 are performed, at least one processor (210) can obtain information about an address using an external search engine based on keywords generated from text and / or graphic objects within the image. At least one processor (210) can store the image and store information about the address in the image description. At least one processor (210) can store an image of a captured screen and store information about the address of the place represented by the image together. If at least one processor (210) cannot obtain information about the address using text objects and / or graphic objects within the image, the processor (210) can generate keywords from the text objects and / or graphic objects and obtain information about the address using an external search engine based on the keywords.

[0125] Figure 8 illustrates a method for storing information about an address.

[0126] Referring to FIG. 8, the screen (800) may include an image (810) and an image description (820). At least one processor (210) may display the image description (820), which is stored to be associated with the image (810), within the screen (800). For example, at least one processor (210) may run a gallery application. At least one processor (210) may display the image (810) and the image description (820) through a display (230) while displaying the execution screen of the gallery application.

[0127] For example, information stored in an image description may correspond to at least a portion of the image's metadata. For example, the image description (820) may include information about the address of a place (821), information about the place represented by the image (810) (822), and / or information in the form of a tag about the address of the place (823).

[0128] For example, information (821) about the address of a place represented by an image (810) may include at least one of longitude / latitude and / or an address indicating said place (e.g., city name, state code, zip code, street name, floor information or room number information). For example, information (822) about the place represented by an image (810) may include at least one of a business name, text included on a signboard, reviews regarding said place, goods or food sold at said place, and / or services provided at said place. For example, information (823) in the form of a tag about the address of a place may be described as information with the text "#Location : " added before the information about the address.

[0129] FIG. 9 illustrates a method for obtaining information about the address of a place represented by a second image using information from a first image.

[0130] Referring to FIG. 9, at least one processor (210) can store a first image (910) obtained by capturing a first screen and can store the time of capture (911) in an image description (930). At least one processor (210) can obtain information (913) about the address of the place represented by the first image (910) and store the information about the address (913) in the image description (930) of the first image (910). At least one processor (210) can receive an input for capturing a second screen after storing the information about the address (913). At least one processor (210) can store a second image (920) of the second screen based on receiving the input and can store the time of capture (921) in the image description (940) of the second image (920). At least one processor (210) can recognize the second image (920). Based on the recognition of the second image (920), at least one processor (210) can identify that the second image (920) represents a place and that the second image (920) does not include a part representing an address. Based on the identification, at least one processor (210) can identify the information regarding the address of the place represented by the second image (920) as the information regarding the address of the first image (910) by comparing the information of the first image (910) and the information of the second image (920).

[0131] According to one embodiment, information of an image (e.g., information of the first image (910) or information of the second image (920)) may include the time at which the image was captured, objects included in the image, and / or text obtained from graphic objects within the image. For example, at least one processor (210) may identify whether the time at which the second image (920) was captured (921) is within a reference time range from the time at which the first image (910) was captured (911). For example, the time at which the first image (910) was captured (911) may be later than the time at which the second image (920) was captured (921). For example, at least one processor (210) may identify the address of the place represented by the second image (920) as the address of the place represented by the first image (910) (913), based on identifying that the time (921) at which the second image (920) was captured is within a reference time range from the time (911) at which the first image (910) was captured. For example, the reference time may be set by a user. The reference time may be set in advance.

[0132] For example, at least one processor (210) may identify the address of a place represented by the second image (920) as the address (913) of a place represented by the first image (910), based on identifying the common parts (912 and 922) of the first image (910) and the second image (920). For example, the common parts (912 and 922) may be the exterior of a building, a signboard, food, the internal structure of a building, the internal structure of a building, and / or an array of objects placed inside a building.

[0133] For example, at least one processor (210) can compare text obtained from a graphic object (912) in the first image (910) (e.g., a business name included in a signboard, a company name included in a placard) and text obtained from a graphic object (922) in the second image (920). At least one processor (210) can identify that the first image (910) represents a place represented by the second image (920). At least one processor (210) can identify the address of the place represented by the second image (920) as the address (913) of the place represented by the first image (910).

[0134] FIG. 10a is a flowchart illustrating the operation of enabling the display of an augmented reality (AR) graphic object.

[0135] FIG. 10b illustrates a method for enabling the display of AR graphic objects.

[0136] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0137] Referring to FIG. 10a, in operation 1011, at least one processor (210) can enable displaying an AR graphic object on a camera preview image. For example, at least one processor (210) can enable displaying an AR graphic object on a preview image upon receiving user input.

[0138] According to one embodiment, at least one processor (210) can display a preview image of a camera (220) and, after a reference time has elapsed, enable the display of an AR graphic object on the camera preview image regardless of user input. For example, the reference time may be set by a user.

[0139] In operation 1012, at least one processor (210) may enable identifying whether an electronic device (200) is located within a threshold distance from said address by using information about a stored address while displaying said preview image, based on enabling displaying an AR graphical object on said preview image. For example, at least one processor (210) may not identify whether the electronic device (200) is located within a threshold distance from said address before enabling displaying an AR graphical object on said preview image. At least one processor (210) may identify whether the electronic device (200) is located within a threshold distance from said address by using information about a stored address in response to receiving user input enabling displaying an AR graphical object on said preview image.

[0140] In operation 1013, at least one processor (210) may display information related to an image through a display (230) so as to be associated with a visual object within a preview image corresponding to a place. For example, operation 1013 may correspond to operation 410 of FIG. 4.

[0141] Referring to FIG. 10b, in state (1031), at least one processor (210) can execute a software application for a camera (220) and display an execution screen including a preview image (1020) through a display (230). For example, the execution screen of the software application for the camera (220) may include a preview image (1020) and an element (1021) for changing setting information regarding camera shooting or the preview image (1020). At least one processor (210) can identify an input (1022-1) for the element (1021). In response to the input (1022-1) for the element (1021), at least one processor (210) can change the state of the electronic device (200) from state (1031) to state (1032).

[0142] In state (1032), at least one processor (210) may, in response to input (1022-1), display an object (1023) (e.g., a menu, a window, or a pop-up window) representing a list for changing setting information regarding camera capture or preview image (1020). For example, at least one processor (210) may, in response to input (1022-1), display the object (1023) to be associated with an element (1021).

[0143] For example, the object (1023) may include an executable object (1024) that enables the display of an AR graphical object. User input (1022-2) may be referred to as user input that enables the display of an AR graphical object on a preview image (1020). At least one processor (210) may change the state of the electronic device (200) from state (1032) to state (1033) in response to receiving user input (1022-2) for the executable object (1024).

[0144] In state (1033), at least one processor (210) may enable identifying whether the electronic device (200) is located within a threshold distance from the address of a place based on receiving user input (1022-2). Based on identifying that the electronic device (200) is located within a threshold distance from the address of a place, at least one processor (210) may display information related to the image using an AR graphic object (1025) so as to be associated with a visual object (1026) within the preview image (1020).

[0145] As described above, according to the operation described in FIG. 10a and the method described in FIG. 10b, at least one processor (210) can display a preview image (e.g., preview image (1020)) based on running a software application for a camera (220). While displaying the preview image, at least one processor (210) can identify whether an electronic device (200) is located within a threshold distance from an address based on user input to enable displaying an AR graphical object on the preview image. After the user input is received, it is identified whether the electronic device (200) is located within a threshold distance from an address, so the power consumption and heat generation of the electronic device (200) can be reduced.

[0146] Figure 11 illustrates information related to a visually emphasized image.

[0147] Referring to FIG. 11, at least one processor (210) can identify the location of an electronic device (200) that is separated from a place by a first distance. At least one processor (210) can identify the location of an electronic device (200) that is separated from the place by a second distance that is longer than the first distance. At least one processor (210) can visually highlight information related to an image displayed through a display (230) when the electronic device (200) is separated from the place by a first distance, in relation to information related to an image displayed through a display (230) when the electronic device (200) is separated from the place by a second distance.

[0148] According to one embodiment, at least one processor (210) can visually highlight and display information (1110) related to an image representing a place in relation to a visual object (1130) within a preview image (1100), and information (1120) related to another image representing a different place in relation to another visual object (1140) within a preview image (1100), based on displaying information (1110) related to an image representing a place located closer to the location of the electronic device (200) in relation to information (1120) related to another image representing a different place.

[0149] For example, information (1110) related to the image above may be displayed more prominently through the display (230) within the preview image (1100) in relation to information (1120) related to the other image above. For example, information (1110) related to the image above may be displayed more sharply through the display (230) within the preview image (1100) in relation to information (1120) related to the other image above. For example, at least one processor (210) may display information related to the image (1110) with visual emphasis as the position of the electronic device (200) gets closer to an external object corresponding to a visual object (1130) within the preview image (1100).

[0150] FIG. 12a is a flowchart illustrating the operation of displaying an object to guide a path from the location of an electronic device to an address.

[0151] FIG. 12b illustrates a method of displaying an object to guide a path from the location of an electronic device to an address.

[0152] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0153] Referring to FIG. 12a, in operation 1211, at least one processor (210) can identify that the electronic device (200) is located within a threshold distance from an address while displaying a preview image. For example, at least one processor (210) can receive user input for a portion of the display (230) where information related to the image is displayed. For example, information related to the image may be displayed in a portion of the display (230). For example, the user input may include at least one of a single tap input, a double tap input, a hovering input, a hold and release input, or a drag and drop input.

[0154] For example, the user input may be described as a voice input from a user requesting a path from the location of the electronic device (200) to an address. For example, at least one processor (210) may convert the user's voice input into a text input. At least one processor (210) may identify the user's voice input requesting a path from the location of the electronic device (200) to an address by identifying the text input.

[0155] In operation 1212, at least one processor (210) may display, via a display (230), an object for guiding a path from the location of the electronic device (200) to an address on a preview image. For example, the object for guiding the path may be at least one arrow. For example, the object for guiding the path may be an AR graphical object. For example, at least one processor (210) may change the shape of the object for guiding the path as the location of the electronic device (200) changes. For example, if the object for guiding the path is an arrow, at least one processor (210) may change the shape of the arrow as the electronic device (200) gets closer to the address. For example, at least one processor (210) may adjust the length of the arrow as the electronic device (200) gets closer to the address.

[0156] For example, at least one processor (210) may output an audio signal through a speaker of the electronic device (200) to guide a path from the location of the electronic device (200) to an address. For example, the audio signal may include at least one of information regarding a direction related to the path from the location of the electronic device (200) to an address and information regarding a distance to be traveled in said direction.

[0157] Referring to FIG. 12b, at least one processor (210) can identify that the electronic device (200) is located within a threshold distance from an address while displaying a preview image (1220). At least one processor (210) can change the state of the electronic device (200) from state (1231) to state (1232) based on receiving a user touch input (1223) for a portion of the display (230) where information (1221) related to the image is displayed.

[0158] In state (1232), at least one processor (210) can display an object (1224) for guiding a path from the location of the electronic device (200) to the address on the preview image (1220) via a display (230).

[0159] As described above, according to the operation described in FIG. 12a and the method described in FIG. 12b, at least one processor (210) can provide the effect of guiding a path from the location of the electronic device (200) to the location represented by the stored image to the user of the electronic device (200). At least one processor (210) can enable the user to easily identify the location of the place represented by the image by displaying an object that guides the path.

[0160] FIG. 13a is a flowchart illustrating the operation of displaying an image of the exterior of a place as a superposition on a visual object.

[0161] FIG. 13b illustrates a method of displaying an image of the appearance of a place by superimposing it onto a visual object.

[0162] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0163] Referring to FIG. 13a, in operation 1311, at least one processor (210) can identify that a stored image representing a place includes a graphic object representing the appearance of the place. For example, the stored image may include a graphic object that captures the appearance of a building. For example, the stored image may include a graphic object that captures the sign of a store. For example, at least one processor (210) can identify that the stored image includes a graphic object representing the appearance of the place based on identifying that a part of the graphic object within the stored image represents the appearance of the place.

[0164] In operation 1312, at least one processor (210) may display the graphic object superimposed on a visual object corresponding to the place displayed in the preview image to guide the location of the place on the preview image. For example, operation 1312 may correspond to operation 410 of FIG. 4.

[0165] For example, if only a portion of the graphic object represents the appearance of the place, at least one processor (210) may display the portion of the graphic object superimposed on a visual object corresponding to the place. For example, at least one processor (210) may adjust the size and / or ratio of the graphic object to display the graphic object superimposed on the visual object.

[0166] Referring to FIG. 13b, at least one processor (210) can identify that a stored image (1320) representing a place includes a graphic object (1321) representing the appearance of the place. At least one processor (210) can identify the graphic object (1321). For example, at least one processor (210) can adjust the size and / or ratio of the graphic object (1321) based on identifying the graphic object. For example, at least one processor (210) can display the size and / or ratio-adjusted graphic object (1330) through a display (230) by superimposing it onto a visual object (1341) corresponding to the place in a preview image (1340).

[0167] For example, at least one processor (210) can identify a common part (1342) of a graphic object (1330) representing the appearance of the place and a visual object (1341) corresponding to the place. For example, at least one processor (210) can display all or part of the graphic object (1330) overlaid on a part or all of the visual object (1341) through a display (230).

[0168] As described above, according to the operation described in FIG. 13a and the method described in FIG. 13b, at least one processor (210) can display the graphic object in superposition with the visual object in the preview image, thereby enabling the user of the electronic device (200) to easily identify the location of the place corresponding to the visual object. For example, the user can identify that the graphic object is displayed in superposition with the visual object in the preview image and recognize the capture history of the image corresponding to the graphic object.

[0169] FIG. 14a is a flowchart illustrating an operation of displaying information related to an image based on input for a visual indication.

[0170] FIG. 14b illustrates a method of displaying information related to an image based on input for a visual indication.

[0171] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0172] Referring to FIG. 14a, in operation 1411, at least one processor (210) may display a visual indication via a display (230) that is associated with a visual object and indicates the existence of information related to an image representing a place, based on identifying that the electronic device (200) is located within a threshold distance from an address while displaying a preview image. For example, the visual indication may be described as an executable icon object. For example, at least one processor (210) may display the visual indication superimposed on the visual object. For example, at least one processor (210) may display the visual indication in an area different from the area where the visual object is displayed. For example, if there are two or more stored images representing the place, at least one processor (210) may display a visual object representing the number of stored images representing the place via the display (230) so as to be associated with the visual indication.

[0173] In operation 1412, at least one processor (210) may receive input for the visual indication. For example, the input may be a touch input. For example, the touch input may include at least one of a single tap input, a double tap input, a hovering input, a hold-and-release input, or a drag-and-drop input.

[0174] In operation 1413, at least one processor (210) may display information related to an image through a display (230) so as to associate it with the visual object, based on receiving input for the visual indication. For example, operation 1413 may correspond to operation 410 of FIG. 4.

[0175] Referring to FIG. 14b, at least one processor (210) may display a visual indication (1421) through a display (230) that indicates the presence of information (1425) related to the image, which is associated with a visual object (1424) corresponding to a place within the preview image (1420). At least one processor (210) may change the state of the electronic device (200) from state (1431) to state (1432) based on receiving a user's touch input (1423) for the visual indication (1421).

[0176] In state (1432), at least one processor (210) can display information (1425) related to an image through a display (230) so as to be associated with the visual object (1424) within the preview image (1420).

[0177] As described above, according to the operation described in FIG. 14a and the method described in FIG. 14b, at least one processor (210) can provide a user with an image representing a place corresponding to a visual object in a preview image by displaying a visual indication through a display (230) so as to associate it with a visual object in a preview image. At least one processor (210) can provide the user with information related to the image through the display (230) based on receiving user input regarding the visual indication.

[0178] FIG. 15 illustrates a method for displaying multiple contents included in image-related information within a user interface (UI) in a camera preview.

[0179] Referring to FIG. 15, at least one processor (210) can identify that a plurality of contents are included in information related to an image displayed in association with a visual object (1522) within a preview image (1510). At least one processor (210) can display the plurality of contents as a single group image (1520). For example, at least one processor (210) can display the group image (1520) through a display (230) so as to be associated with the visual object (1522).

[0180] For example, at least some of the multiple contents included in the group image (1520) may be displayed superimposed on the remaining contents. For example, at least one processor (210) may change the state of the electronic device (200) from state (1551) to state (1552) based on receiving user input (1521) for a part area of ​​the display where the group image (1520) is displayed.

[0181] In state (1552), at least one processor (210) can display the plurality of contents (1535, 1536, and 1537) without being overlapped. For example, the user input may include at least one of a single tap input, a double tap input, a hovering input, a hold-and-release input, or a drag-and-drop input.

[0182] According to one embodiment, at least one processor (210) may display a scroll (1531) within an area where the plurality of contents (1535, 1536, and 1537) are displayed when some of the content included in the information related to the image is not displayed due to spatial constraints of the screen. For example, at least one processor (210) may display a scroll (1531) below an area where the plurality of contents (1535, 1536, and 1537) are displayed. For example, at least one processor (210) may display, through a display (230), at least one content that is distinguished from the content (1535, 1536, and 1537) among the plurality of contents included in the information related to the image, based on receiving an input for the scroll (1531) (e.g., left / right scroll input or up / down scroll input). For example, at least one processor (210) may stop displaying all or part of the contents (1535, 1536, and 1537) based on receiving input for the scroll (1531).

[0183] According to one embodiment, at least one processor (210) may display a scroll (1531) within an area where a plurality of contents (1535, 1536, and 1537) are displayed. At least one processor (210) may change the order in which the plurality of contents (1535, 1536, and 1537) are displayed based on receiving input for the scroll (1531). For example, at least one processor (210) may display content (1536) and / or content (1537) in the left area of ​​the group image (1534) and display content (1535) in the right area of ​​the group image (1534) based on receiving input for the scroll (1531). For example, at least one processor (210) may display content (1536), content (1537), and content (1535) in order from left to right based on a first input to the scroll (1531). For example, at least one processor (210) may display content (1537), content (1535), and content (1536) in order from left to right based on a second input to the scroll (1531). For example, the plurality of contents may be displayed in order from left to right according to the stored time order. For example, at least one processor (210) may display the most recently stored content among the plurality of contents in the leftmost area of ​​the group image (1534). For example, at least one processor (210) can display the oldest stored content among the plurality of contents in the rightmost area of ​​the group image (1534).

[0184] According to one embodiment, at least one processor (210) can identify that the priority of an image corresponding to at least one content (1535) among a plurality of contents included in information related to the image is higher than the priority of images corresponding to contents (1536 and 1537) among the plurality of contents excluding the content (1535). For example, at least one processor (210) can identify that the priority of content (1535) corresponding to an image set as a favorite by a user is higher than the priority of contents (1536 and 1537) not set as a favorite by a user.

[0185] For example, at least one processor (210) may highlight content (1535) having a higher priority over other content (1536 and 1537). For example, at least one processor (210) may display a graphic object (1532) highlighting content (1535) through the display (230) based on identifying that the priority of content (1535) is higher than the priority of content (1536 and 1537). For example, at least one processor (210) may display a graphic object (e.g., a graphic object having a star shape) indicating that it is set as a favorite through the display (230) superimposed on the content (1535). For example, at least one processor (210) can display the content (1535) larger than other content (1536 and 1537) through the display (230).

[0186] According to one embodiment, at least one processor (210) can change the state of an electronic device (200) from state (1552) to state (1553) based on receiving user input (1533) for a part area of ​​a display in which one of the horizontally displayed contents (1535) is displayed.

[0187] In state (1553), at least one processor (210) may stop displaying the contents (1536 and 1537) excluding the one content (1535) among the plurality of contents, and display the one content (1540) through the display (230) so as to be associated with the visual object (1522) in the preview image (1510), but is not limited to the example described above.

[0188] FIG. 16 illustrates a method of displaying information related to an image on a map application execution screen.

[0189] Referring to FIG. 16, at least one processor (210) can receive input for displaying a screen through a display (230) and capturing the screen according to operations 401 and 402 of FIG. 4. At least one processor (210) can store an image of the screen and recognize the image based on the input according to operations 403 and 404 of FIG. 4. At least one processor (210) can identify that the image represents a place according to the recognition according to operation 405 of FIG. 4. At least one processor (210) can acquire information (1610) about the address of the place according to operation 406 of FIG. 4 and store the information (1610). At least one processor (210) can execute a software application for a map. At least one processor (210) can display the execution screen (1600) of the application through a display (230). While displaying the execution screen (1600), at least one processor (210) can identify whether the electronic device (200) is located within a threshold distance from the address using information regarding the stored address. Based on identifying that the electronic device (200) is located within a threshold distance from the address, at least one processor (210) can display information related to an image (1610) through the display (230) so as to be associated with a visual object (1640) within the execution screen (1600) corresponding to the location.

[0190] According to one embodiment, at least one processor (210) may display an object (1650) for guiding a path from a location (1630) of an electronic device (200) to an address (1640) on the execution screen (1600) via a display (230). For example, the object (1650) for guiding the path may be at least one arrow. For example, at least one processor (210) may change the shape of the object (1650) for guiding the path as the location (1630) of the electronic device (200) moves. For example, if the object (1650) for guiding the path is an arrow, at least one processor (210) may change the shape of the arrow as the location (1630) of the electronic device (200) approaches the address (1640). For example, at least one processor (210) can adjust the length of the arrow as the electronic device (200) approaches the address.

[0191] According to one embodiment, at least one processor (210) may display information related to an image (1610) through a display (230) to be associated with a visual object within the execution screen (1600), based on identifying that the electronic device (200) is located within a threshold distance from an address (1640) while displaying the execution screen (1600). At least one processor (210) may change the state of the electronic device (200) from state (1661) to state (1662) based on receiving a user touch input (1620) for a portion of the display (230) where the information related to the image (1610) is displayed.

[0192] In state (1662), at least one processor (210) can display an object (1650) for guiding a path from the location (1630) of the electronic device (200) to the address (1640) on the execution screen (1600) via a display (230).

[0193] FIG. 17 illustrates a method of displaying an execution screen including a map screen and another execution screen including additional information about an image as a split view.

[0194] Referring to FIG. 17, at least one processor (210) can execute a software application for a map and display an execution screen (1710) through a display (230). While displaying the execution screen (1710), the at least one processor (210) can display another execution screen (1720) of the software application, which includes the execution screen (1710) in which information (1712) related to an image can be displayed and additional information (1721) of the image, through the display (230) as a split view, based on identifying that the electronic device (200) is located within a threshold distance from an address (1711). For example, the additional information (1721) of the image may include at least one of the business name of the place represented by the image, text included in the signboard, reviews by people who have visited the place, goods or food sold at the place, and / or services provided at the place. For example, additional information (1721) of an image can be described as meta-information of the image. For example, at least one processor (210) can adjust the size and / or ratio of the execution screen (1710) and the other execution screen (1720).

[0195] According to one embodiment, at least one processor (210) can execute a software application for a map and display an execution screen through a display (230). For example, at least one processor (210) can display information related to an image so as to be associated with a visual object within the execution screen. For example, at least one processor (210) can receive user input for a portion of the display (230) where the information related to the image is displayed. For example, based on receiving the input, at least one processor (210) can display another execution screen (1720) of the software application, including the execution screen (1710) and additional information (1721) of the image, as a split view through the display (230).

[0196] According to one embodiment, at least one processor (210) can execute a software application for a map and display an execution screen (1710) through a display (230). While displaying the execution screen (1710), the at least one processor (210) can display the execution screen (1710) and the execution screen of another software application as a split view through the display (230), based on identifying that the electronic device (200) is located within a threshold distance from an address (1711). For example, the other software application may include a gallery application and / or a search engine application.

[0197] For example, if the other software application is a gallery application, at least one processor (210) may receive user input for a portion of a display (230) on which information (1712) related to the image is displayed. Based on receiving the input, at least one processor (210) may identify an image within the gallery application that corresponds to the information (1712) related to the image. At least one processor (210) may display the image on the execution screen of the gallery application through the display (230). For example, at least one processor (210) may display the image on the execution screen of the gallery application and together display information (e.g., information about the location represented by the image) stored in an image description corresponding to the meta information of the image.

[0198] For example, if the other software application is a search engine application, at least one processor (210) may receive user input for a portion of a display (230) on which information (1712) related to the image is displayed. For example, based on receiving the input, at least one processor (210) may perform a search process through the search engine application using the information related to the image. For example, at least one processor (210) may search for a webpage related to a place corresponding to the information related to the image through the search process. For example, at least one processor (210) may perform the search process through the search engine application using information stored in the image description of the image. For example, at least one processor (210) may perform the search process through the search engine application using the text based on identifying that the information stored in the image description contains text representing a uniform resource identifier (URI) related to the place.

[0199] According to one embodiment, at least one processor (210) can execute a software application for a camera (220). At least one processor (210) can display an execution screen through a display (230). At least one processor (210) can display a preview image within the execution screen through the display (230). At least one processor (210) can identify that an electronic device (200) is located within a threshold distance from an address while displaying the preview image. At least one processor (210) can display the execution screen containing information related to the image and additional information of the image, which is displayed to be associated with the preview image and visual objects within the preview image, and another execution screen of the software application containing additional information of the image, as a split view through the display (230).

[0200] FIG. 18 illustrates a method of displaying information related to an image on the screen of a display of an XR (extended reality) device.

[0201] Referring to FIG. 18, an electronic device (200) may be connected to an external electronic device (1810) for providing an XR (extended reality) service. The external electronic device (1810) may display information (1812) related to an image on a screen (1811) based on receiving information (1812) related to an image transmitted by the electronic device (200). For example, the screen (1811) may display the external environment of the external electronic device (1810).

[0202] According to one embodiment, the electronic device (200) may receive an input to display a screen through the display and capture the screen according to operations 401 and 402 of FIG. 4. The electronic device (200) may store an image of the screen and recognize the image based on the input according to operations 403 and 404 of FIG. 4. The electronic device (200) may obtain information about the address of the place and store the information based on identifying that the image represents a place according to operations 405 and 406 of FIG. 4.

[0203] For example, an external electronic device (1810) can display a screen (1811) through a display. The external electronic device (1810) can identify the location of the external electronic device (1810). The external electronic device (1810) can transmit the location information to an electronic device (200). The electronic device (200) can receive the location information of the external electronic device (1810). The electronic device (200) can identify whether the external electronic device (1810) is located within a critical distance from the address of the location, according to operation 409 of FIG. 4, by using the stored information regarding the location information and the address.

[0204] For example, the electronic device (200) may transmit information (1812) related to an image representing a place and / or information about an address to the external electronic device (1810) based on identifying that the external electronic device (1810) is located within a threshold distance from the address. The external electronic device (1810) may display the information related to the image (1812) and / or information about the address in conjunction with a visual object (1820) within the screen (1811) of the second electronic device, based on receiving the information related to the image (1812) and / or information about the address, in accordance with operation 410 of FIG. 4, but is not limited to the examples described above.

[0205] According to an embodiment, the electronic device (200) may provide XR services. The electronic device (200) may perform at least some or all of the operations of the external electronic device (1810) described above. For example, at least one processor (210) of the electronic device (200) may display a screen (1811) representing the external environment of the electronic device (200) through the display (230) of the electronic device (200). According to operation 409 of FIG. 4, while displaying the screen (1811), at least one processor (210) may identify whether the electronic device (200) is located within a threshold distance from the address using stored information regarding the address. At least one processor (210) can display information related to an image (1812) through a display (230) so as to be associated with a visual object (1820) corresponding to a location according to the address within the screen (1811), based on identifying that the electronic device (200) is located within a threshold distance from the address while displaying the screen (1811), in accordance with operation 410 of FIG. 4.

[0206] According to one embodiment, the electronic device may include at least one processor comprising a display, a camera, and a processing circuit, and a memory comprising one or more storage media for storing instructions. The instructions may cause the electronic device to display a screen through the display when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to receive an input for capturing the screen when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to store an image of the screen based on the input when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to recognize the image when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to obtain information about the address of the place based on identifying that the image represents a place according to the recognition when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to store the information when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to display a preview image within the execution screen of the software application through the display, based on executing the software application for the camera, after the information has been stored, when executed individually or collectively by the at least one processor.The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to identify whether the electronic device is located within a threshold distance from the address using the stored information while displaying the preview image. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to display information related to the image through the display, so as to be associated with a visual object within the preview image corresponding to the location, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image.

[0207] For example, the above instructions may cause the electronic device to enable identifying whether the electronic device is located within the threshold distance from the address using the stored information while displaying the preview image, based on enabling displaying an augmented reality (AR) graphic object on the preview image when executed individually or collectively by the at least one processor.

[0208] For example, the instructions may cause the electronic device to store a portion of the image representing the location so as to be associated with the information regarding the address, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to identify the portion of the image stored so as to be associated with the information regarding the address, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to display the information related to the image associated with the visual object through the display by displaying the portion of the image so as to be associated with the visual object, when executed individually or collectively by the at least one processor.

[0209] For example, the instructions may cause the electronic device to identify a text object contained within the image based on recognizing the image when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to identify that the image represents the place based on identifying that the text object represents the place when executed individually or collectively by the at least one processor.

[0210] For example, the instructions may cause the electronic device to identify text objects and / or graphic objects within the image based on recognizing the image when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to obtain information regarding the address of the place from a webpage searched using the text objects and / or graphic objects when executed individually or collectively by the at least one processor.

[0211] For example, the electronic device may further include a communication circuit. The instructions may cause the electronic device to generate keywords using the text object and / or the graphic object when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to obtain information about the address of the place using an external search engine based on the generated keywords when executed individually or collectively by the at least one processor.

[0212] For example, the electronic device may further include a communication circuit. The screen may include a representation of a webpage. The instructions may cause the electronic device to obtain information about the address of the location using a text object representing the address of the webpage and / or the address of another webpage within the webpage, when executed individually or collectively by the at least one processor.

[0213] For example, the above screen may be a first screen. The above image may be a first image. The above instructions may cause the electronic device to receive another input for capturing a second screen displayed after the first screen when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to store a second image of the second screen based on the other input when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to recognize the second image when executed individually or collectively by the at least one processor. The above instructions may cause the electronic device to identify that the time at which the second image was acquired is within a reference time range from the time at which the first image was acquired, based on identifying, according to the recognition of the second image, that the second image represents a place and does not include a part indicating an address. The above instructions may cause the electronic device to identify the address of the place represented by the second image as the address of the place represented by the first image, based on identifying that the time at which the second image was acquired is within the reference time range from the time at which the first image was acquired, when executed individually or collectively by the at least one processor.

[0214] For example, the above instructions may cause the electronic device to identify the address of the place represented by the second image as the address of the place represented by the first image, based on identifying that the first image includes a part common to the part of the second image when executed individually or collectively by the at least one processor.

[0215] For example, the above instructions may cause the electronic device to identify the address of the place represented by the second image as the address of the place represented by the first image, based on identifying that the image of the first screen represents the place represented by the image of the second screen by comparing text obtained from a graphic object within the image of the first screen and text obtained from a graphic object within the image of the second screen when executed individually or collectively by the at least one processor.

[0216] For example, the information related to the image displayed to be associated with the visual object based on identifying the location of the electronic device separated by a first distance from the above location may be visually highlighted in relation to the information related to the image displayed to be associated with the visual object based on identifying the location separated by a second distance from the above location. The first distance may be shorter than the second distance.

[0217] For example, the instructions may cause the electronic device to identify the image representing the location and the other image representing the location, based on identifying that the electronic device is located within the threshold distance from the address, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to emphasize the other image relative to the image, based on identifying that the priority of the other image is higher than the priority of the image, when executed individually or collectively by the at least one processor.

[0218] For example, when the above instructions are executed individually or collectively by the at least one processor, while displaying the preview image, the electronic device may be caused to display, through the display, an object for guiding a path from the location of the electronic device to the address, based on identifying that the electronic device is located within the threshold distance from the address and receiving user input for a portion of the display where the information related to the image is displayed.

[0219] For example, the instructions may cause the electronic device to identify that the stored image includes a graphic object representing the appearance of the place when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to display the information related to the image through the display to be associated with the visual object by displaying the graphic object superimposed on the visual object corresponding to the place displayed in the preview image, in order to guide the location of the place on the preview image when executed individually or collectively by the at least one processor.

[0220] For example, the instructions may cause the electronic device to display, through the display, a visual indication that the information associated with the image exists, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image, when executed individually or collectively by the at least one processor. The instructions may cause the electronic device to display, through the display, the information associated with the image, associated with the visual object, based on receiving an input for the visual indication, when executed individually or collectively by the at least one processor.

[0221] For example, the above instructions may cause the electronic device to display, through the display, as a split view, the execution screen containing information related to the image and another execution screen of the software application containing additional information of the image, which is displayed to be associated with the preview image and the visual object within the preview image, based on identifying that the electronic device is located within the threshold distance from the address while the preview image is displayed when executed individually or collectively by the at least one processor.

[0222] According to one embodiment, a method performed by an electronic device may include an operation of displaying a screen through a display of the electronic device. The method may include an operation of receiving an input for capturing the screen. The method may include an operation of storing an image of the screen based on the input. The method may include an operation of recognizing the image. The method may include an operation of obtaining information about the address of the location based on identifying that the image represents a location according to the recognition. The method may include an operation of storing the information. The method may include an operation of displaying a preview image within the execution screen of the software application through the display, based on executing a software application for the camera of the electronic device after the information is stored. The method may include an operation of identifying whether the electronic device is located within a threshold distance from the address using the stored information while displaying the preview image. The above method may include an operation of displaying information related to the image through the display, such that it is associated with a visual object within the preview image corresponding to the location, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image.

[0223] For example, the above method may include an operation of enabling the identification of whether the electronic device is located within the threshold distance from the address using the stored information while displaying the preview image, based on enabling the display of an augmented reality (AR) graphic object on the preview image.

[0224] For example, the method may include the operation of storing a portion of the image representing the location so as to be associated with the information regarding the address. The method may include the operation of identifying the portion of the image stored to be associated with the information regarding the address, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image. By displaying the portion of the image to be associated with the visual object, the method may include the operation of displaying the information related to the image associated with the visual object through the display.

[0225] For example, the above method may include an operation of identifying a text object included within the image based on recognizing the image. The above method may include an operation of identifying that the image represents the place based on identifying that the text object represents the place.

[0226] For example, the above method may include an operation of identifying text objects and / or graphic objects within the image based on recognizing the image. The above method may include an operation of obtaining information regarding the address of the place from a webpage searched using the text objects and / or graphic objects.

[0227] For example, the above method may include an operation of generating keywords using the text object and / or the graphic object. The above method may include an operation of obtaining information regarding the address of the place using an external search engine based on the generated keywords.

[0228] For example, the above screen may include a representation of a webpage. The above method may include an operation of obtaining information about the address of the above location using a text object representing the address of the webpage and / or the address of another webpage within the webpage.

[0229] For example, the above screen may be a first screen. The above image may be a first image. The method may include an operation of receiving another input for capturing a second screen displayed after the first screen. The method may include an operation of storing a second image of the second screen based on the other input. The method may include an operation of recognizing the second image. The method may include an operation of identifying that the time at which the second image was acquired is within a reference time range from the time at which the first image was acquired, based on identifying that the second image represents a place and does not include a part indicating an address, according to the recognition of the second image. The method may include an operation of identifying the address of the place represented by the second image as the address of the place represented by the first image, based on identifying that the time at which the second image was acquired is within the reference time range from the time at which the first image was acquired.

[0230] For example, the above method may include an operation of identifying the address of the place represented by the second image as the address of the place represented by the first image, based on identifying that the first image includes a part that is common to a part of the second image.

[0231] For example, the above method may include an operation of identifying the address of the place expressed by the second image as the address of the place expressed by the first image, based on identifying that the image of the first screen expresses the place expressed by the image of the second screen by comparing text obtained from a graphic object within the image of the first screen and text obtained from a graphic object within the image of the second screen.

[0232] For example, the information related to the image displayed to be associated with the visual object based on identifying the location of the electronic device separated by a first distance from the above location may be visually highlighted in relation to the information related to the image displayed to be associated with the visual object based on identifying the location separated by a second distance from the above location. The first distance may be shorter than the second distance.

[0233] For example, the method may include an operation of identifying the image representing the location and the other image representing the location based on identifying that the electronic device is located within the threshold distance from the address. The method may include an operation of highlighting the other image with respect to the image based on identifying that the priority of the other image is higher than the priority of the image.

[0234] For example, the above method may include the operation of displaying, through the display, an object for guiding a path from the location of the electronic device to the address on the preview image, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image and receiving user input for a portion of the display on which the information related to the image is displayed.

[0235] For example, the above method may include an operation of identifying that the stored image includes a graphic object representing the appearance of the place. The above method may include an operation of displaying the information related to the image through the display to be associated with the visual object by displaying the graphic object superimposed on the visual object corresponding to the place displayed in the preview image in order to guide the location of the place on the preview image.

[0236] For example, the method may include an operation of displaying a visual indication through the display that is associated with the visual object and indicates the existence of the information associated with the image, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image. The method may include an operation of displaying the information associated with the image through the display so as to be associated with the visual object, based on receiving an input for the visual indication.

[0237] For example, the above method may include an operation of displaying, through the display, as a split view, the execution screen containing information related to the image and another execution screen of the software application containing additional information of the image, which is displayed to be associated with the preview image and the visual object within the preview image, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image.

[0238] According to one embodiment, a non-transient computer-readable storage medium storing one or more programs may store one or more programs. The one or more programs may include instructions that cause the electronic device to display a screen through the display when executed by at least one processor of an electronic device having a display and a camera. The one or more programs may include instructions that cause the electronic device to receive an input for capturing the screen when executed by the at least one processor. The one or more programs may include instructions that cause the electronic device to store an image of the screen based on the input when executed by the at least one processor. The one or more programs may include instructions that cause the electronic device to recognize the image when executed by the at least one processor. The one or more programs may include instructions that cause the electronic device to obtain information about the address of the place based on identifying that the image represents a place according to the recognition when executed by the at least one processor. The above one or more programs may include instructions that cause the electronic device to store the information when executed by the at least one processor. The above one or more programs may include instructions that cause the electronic device to display a preview image within the execution screen of the software application through the display, based on executing a software application for the camera after the information has been stored when executed by the at least one processor.The above one or more programs may include instructions that cause the electronic device to identify, using the stored information, whether the electronic device is located within a threshold distance from the address while displaying the preview image when executed by the at least one processor. The above one or more programs may include instructions that cause the electronic device to display information related to the image through the display, so as to be associated with a visual object within the preview image corresponding to the location, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image when executed by the at least one processor.

[0239] The electronic device according to the embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0240] The embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order). Where any component (e.g., the first) is referred to as "coupled" or "connected" to another component (e.g., the second), with or without the terms "functionally" or "communicationally," it means that said component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0241] In one embodiment of this document, the term “module” used may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0242] One embodiment of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0243] According to one embodiment, the method according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., CD-ROM (compact disc read-only memory)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0244] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

In electronic devices, display; camera; At least one processor including a processing circuit; and It includes memory that stores instructions and includes one or more storage media, and When the above instructions are executed individually or collectively by the at least one processor, Displaying the screen through the above display, Receive input for capturing the above screen, and Based on the above input, save the image of the above screen, and Recognize the above image, and Based on identifying that the image represents a place according to the above recognition, information regarding the address of the place is obtained, and Store the above information, and After the above information is stored, based on executing a software application for the camera, a preview image is displayed on the display within the execution screen of the software application. While displaying the above preview image, the stored information is used to identify whether the electronic device is located within a critical distance from the address, and Causing the electronic device to display information related to the image through the display, based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image, so as to be associated with a visual object within the preview image corresponding to the location. Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, Based on enabling the display of an AR (augmented reality) graphic object on the above preview image, the electronic device causing the electronic device to enable identifying whether the electronic device is located within the threshold distance from the address using the stored information while displaying the above preview image, Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, A portion of the image representing the above place is stored so as to be associated with the information regarding the above address, and Based on identifying that the electronic device is located within the threshold distance from the address while displaying the above preview image, the portion of the image stored to be associated with the information regarding the address is identified, and By displaying the above portion of the above image in association with the above visual object, the electronic device causes the above information related to the above image associated with the above visual object to be displayed through the display. Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, Based on recognizing the above image, text objects included within the above image are identified, and Based on identifying that the text object represents the place, the electronic device causes to identify that the image represents the place, Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, Based on recognizing the above image, text objects and / or graphic objects within the above image are identified, and Causing the electronic device to obtain the information regarding the address of the place from a webpage searched using the text object and / or the graphic object, Electronic device. In claim 5, The above electronic device further includes a communication circuit, When the above instructions are executed individually or collectively by the at least one processor, Generate keywords using the above text object and / or the above graphic object, and Causing the electronic device to obtain the information regarding the address of the above place using an external search engine based on the above generated keyword, Electronic device. In claim 1, The above electronic device further includes a communication circuit, The above screen includes a representation of a webpage, and When the above instructions are executed individually or collectively by the at least one processor, Causing the electronic device to obtain the information regarding the address of the said place by using a text object representing the address of the said webpage and / or the address of another webpage within the said webpage, Electronic device. In claim 1, the screen is a first screen, and The above image is the first image, and When the above instructions are executed individually or collectively by the at least one processor, Receiving another input for capturing a second screen displayed after the first screen above, and Based on the above other input, the second image of the second screen is stored, and Recognizing the second image above, Based on the recognition of the second image, identifying that the second image represents a place and does not include a part indicating an address, and identifying that the time at which the second image was acquired is within a reference time range from the time at which the first image was acquired, and Based on identifying that the time at which the second image is acquired is within the reference time range from the time at which the first image is acquired, the electronic device causes the address of the place represented by the second image to be identified as the address of the place represented by the first image. Electronic device. In claim 8, when the instructions are executed individually or collectively by the at least one processor, Based on identifying that the first image includes a portion common to a portion of the second image, the electronic device causes the address of the place represented by the second image to be identified as the address of the place represented by the first image. Electronic device. In claim 9, when the instructions are executed individually or collectively by the at least one processor, By comparing text obtained from a graphic object within the image of the first screen and text obtained from a graphic object within the image of the second screen, based on identifying that the image of the first screen represents the place represented by the image of the second screen, the electronic device causes the address of the place represented by the second image to be identified as the address of the place represented by the first image. Electronic device. In claim 1, the information related to the image displayed to be associated with the visual object based on identifying the location of the electronic device spaced apart by a first distance from the location, Visually emphasized in relation to the information associated with the image displayed to be associated with the visual object based on identifying the location spaced apart by a second distance from the above location, and The above first distance is shorter than the above second distance, Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, Based on identifying that the electronic device is located within the threshold distance from the address, the image representing the location and another image representing the location are identified, and Causing the electronic device to emphasize the other image relative to the image based on identifying that the priority of the other image is higher than the priority of the image, Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, While displaying the above preview image, the electronic device identifies that it is located within the threshold distance from the above address, and based on receiving user input for a portion of the display on which the information related to the image is displayed, causes the electronic device to display, on the preview image, through the display, an object for guiding a path from the location of the electronic device to the above address. Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, Identifying that the above-mentioned stored image includes a graphic object representing the appearance of the above-mentioned place, and An electronic device that causes the graphic object to be superimposed on the visual object corresponding to the location displayed in the preview image in order to guide the location of the place on the preview image, thereby displaying the information related to the image through the display so as to be associated with the visual object. Electronic device. In claim 1, when the instructions are executed individually or collectively by the at least one processor, Based on identifying that the electronic device is located within the threshold distance from the address while displaying the preview image, a visual indication is displayed through the display that is associated with the visual object and indicates the existence of the information associated with the image. Causing the electronic device to display the information related to the image through the display, so as to be associated with the visual object, based on receiving input for the above visual indication. Electronic device.

Images