Electronic device for generating content, and operating method of electronic device
The electronic device addresses the challenge of generating personalized content by filtering images and creating 3D content based on spatial and object information, enabling interactive and dynamic experiences for users.
Patent Information
- Application Number
- PCT/KR2024/006641
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-07-13
- Filing Date
- 2024-05-16
- Publication Date
- 2024-11-21
AI Technical Summary
Current electronic devices lack the capability to effectively classify and generate content based on pre-photographed images using spatial and object information, limiting their ability to provide users with personalized and dynamic content experiences.
An electronic device equipped with a camera, display, and processor that filters images based on GPS information and object detection, generates shooting guides, and creates 3D content by acquiring spatial information and arranging detected objects in a 3D space, allowing users to interact with the generated content.
Enables the creation of personalized and dynamic content experiences by classifying images based on spatial and object information, providing users with interactive 3D content that can be viewed from different perspectives, enhancing user engagement and memory preservation.
Smart Images

Figure KR2024006641_21112024_PF_FP_ABST
Abstract
Description
Electronic devices generating content and methods of operating electronic devices
[0001] The present disclosure relates to an electronic device and a method of operating the electronic device. Specifically, the present disclosure relates to an electronic device that generates content and a method of operating the electronic device.
[0002] As technology advances, technologies are being developed to provide services to users of electronic devices using multiple pre-captured images.
[0003] In particular, services are being developed that classify multiple images into at least one image taken on a similar or recurring date based on the date the images were taken, or classify multiple images into at least one image taken in a nearby location using GPS information, etc., and provide the classified images to users of electronic devices.
[0004] One embodiment of the present disclosure provides an electronic device. The electronic device may include a camera. The electronic device may include a display. The electronic device may include a memory storing at least one instruction and at least one processor executing the at least one instruction stored in the memory. The at least one processor may, by executing the at least one instruction, detect at least one preset object from at least one image stored in the memory. The at least one processor may, by executing the at least one instruction, display a photographing guide based on the at least one image through the display. The at least one processor may, by executing the at least one instruction, capture a real space corresponding to a space included in the at least one image along the displayed photographing guide through a camera to acquire spatial information. The at least one processor may, by executing the at least one instruction, generate content based on the acquired spatial information and the at least one acquired object.
[0005] One embodiment of the present disclosure provides a method for operating an electronic device. The method may include a step of acquiring at least one preset object from at least one image stored in a memory. The method may include a step of displaying a photographing guide based on the at least one image through a display. The method may include a step of acquiring spatial information by photographing a real space corresponding to a space included in the at least one image along the displayed photographing guide using a camera. The method may include a step of generating content based on the acquired spatial information and the at least one acquired object.
[0006] As one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for performing at least one method of the disclosed operating method on a computer may be provided.
[0007] The present disclosure can be readily understood by the combination of the following detailed description and the accompanying drawings, wherein reference numerals refer to structural elements.
[0008] FIG. 1 is a drawing for explaining an electronic device according to one embodiment of the present disclosure.
[0009] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.
[0010] FIG. 3A is a flowchart for explaining an operation method of an electronic device according to one embodiment of the present disclosure.
[0011] FIG. 3b is a flowchart illustrating an operation method of an electronic device that creates a 3D object and a 3D space according to one embodiment of the present disclosure and creates content by placing the 3D object in the 3D space.
[0012] FIG. 3c is a flowchart illustrating an operation of filtering at least one image from a plurality of captured images and acquiring at least one object based on a selection signal according to one embodiment of the present disclosure.
[0013] FIG. 4 is a drawing for explaining the operation of an electronic device according to one embodiment of the present disclosure.
[0014] FIG. 5A is a diagram illustrating an operation of obtaining at least one object included in an object list from at least one image according to one embodiment of the present disclosure.
[0015] FIG. 5b is a diagram illustrating an operation of acquiring at least one object included in a selection signal in at least one image according to one embodiment of the present disclosure.
[0016] FIG. 6 is a drawing for explaining an operation of acquiring spatial information according to a shooting guide according to one embodiment of the present disclosure.
[0017] FIG. 7A is a drawing for explaining an operation of acquiring spatial information by photographing a real space according to a photographing guide displayed on a display, according to one embodiment of the present disclosure.
[0018] FIG. 7b is a drawing for explaining an operation of acquiring spatial information by photographing a real space according to a photographing guide displayed on a display, according to one embodiment of the present disclosure.
[0019] FIG. 7c is a drawing for explaining an operation of acquiring spatial information by photographing a real space according to a photographing guide displayed on a display, according to one embodiment of the present disclosure.
[0020] FIG. 7d is a drawing for explaining an operation of acquiring spatial information by photographing a real space according to a photographing guide displayed on a display, according to one embodiment of the present disclosure.
[0021] FIG. 8A is a diagram illustrating content generated based on acquired spatial information and at least one detected object according to one embodiment of the present disclosure.
[0022] FIG. 8B is a diagram for explaining an operation of determining at least one of a position, shape, or size of at least one object acquired within content based on a control signal according to one embodiment of the present disclosure.
[0023] FIG. 9 is a flowchart illustrating an operation of displaying generated content on a display according to one embodiment of the present disclosure.
[0024] FIG. 10 is a diagram for explaining an operation of displaying content generated on an electronic device in the form of a mobile device according to one embodiment of the present disclosure.
[0025] FIG. 11 is a drawing for explaining an operation of displaying content generated on an electronic device in the form of a head mounted display (HMD) according to one embodiment of the present disclosure.
[0026] The terms used in this disclosure will be briefly explained, and one embodiment of the present disclosure will be specifically described.
[0027] The terms used in this disclosure are selected from widely used, current terms, taking into account the functions of one embodiment of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant embodiments of the disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of the disclosure.
[0028] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art described herein.
[0029] Throughout this disclosure, when a part is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," and the like described herein refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.
[0030] The expression “configured to” as used herein can be used interchangeably with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” does not necessarily mean something that is “specifically designed to” in terms of hardware. Instead, in some contexts, the expression “a system configured to” can mean that the system is “capable of” doing something together with other devices or components. For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor (e.g., an embedded processor) for performing those operations, or a generic-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in memory.
[0031] Additionally, when a component is referred to as being “connected” or “connected” to another component in the present disclosure, it should be understood that the component may be directly connected or connected to the other component, but may also be connected or connected via another component in between, unless otherwise specifically stated.
[0032] In the present disclosure, an 'electronic device' may be a smart phone. However, the present disclosure is not limited thereto, and the 'electronic device' may be implemented as electronic devices of various shapes, such as a TV, a mobile device, a head-mounted display (HMD) device, a laptop computer, a desktop, a tablet PC, an e-book reader, a digital broadcasting terminal, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), a wearable device, etc.
[0033] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, one embodiment of the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted to clearly describe one embodiment of the present disclosure, and similar parts are designated with similar drawing reference numerals throughout the present disclosure.
[0034] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.
[0035] FIG. 1 is a drawing for explaining an electronic device according to one embodiment of the present disclosure.
[0036] Referring to FIG. 1, in one embodiment, the electronic device (100) may be an electronic device for generating content (140) based on spatial information acquired through the electronic device (100) and at least one object (121, 122) acquired from at least one image (111) stored in the electronic device (100). In one embodiment, the electronic device (100) may also provide the generated content (140) to a user using the electronic device (100).
[0037] Although FIG. 1 illustrates an electronic device (100) having the shape of a smart phone, the present disclosure is not limited thereto. The electronic device (100) may be implemented as a display device of various shapes, such as a mobile device, a wearable device, a head-mounted display (HMD) device, a tablet PC, a laptop computer, etc.
[0038] In one embodiment, the electronic device (100) can filter a plurality of captured images stored in the electronic device (100) according to preset conditions to obtain at least one image (111) (110). In one embodiment, the plurality of captured images may be captured in advance through the electronic device (100) and stored in the electronic device (100), or may be provided to the electronic device (100) from an external server or a peripheral electronic device. In one embodiment, the preset condition may be filtering at least one captured image captured at a nearby location based on a Global Positioning System (GPS). The preset condition may be extracting key points of each captured image and filtering at least one captured image having the same key point. Hereinafter, the operation (110) of filtering a plurality of captured images to obtain at least one image (111) will be described later with reference to FIG. 4.
[0039] In one embodiment, the electronic device (100) may obtain at least one object included in at least one filtered image (111) (120). In one embodiment, the electronic device (100) may obtain at least one preset object (121) from at least one image (111) (120). In one embodiment, the at least one preset object (121) may include an object that may be of interest to a user using the electronic device (100). In one embodiment, when the filtered images from a plurality of captured images are two or more images, the at least one preset object may include an object that is included in each of the two or more images and is overlapping. In one embodiment, the preset object (121) may also be preset in the form of an object list.
[0040] In one embodiment, the electronic device (100) can obtain a selection signal for selecting at least one object (122) included in at least one image (111). Based on the obtained selection signal, the electronic device (100) can obtain (120) at least one object (122) included in the selection signal from at least one image (111). In one embodiment, the selection signal can be obtained from a user using the electronic device (100) through a user interface included in the electronic device (100). Hereinafter, an operation (120) of obtaining at least one object from at least one image (111) will be described below with reference to FIGS. 4 to 5B.
[0041] In one embodiment, the electronic device (100) may provide a user with a shooting guide (131) based on at least one filtered image (111). In one embodiment, the electronic device (100) may display and provide the user with the shooting guide (131) based on at least one image (111) through a display included in the electronic device (100). In one embodiment, the at least one image (111) may include a space corresponding to a background, and the shooting guide (131) may be a guide for providing the user with information such as a location of a real space (132) corresponding to the space of the at least one image (111). In one embodiment, the real space (132) may mean a space in which a user using the electronic device (100) is located.
[0042] In one embodiment, the electronic device (100) can acquire spatial information (130) by photographing a real space (132) corresponding to the photographing guide (131) through a camera included in the electronic device (100) along a displayed photographing guide (131). In one embodiment, the electronic device (100) can acquire spatial information (130) by photographing a real space corresponding to a space included in at least one image along a photographing guide displayed on a display through a camera. Hereinafter, the operation (130) of acquiring spatial information will be described later with reference to FIGS. 4 and 6 to 7d.
[0043] In one embodiment, the electronic device (100) can generate content (140) based on the acquired spatial information (130) and at least one acquired object (121, 122). In one embodiment, the electronic device (100) can generate content (140) in which at least one 3D object is arranged in a 3D space based on the acquired spatial information (130) and at least one acquired object (121, 122). In one embodiment, the electronic device (100) can generate a 3D space based on the acquired spatial information (130). The electronic device (100) can generate at least one 3D object corresponding to each of the acquired at least one object (121, 122). The electronic device (100) can generate content (140) by arranging the generated at least one 3D object in the generated 3D space. Hereinafter, the operation of generating content (140) will be described later with reference to FIG. 4, FIG. 8a, and FIG. 8b.
[0044] In one embodiment, the electronic device (100) can provide the generated content (140) to a user using the electronic device (100). In one embodiment, the electronic device (100) can display the generated content (140) through a display and provide it to the user. In one embodiment, if the generated content (140) includes a 3D space and at least one 3D object, the user can change the position of the electronic device (100) and change the viewpoint from which the content (140) displayed on the electronic device (100) is observed to view the content (140). Hereinafter, the operation of providing the generated content (140) to the user will be described later with reference to FIGS. 9 to 11.
[0045] However, the present disclosure is not limited thereto, and the electronic device (100) may provide the generated content (140) to an external server or peripheral electronic device, and the user may receive the generated content (140) through the electronic device (100) through another electronic device.
[0046] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.
[0047] Referring to FIGS. 1 and 2, the electronic device (100) may include a display (200), a camera (210), a memory (220), at least one processor (230), and a communication interface (240). However, not all of the components illustrated in FIG. 2 are essential components. The electronic device (100) may be implemented with more components than the components illustrated in FIG. 2, or may be implemented with fewer components. In one embodiment, the electronic device (100) may further include a user interface for obtaining user input.
[0048] In one embodiment, the display (200), the camera (210), the memory (220), at least one processor (230), and the communication interface (240) may each be electrically and / or physically connected to one another.
[0049] In one embodiment, the display (200) can display a shooting guide or a shooting guide and content. The display (200) can include any one of a liquid crystal display, a plasma display, an organic light emitting diode display, and an inorganic light emitting diode display. However, the present disclosure is not limited thereto, and the display (200) can include other types of displays that can provide a shooting guide and content to a user using the electronic device (100).
[0050] In one embodiment, the camera (210) can capture an image by photographing the real space (130). In one embodiment, the camera (210) can include an RGB camera that can capture an image including RGB information by photographing the real space (130). In one embodiment, the camera (210) can include a plurality of RGB cameras. At least one processor (230) can obtain depth information using an image including RGB information captured by each of the plurality of RGB cameras. Each of the plurality of RGB cameras can capture the real space (130) from a different position. At least one processor (230) can obtain depth information of the real space (130) based on a positional relationship between each of the plurality of RGB cameras, a distance between a plurality of pixels captured at the same point included in an image captured by each of the plurality of RGB cameras, a size of each of the plurality of pixels, and the like. In one embodiment, the camera (210) can include a stereo camera including two RGB cameras.
[0051] Additionally, the camera (210) may include an RGB-Depth camera that captures a real space (130) to obtain an image including RGB information and depth information. In one embodiment, at least one processor (230) may display the real space (132) captured by the camera (210) on the display (200).
[0052] In one embodiment, the memory (220) may store instructions, data structures, and program codes that can be read by at least one processor (230). In one embodiment, there may be more than one memory (220). In the disclosed embodiments, operations performed by at least one processor (230) may be implemented by executing instructions or codes of a program stored in the memory (220).
[0053] In one embodiment, the memory (220) may include at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a mask ROM, a flash ROM, a hard disk drive (HDD), or a solid state drive (SSD). The memory (220) may store instructions or program codes for performing functions or operations of the electronic device (100). Instructions, algorithms, data structures, program codes, and application programs stored in the memory (220) may be implemented in a programming or scripting language such as, for example, C, C++, Java, or an assembler.
[0054] In one embodiment, the memory (220) may store an image filtering module (221), an object acquisition module (222), an object selection module (223), a guide provision module (224), a background shooting module (225), a 3D generation module (226), and a content generation module (227). However, not all of the modules illustrated in FIG. 2 are essential modules. The memory (220) may store more or fewer modules than the modules illustrated in FIG. 2. In addition, the memory (220) may store a plurality of pre-captured captured images.
[0055] In one embodiment, a 'module' included in the memory (220) may mean a unit that processes a function or operation performed by at least one processor (230). The 'module' included in the memory (220) may be implemented as software such as instructions, an algorithm, a data structure, or a program code.
[0056] In one embodiment, the image filtering module (221) may be configured with instructions or program codes related to an operation or function of filtering a plurality of captured images stored in the memory (220) according to preset conditions to obtain at least one filtered image (111). At least one processor (230) may execute the instructions or program codes of the image filtering module (221) to filter a plurality of captured images stored in the memory (220) according to preset conditions to obtain at least one filtered image (111).
[0057] In one embodiment, the image filtering module (221) may be configured with commands or program codes for filtering a plurality of captured images into at least one image having GPS information of adjacent latitude and longitude based on GPS information of each of the plurality of captured images stored in the memory (220). At this time, the criteria for adjacent latitude and longitude may be preset as latitude and longitude included within several hundred meters or several kilometers based on the latitude and longitude of a specific location, or is not limited to any one. At this time, the specific location may be a location having a lot of overlapping latitude and longitude information among the latitudes and longitudes of each of the plurality of captured images. Additionally, it may be the latitude and longitude at which the electronic device (100) is located. Hereinafter, for convenience of explanation, latitude and longitude will be referred to as a location. At least one processor (230) can obtain at least one image filtered to have adjacent location information based on each GPS information among a plurality of captured images stored in the memory (220) by executing commands or program codes of the image filtering module (221).
[0058] In one embodiment, the image filtering module (221) may be configured with commands or program codes for extracting key points from each of a plurality of captured images stored in the memory (220) and filtering the plurality of captured images into at least one image (111) having the same key point. At this time, the key point refers to a part that can be a feature within the image and may include information about the relationship between a specific pixel and surrounding pixels within the image. The key point may be extracted based on information about brightness, color, direction, size, position, etc. between a specific pixel and surrounding pixels.
[0059] In one embodiment, the image filtering module (221) may include Sacle-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Oriented and Rotated BRIEF (ORB), Harris Corner Detector algorithms, etc. In addition, the image filtering module (221) may extract feature points of each of a plurality of captured images using a deep learning-based model such as a Convolutional Neutral Network (CNN), a Vision Transformer (ViT), a Shifted window (Swin) Transformer, etc.
[0060] In one embodiment, the image filtering module (221) may include an algorithm such as K-means Clustering or K-NN (K-Nearest Neighbors). The image filtering module (221) may filter a plurality of captured images into at least one image (111) having the same feature point through the above-described algorithm.
[0061] In one embodiment, at least one processor (230) can extract key points of each of a plurality of captured images stored in the memory (220) by executing instructions or program codes of the image filtering module (221), and obtain at least one image (111) filtered to have the same key point based on each extracted key point.
[0062] In one embodiment, the object acquisition module (222) may be configured with instructions or program codes related to an operation or function for acquiring at least one object from at least one image (111) stored in the memory (220). In one embodiment, the object acquisition module (222) may include an algorithm for detecting an object included in at least one image from at least one image. However, the present disclosure is not limited thereto, and the object acquisition module (222) may also include an algorithm for segmenting an object included in at least one image (111) from at least one image. At least one processor (230) may acquire an object from at least one image stored in the memory (220) by executing the instructions, program codes, or algorithms of the object acquisition module (222).
[0063] In one embodiment, the object acquisition module (222) may be configured with instructions or program codes related to an operation or function for acquiring at least one preset object from at least one image stored in the memory (220) based on at least one preset object. In one embodiment, the at least one preset object may be stored in the form of an object list. The object acquisition module (222) may be configured with instructions or program codes related to an operation or function for acquiring at least one preset object from at least one image stored in the memory (220) based on the object list. The object list may be preset to include people (e.g., parents), pets, etc. However, the objects included in the object list are not limited to any one. At least one processor (230) may detect at least one preset object from at least one image stored in the memory (220) by executing instructions, program codes, or algorithms of the object acquisition module (222). At least one processor (230) can detect an object included in an object list from at least one image stored in the memory (220) by executing a command, program code, or algorithm of the object acquisition module (222).
[0064] In one embodiment, the object acquisition module (222) may be configured with instructions or program codes relating to an operation or function for acquiring at least one object from at least one image (111) filtered from a plurality of captured images stored in the memory (220) using the image filtering module (221). At least one processor (230) may acquire at least one object from at least one image (111) filtered from a plurality of captured images stored in the memory (220) by executing the instructions, program codes or algorithms of the object acquisition module (222).
[0065] In one embodiment, the object selection module (223) may be configured with instructions or program codes related to an operation or function of obtaining a selection signal generated through an operation of selecting an object included in at least one image through a user interface. In one embodiment, the object selection module (223) may be configured with instructions or program codes related to an operation or function of obtaining a selection signal for selecting at least one object included in at least one image (111) stored in the memory (220), and obtaining at least one object included in the selection signal from at least one image based on the obtained selection signal. At least one processor (230) may obtain a selection signal generated through an operation of selecting an object included in at least one image through a user interface by executing the instructions, program codes, or algorithms of the object selection module (223). At least one processor (230) may obtain at least one object included in the selection signal from at least one image based on the obtained selection signal.
[0066] In one embodiment, the object selection module (223) may be configured with instructions or program codes related to an operation or function of obtaining a selection signal generated through an operation of selecting at least one object from at least one image (111) filtered from a plurality of captured images stored in the memory (220) using the image filtering module (221). In one embodiment, the object selection module (223) may be configured with instructions or program codes related to an operation or function of obtaining a selection signal for selecting at least one object included in at least one image (111) filtered from a plurality of captured images stored in the memory (220), and detecting at least one object included in the selection signal from at least one filtered image (111) based on the obtained selection signal. At least one processor (230) may detect at least one object included in the selection signal from at least one filtered image (111) based on the obtained selection signal.
[0067] However, the present disclosure is not limited thereto, and the object selection module (223) is configured with commands or program codes related to an operation or function of obtaining a selection signal generated through an operation of selecting an object included in at least one image or at least one filtered image (111) through a user interface, and the operation or function of obtaining at least one object included in the selection signal among at least one image or at least one filtered image (111) based on the selection signal may be performed through the object obtaining module (222).
[0068] In one embodiment, the guide providing module (224) may be configured with commands or program codes related to an operation or function for providing a shooting guide (131) based on at least one image stored in the memory (220) or at least one filtered image (111) among a plurality of captured images stored in the memory (220). In one embodiment, the guide providing module (224) may be configured with commands or program codes related to an operation or function for displaying a shooting guide (131) based on at least one image stored in the memory (220) or at least one filtered image (111) among a plurality of captured images stored in the memory (220) on the display (200). At least one processor (230) may provide the shooting guide (131) to the user by executing the commands, program codes, or algorithms of the guide providing module (224). At least one processor (230) can provide a shooting guide (131) to a user through a display (200) by executing a command, program code, or algorithm of a guide providing module (224). Hereinafter, for convenience of explanation, the shooting guide (131) is described as being provided to the user by being displayed on the display (200).
[0069] In one embodiment, at least one image or each of a plurality of captured images may be displayed on the display (200) as a shooting guide (131). In one embodiment, the shooting guide (131) may be displayed translucently on the display (200). In one embodiment, the display (200) may display a real space (132) captured by the camera (210). The shooting guide (131) may be displayed on the display (200) so as to overlap with the real space (132) displayed on the display (200). The shooting guide (131) may be translucent and may be displayed on the display (200) together with the overlapping real space (132).
[0070] In one embodiment, when at least one image or at least one image filtered from a plurality of captured images includes a plurality of images, the shooting guide (131) may include a plurality of sub-shooting guides corresponding to each of the plurality of images. In one embodiment, the guide providing module (224) may be configured with commands or program codes related to an operation or function for sequentially displaying the plurality of sub-shooting guides on the display (200).
[0071] In one embodiment, the background shooting module (225) may be configured with commands or program codes related to operations or functions for obtaining spatial information by shooting a real space (132) corresponding to the shooting guide (131) along the shooting guide (131) displayed on the display (200) through the camera (210). At least one processor (230) may execute the commands, program codes, or algorithms of the background shooting module (225) to shoot a real space (132) corresponding to the shooting guide (131) and obtain spatial information. In one embodiment, the spatial information may include RGB information obtained by shooting the space, or RGB information and depth information. In one embodiment, the spatial information may be information generated by shooting a real space (132) corresponding to the shooting guide (131).
[0072] In one embodiment, when at least one image or at least one filtered image is displayed on the display (200) as a shooting guide (131), the background shooting module (225) may be configured with commands or program codes related to an operation or function for capturing a real space (132) corresponding to a space included in the at least one image along the shooting guide (131) via the camera (210) to obtain spatial information. At least one processor (230) may acquire spatial information generated by capturing the real space (132) corresponding to the space included in the at least one image by executing the commands, program codes, or algorithms of the background shooting module (225). At this time, the real space (132) corresponding to the space included in the at least one image may mean a real space corresponding to a space included in at least one image among a part of the surrounding real space where a user using the electronic device (100) is located.
[0073] In one embodiment, when a plurality of sub-shooting guides are sequentially displayed on the display (200), at least one processor (230) can obtain spatial information including a plurality of sub-space information generated by photographing a real space corresponding to a space included in an image corresponding to each of the plurality of sub-shooting guides, along each of the plurality of sub-shooting guides that are sequentially displayed, by executing a command, program code, or algorithm of the background shooting module (225).
[0074] In one embodiment, the 3D generation module (226) may be configured with commands or program codes related to an operation or function of generating at least one 3D object corresponding to at least one object detected through the object acquisition module (222) or the object selection module (223). In one embodiment, the 3D generation module (226) may be configured with commands or program codes related to an operation or function of generating a 3D space based on spatial information acquired through the background shooting module (225).
[0075] In one embodiment, the 3D generation module (226) may include a model that generates a 3D model based on RGB information or RGB information and depth information included in the spatial information. In one embodiment, the 3D generation module (226) may include, but is not limited to, a Generative Adversarial Network (GAN), a Variational Auto-Encoder (VAE), a Diffusion model based on multi-modality (e.g., an SDFusion model), and the like.
[0076] In one embodiment, at least one processor (230) can generate at least one 3D object corresponding to at least one object detected through the object acquisition module (222) or the object selection module (223) by executing a command, program code, or algorithm of the 3D generation module (226). At least one processor (230) can generate a 3D space corresponding to the acquired spatial information by executing a command, program code, or algorithm of the 3D generation module (226).
[0077] In one embodiment, the content creation module (227) may be configured with instructions or program codes related to an operation or function of creating content (140) based on acquired spatial information and at least one detected object. In one embodiment, the content creation module (227) may be configured with instructions or program codes related to an operation or function of creating content (140) by arranging at least one acquired object in a space created based on acquired spatial information. In one embodiment, the content creation module (227) may be configured with instructions or program codes related to an operation or function of creating content (140) by arranging at least one 3D object created through the 3D creation module (226) in a 3D space created through the 3D creation module (226). In this case, the content (140) created by arranging at least one 3D object in the 3D space may be 3D content including 3D information.
[0078] In one embodiment, the memory (220) may store a content provision module. The content provision module may be configured with instructions or program codes related to operations or functions for displaying generated content through the display (200). In one embodiment, at least one processor (230) may display the generated content on the display (200) by executing the instructions, program codes, or algorithms of the content provision module.
[0079] In one embodiment, the memory (220) may further store a control signal module comprising instructions or program codes for an operation or function of obtaining a control signal including information about at least one of a position, shape, or size of at least one generated 3D object within the generated 3D space through a user interface.
[0080] In one embodiment, the content generation module (227) may include instructions or program codes relating to an operation or function of generating content (140) by determining at least one of a position, shape, or size of at least one generated 3D object within the generated 3D space based on the acquired control signal.
[0081] In one embodiment, at least one processor (230) may be configured as at least one of a Central Processing Unit, a microprocessor, a Graphic Processing Unit, an Application Processor (AP), an Application Specific Integrated Circuits (ASICs), a Digital Signal Processor (DSPs), a Digital Signal Processing Device (DSPDs), a Programmable Logic Device (PLDs), a Field Programmable Gate Array (FPGAs), and a Neural Processing Unit or an artificial intelligence (AI) processor designed with a hardware structure specialized for learning and processing an artificial intelligence (AI) model, but is not limited thereto.
[0082] In one embodiment of the present disclosure, if one or more processors included in at least one processor (230) are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.
[0083] In one embodiment of the present disclosure, at least one processor (230) may be configured as a circuit (Circuitry) such as a System on Chip (SoC) or an Integrated Circuit (IC).
[0084] In one embodiment, at least one processor (230) may control the operation of an electronic device (100) that generates content (140) by executing at least one instruction stored in memory (220).
[0085] In one embodiment of the present disclosure, at least one processor (230) can execute various types of modules stored in the memory (220). At least one processor (230) can execute at least one instruction constituting the various types of modules stored in the memory (220). By executing the program or at least one instruction stored in the memory (220), at least one processor (230) can process data according to predefined operation rules or artificial intelligence models.
[0086] In one embodiment of the present disclosure, at least one processor (230) may include multiple processors. In one embodiment of the present disclosure, at least one module among the multiple modules in the memory (220) may be executed by any one of the multiple processors. The remaining modules among the multiple modules stored in the memory (220) may be executed by other processors among the multiple processors.
[0087] In one embodiment, the communication interface (240) can perform data communication with an external server under the control of at least one processor (230). In addition, the communication interface (240) can perform data communication not only with the external server but also with other peripheral electronic devices.
[0088] In one embodiment, the communication interface (240) may perform data communication with a server or other peripheral electronic devices using at least one of data communication methods including, for example, wired LAN, wireless LAN, Wi-Fi, Bluetooth, zigbee, Wi-Fi Direct (WFD), infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), World Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliance (WiGig), and RF communication.
[0089] In one embodiment, at least one processor (230) may receive a plurality of pre-captured images from an external server or peripheral electronic devices via a communication interface (240). At least one processor (230) may also provide generated content (140) to the external server or peripheral electronic devices via the communication interface (240).
[0090] In one embodiment, the electronic device (100) may further include a user interface. In one embodiment, the user interface may include a touch unit, a push button, a voice recognition unit, a motion recognition unit, etc. In one embodiment, the electronic device (100) may obtain a user input provided by a user through the user interface. In one embodiment, the electronic device (100) may obtain a user input from a user who touches, presses, provides a voice, or performs a gesture, etc. on the user interface. In one embodiment, the user may add a new object to the object list or delete an object included in the object list through the user interface. In addition, the user may provide an input for selecting at least one object to be detected from at least one image through the user interface. The electronic device (100) may generate and obtain a selection signal including information for selecting at least one object to be detected from at least one image based on the user input provided through the user interface.
[0091] FIG. 3A is a flowchart illustrating an operation method of an electronic device according to an embodiment of the present disclosure. FIG. 3B is a flowchart illustrating an operation method of an electronic device for generating a 3D object and a 3D space and arranging the 3D object in the 3D space to generate content according to an embodiment of the present disclosure. FIG. 3C is a flowchart illustrating an operation of filtering at least one image from a plurality of captured images and acquiring at least one object based on a selection signal according to an embodiment of the present disclosure.
[0092] Referring to FIGS. 1, 2, and 3A, in one embodiment, the operating method of the electronic device (100) may include a step (S100) of acquiring at least one preset object (121) from at least one image (111). In one embodiment, in the step (S100) of acquiring at least one object (121, 122) from at least one image (111), at least one processor (230) may execute commands or program codes of an object acquisition module (222) to detect at least one object (121) included in an object list from at least one image (111). At this time, the at least one preset object (121) or object list may be captured in advance by a camera (210) included in the electronic device (100) and stored in the memory (220), or may be provided from an external server or a peripheral electronic device through a communication interface (240) and stored in the memory (220).
[0093] In one embodiment, the method of operating the electronic device (100) may include a step (S200) of displaying a shooting guide (131) based on at least one image (111) through a display (200). In one embodiment, in the step (S200) of displaying the shooting guide (131), at least one processor (230) may display the shooting guide (131) on the display (200) by executing commands or program codes of a guide providing module (224).
[0094] In one embodiment, the operating method of the electronic device (100) may include a step (S300) of acquiring spatial information by photographing a real space (132) corresponding to a space included in at least one image (111) according to a displayed photographing guide (131) through a camera (210). In one embodiment, in the step (S300) of acquiring spatial information, at least one processor (230) may execute commands or program codes of a background photographing module (225) to correspond a real space photographed through the camera (210) with a photographing guide (131) displayed on the display (200), and then acquire spatial information generated by photographing the corresponding space.
[0095] In one embodiment, the operating method of the electronic device (100) may include a step (S400) of generating content (140) based on acquired spatial information and at least one acquired object (121). In one embodiment, in the step (S400) of generating content (140), at least one processor (230) may execute commands or program codes of a content generation module (227) to model a space based on the acquired spatial information and place at least one acquired object in the modeled space, thereby generating content (140).
[0096] FIG. 3b illustrates an operation method of an electronic device for generating content using a 3D object generated using at least one acquired object and a 3D space generated based on acquired spatial information according to an embodiment of the present disclosure. Hereinafter, steps identical to those described in FIG. 3a are assigned the same reference numerals, and redundant descriptions are omitted.
[0097] Referring to FIGS. 2, 3A, and 3B, the operating method of the electronic device (100) may include a step (S150) of generating at least one 3D object corresponding to at least one object obtained in the step (S100) of obtaining at least one object (121). In one embodiment, in the step (S150) of generating at least one 3D object, at least one processor (230) may generate at least one 3D object corresponding to at least one object (121) obtained by executing instructions or program codes of a 3D generation module (226).
[0098] In one embodiment, the operating method of the electronic device (100) may include a step (S350) of generating a 3D space based on the spatial information acquired in the step (S300) of acquiring spatial information. In one embodiment, in the step (S350) of generating the 3D space, at least one processor (230) may generate the 3D space based on the acquired spatial information by executing commands or program codes of the 3D generation module (226). In one embodiment, at least one processor (230) may generate the 3D space based on spatial information including RGB information or RGB information and depth information, depending on the type of the camera (210), by using the 3D generation module (226).
[0099] In one embodiment, the operating method of the electronic device (100) may include a step (S410) of generating content by arranging at least one 3D object generated in a generated 3D space. In one embodiment, in the step (S400) of generating content based on spatial information and a detected object, the content may be generated by arranging the 3D object generated in the step (S150) of generating the 3D object in the 3D space generated in the step (S350) of generating the 3D space. At this time, the content in which the 3D object is arranged in the 3D space may be 3D content including 3D information. In one embodiment, in the step (S410) of generating content by arranging at least one 3D object generated in the generated 3D space, at least one processor (230) may generate content by arranging the 3D object generated in the generated 3D space by executing commands or program codes of the content generation module (227).
[0100] FIG. 3C illustrates an operation method of an electronic device for filtering at least one image from a plurality of captured images according to preset conditions and obtaining an object included in a selection signal for selecting an object from at least one image according to an embodiment of the present disclosure. Hereinafter, steps identical to those described in FIGS. 3A and 3B are given the same reference numerals, and redundant descriptions are omitted.
[0101] Referring to FIGS. 2, 3b, and 3c, the operating method of the electronic device (100) may include a step (S50) of filtering at least one image from a plurality of captured images stored in a memory (220) according to a preset condition. In one embodiment, the plurality of captured images may be images captured in advance through a camera (210) and stored in the memory (220). In addition, the plurality of captured images may be images provided from an external server or peripheral electronic devices and stored in the memory (220). In one embodiment, at least one processor (230) may filter at least one image from a plurality of captured images according to a preset condition by executing commands or program codes of an image filtering module (221).
[0102] In one embodiment, the preset condition may be a condition for filtering at least one image captured at a neighboring location among a plurality of captured images. In this case, the location at which each of the plurality of captured images was captured may be obtained using GPS information, etc. Additionally, the preset condition may be a condition for filtering at least one image containing the same feature point among the plurality of captured images.
[0103] In one embodiment, the step (S50) of filtering at least one image from a plurality of captured images according to a preset condition may include a step of filtering the plurality of captured images into at least one image having an adjacent location based on GPS information of each of the plurality of captured images. The step (S50) of filtering at least one image from the plurality of captured images according to a preset condition may include a step of filtering at least one image having the same feature point based on feature points of each of the plurality of captured images.
[0104] In one embodiment, the step of filtering the plurality of captured images based on feature points may be performed after the step of filtering the plurality of captured images based on GPS information. In one embodiment, at least one processor (230) may filter the plurality of captured images into a plurality of images having adjacent locations based on the GPS information, and then filter the plurality of filtered images into at least one image having the same feature point based on feature points of each of the filtered images. However, the present disclosure is not limited thereto, and the step (S50) of filtering at least one image from the plurality of captured images according to preset conditions may perform filtering based on GPS information or may perform filtering based on feature points.
[0105] In one embodiment, the operating method of the electronic device (100) may include a step of obtaining a selection signal for selecting at least one object included in a plurality of captured images. The operating method of the electronic device (100) may include a step (S60) of obtaining at least one object included in the selection signal from the plurality of captured images based on the selection signal. In one embodiment, at least one processor (230) may obtain a selection signal generated according to a user input for selecting at least one object included in the plurality of captured images through a user interface by executing instructions or program codes of the object selection module (223). In one embodiment, the selection signal may include information about at least one object selected by the user. The at least one object included in the selection signal may be at least one preset object or an object not included in an object list. However, the present disclosure is not limited thereto, and the at least one object included in the selection signal may be identical to at least one preset object or an object included in an object list.
[0106] In one embodiment, in the step (S60) of acquiring at least one object included in the selection signal, at least one processor (230) may acquire at least one object included in the selection signal from a plurality of captured images by executing commands or program codes of the object acquisition module (222) or the object selection module (223).
[0107] In one embodiment, the operating method of the electronic device (100) may include a step of obtaining a selection signal for selecting at least one object included in at least one filtered image in a step (S50) of filtering at least one image. The operating method of the electronic device (100) may include a step (S110) of obtaining at least one object included in the selection signal from at least one filtered image based on the selection signal. In one embodiment, at least one processor (230) may obtain a selection signal generated according to a user input for selecting at least one object included in at least one filtered image through a user interface by executing instructions or program codes of the object selection module (223).
[0108] In one embodiment, in the step (S150) of generating at least one 3D object, at least one processor (230) may generate a 3D object corresponding to at least one object included in a selection signal obtained from a plurality of captured images. At least one processor (230) may generate a 3D object corresponding to at least one object included in a selection signal obtained from at least one filtered image. At least one processor (230) may generate a 3D object corresponding to at least one preset object or at least one object included in an object list obtained from at least one filtered image.
[0109] In one embodiment, in the step of generating content (S410), content can be generated by placing a 3D object generated in the step of generating at least one 3D object (S150) in the generated 3D space.
[0110] In one embodiment, the method of operating the electronic device (100) may include a step of obtaining a control signal including information on at least one of a position, a shape, or a size of a 3D object generated in a step of generating a 3D object (S150) within a 3D space generated in a step of generating a 3D space (S350). In one embodiment, the control signal may be a signal generated by a user input for arranging a 3D object within the 3D space or adjusting the shape or size of the 3D object through a user interface.
[0111] In one embodiment, in the step of generating content (S410), the content may be generated by determining at least one of the position, shape, or size of at least one 3D object generated within the generated 3D space based on the acquired control signal.
[0112] Hereinafter, for convenience of explanation, in FIGS. 4 to 8b, the electronic device (100) is described as operating in the manner illustrated in FIG. 3c. However, the present disclosure is not limited thereto.
[0113] FIG. 4 is a diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure. Hereinafter, the same components as those described in FIG. 2 are assigned the same reference numerals, and redundant descriptions are omitted.
[0114] Referring to FIGS. 2, 3C, and 4, in one embodiment, at least one processor (230) may filter a plurality of captured images stored in a memory (220) according to preset conditions by executing commands or program codes of an image filtering module (221). In one embodiment, the memory (220) may refer to a gallery or the like that stores a plurality of pre-captured captured images. In one embodiment, the image filtering module (221) may include a GPS-based filtering module (400) configured with commands or program codes related to an operation or function of filtering a plurality of captured images into at least one image captured at an adjacent location based on GPS. The image filtering module (221) may include a feature-based filtering module (410) configured with commands or program codes related to an operation or function of filtering a plurality of captured images into at least one image having the same feature point based on feature points.
[0115] In one embodiment, at least one processor (230) may filter a plurality of captured images into a plurality of images captured at adjacent locations based on GPS by executing commands or program codes of a GPS-based filtering module (400). At least one processor (230) may filter a plurality of images filtered based on GPS into at least one image having the same feature point by executing commands or program codes of a feature-based filtering module (410). Although FIG. 4 illustrates filtering a plurality of captured images by the GPS-based filtering module (400) and by the feature-based filtering module (410), the present disclosure is not limited thereto. The image filtering module (221) may include only one module among the GPS-based filtering module (400) and the feature-based filtering module (410), and the operational order using the two modules may also be changed.
[0116] In one embodiment, at least one processor (230) can acquire at least one object included in at least one preset object from at least one image filtered by the image filtering module (221) by executing instructions or program code of the object acquisition module (223).
[0117] In one embodiment, at least one processor (230) can obtain at least one object included in a selection signal from a plurality of captured images stored in the memory (220) or at least one image filtered by the image filtering module (221) by executing instructions or program codes of the object selection module (223).
[0118] In one embodiment, at least one processor (230) can display a shooting guide based on at least one image filtered by the image filtering module (221) on the display (200) by executing instructions or program codes of the guide providing module (224). In one embodiment, at least one processor (230) can display at least one filtered image as a shooting guide on the display (200) in a semi-transparent manner.
[0119] In one embodiment, at least one processor (230) may acquire spatial information by executing commands or program codes of the background shooting module (225) to capture a space corresponding to the shooting guide (131) among the real spaces displayed on the display (200) through the camera (210) along the shooting guide (131) displayed on the display (200). In one embodiment, when at least one filtered image is displayed on the display (200) as the shooting guide (131), the real space corresponding to the shooting guide (131) among the surrounding real spaces displayed on the display (200) may be a space corresponding to a space included in at least one filtered image.
[0120] In one embodiment, at least one processor (230) can generate at least one 3D object corresponding to at least one acquired object by executing instructions or program code of the 3D generation module (226). At least one processor (230) can generate a 3D space based on the acquired spatial information by executing instructions or program code of the 3D generation module (226).
[0121] In one embodiment, the 3D generation module (226) may include a 3D object generation module (430) configured with instructions or program codes related to an operation or function of generating a 3D object corresponding to at least one acquired object. The 3D generation module (226) may include a 3D background generation module (440) configured with instructions or program codes related to an operation or function of generating a 3D space based on acquired spatial information. In one embodiment, at least one processor (230) may generate at least one 3D object corresponding to at least one acquired object by executing instructions or program codes of the 3D object generation module (430). At least one processor (230) may generate a 3D space based on acquired spatial information by executing instructions or program codes of the 3D background generation module (440). However, the present disclosure is not limited thereto, and a 3D object and a 3D space may also be generated using a single module.
[0122] In one embodiment, at least one processor (230) can generate content by arranging at least one 3D object generated within the generated 3D space by executing instructions or program codes of the content generation module (227). In one embodiment, the position, shape, and size of the at least one 3D object within the 3D space can be determined based on the position, shape, and size of the object within the space of the image including the acquired at least one object. In one embodiment, at least one processor (230) can generate content by arranging the 3D object within the 3D space based on the position, shape, and size of the object within the space of the image including the acquired at least one object.
[0123] In one embodiment, when a control signal is acquired, at least one processor (230) can determine and place at least one of a position, shape, or size of a 3D object within a 3D space based on the acquired control signal to generate content. In the case of information not included in the control signal, the 3D object can also be placed based on information on the position, shape, and size of the object within the space of an image including at least one acquired object.
[0124] FIG. 5A is a diagram illustrating an operation of acquiring at least one object included in an object list from at least one image, according to one embodiment of the present disclosure. FIG. 5B is a diagram illustrating an operation of acquiring at least one object included in a selection signal from at least one image, according to one embodiment of the present disclosure.
[0125] Referring to FIGS. 2, 4, and 5A, in one embodiment, at least one image filtered through an image filtering module (221) is illustrated in FIG. 5A. In one embodiment, the at least one image may include multiple images. Although three images (500, 510, and 520) are illustrated in FIG. 5A, the present disclosure is not limited thereto. Depending on the types of the multiple captured images stored in the memory (220) and preset conditions, two images or four or more images may be filtered from the multiple captured images.
[0126] In one embodiment, the first image (500), the second image (510), and the third image (520) may be images obtained by filtering images having adjacent GPS information from a plurality of captured images. The first image (500), the second image (510), and the third image (520) may be images obtained by filtering images having adjacent GPS information from a plurality of captured images, and then filtering images having the same feature points.
[0127] In one embodiment, the first image (500), the second image (510), and the third image (530) may be images having location information within tens or hundreds of meters of a specific location (e.g., 'house'). In addition, the first image (500), the second image (510), and the third image (530) may be images having the same feature point (e.g., 'sofa'). Through this, among the multiple captured images, the first image (500), the second image (510), and the third image (530) captured in the same space (e.g., 'the space in the house where the sofa is located') may be filtered.
[0128] In one embodiment, the at least one preset object may include an object determined to attract the attention of a user using the electronic device (100). In one embodiment, the at least one preset object may be set differently according to preset criteria used when filtering multiple captured images. In one embodiment, the at least one preset object may include an object corresponding to a specific location based on GPS information. The at least one preset object may include an object corresponding to a feature point based on feature point information.
[0129] In one embodiment, when filtering a plurality of captured images to images having GPS information of adjacent locations based on a location of a 'house', at least one preset object may include an object determined to attract the user's attention in the 'house'. In one embodiment, the at least one preset object may be set to include a 'person', a 'photo', a 'pet', etc. However, the present disclosure is not limited thereto, and the at least one preset object may vary depending on a specific location that serves as a filtering criterion. In one embodiment, when filtering a plurality of captured images to images having GPS information of adjacent locations based on a location of a 'park', at least one preset object may include an object determined to attract the user's attention in the 'park'. In one embodiment, the at least one preset object may be set to include a 'person', a 'tree', a 'pet', a 'playground', a 'soccer ball', etc.
[0130] In one embodiment, when filtering a plurality of captured images based on an image having the same feature as a 'sofa', at least one preset object may include an object that is determined to attract the user's attention in the 'sofa'. In one embodiment, the at least one preset object may be set to include 'a person sitting on a sofa', 'a pet sitting on a sofa', 'a person around the sofa', 'a pet around the sofa', etc. However, the present disclosure is not limited thereto, and the at least one preset object may vary depending on the feature that serves as the basis for filtering. In one embodiment, when filtering a plurality of captured images based on the same feature as a 'table', at least one preset object may include an object that is determined to attract the user's attention in the 'table'. In one embodiment, the at least one preset object may be set to include 'a person', 'food', 'a cup', 'a vase', etc.
[0131] However, the present disclosure is not limited thereto, and at least one preset object may be set to include 'person', 'photo', 'pet', etc., regardless of GPS information or feature point information.
[0132] In one embodiment, a first person (501) and a pet (502) may be acquired from a first image (500) through an object acquisition module (222). At this time, a sofa (530) may not be acquired from the first image (500) because it is not included in at least one preset object. In one embodiment, a second person (511) may be acquired from a second image (510) through an object acquisition module (222). A third person (521) may be acquired from a third image (520) through an object acquisition module (222).
[0133] In one embodiment, the multiple captured images may have different capture dates or times. Accordingly, even if the first person (501), the second person (511), and the third person (521) detected in the first image (500), the second image (510), and the third image (520) filtered from the multiple captured images are the same person, the ages, poses, clothing, etc. of the first person (501), the second person (511), and the third person (521) may be different. In one embodiment, the capture date of the first image (500) may be the most recent date, the capture date of the third image (520) may be the oldest date, and the capture date of the second image (510) may be between the capture dates of the first image (500) and the third image (520). In one embodiment, if a first person (501), a second person (511), and a third person (521) are the same person, the first person (501) may be the oldest, the third person (521) may be the youngest, and the second person (511) may be the middle age between the first person (501) and the third person (521).
[0134] In one embodiment, by obtaining the same person located in the same space from the first image (500), the second image (510), and the third image (520) that are filtered from a plurality of captured images captured over time and stored in the memory (220), the change of the person over time can be provided to the user.
[0135] Additionally, when the first image (500), the second image (510), and the third image (520) are captured at different times, objects having various poses of the same person within the same space can be acquired.
[0136] In one embodiment, if the people included in the first image (500), the second image (510), and the third image (520) are different people, by acquiring different people located in the same space, memories with multiple people in that space may be provided to the user.
[0137] Referring to FIG. 5B, in one embodiment, a first image (500), a second image (510), and a third image (520) are illustrated in FIG. 5B. FIG. 5B illustrates that a first person (501) and a pet (502) included in the first image (500) are captured, a second person (511) included in the second image (510) is captured, and a third person (521) included in the third image (520) is captured. In addition, FIG. 5B illustrates that, compared to FIG. 5A, an amusement ride (503) included in the first image (500) is further captured.
[0138] In one embodiment, the amusement ride (503) may be an object included in a selection signal generated by a user input (540) that selects the amusement ride (503) from the first image (500). In one embodiment, the electronic device (100) may receive an input for selecting the amusement ride (503) that the user wants to acquire from the first image (500) through a user interface and generate a selection signal. The electronic device (100) may acquire the amusement ride (503) from the first image (500) based on the selection signal. In one embodiment, when the amusement ride (503) is not acquired from the first image (500) because the amusement ride (503) is not included in the preset objects, the user may provide a user input for selecting the amusement ride (503) so that the amusement ride (503) is acquired from the first image (500).
[0139] FIG. 6 is a drawing for explaining an operation of acquiring spatial information according to a shooting guide according to one embodiment of the present disclosure.
[0140] Referring to FIGS. 2, 4, and 6, in one embodiment, FIG. 6 illustrates a surrounding reality space (600) and a shooting guide (610, 620, 630) provided through a guide provision module (224). The surrounding reality space (600) may refer to a space where a user using the electronic device (100) is located. The surrounding reality space (600) may refer to a space adjacent to a user using the electronic device (100).
[0141] In one embodiment, a user can capture a surrounding real space (600) using an electronic device (100). In one embodiment, the user can capture a surrounding real space (600) through a camera (210). At this time, a portion of the surrounding real space (600) captured through the camera (210) may be displayed on the display (200). The user can change the position of the electronic device (100) to display a different real space on the display (200) among the surrounding real spaces (600). When a desired real space among the surrounding real spaces (600) is displayed on the display (200), the user can capture the corresponding real space through the camera (210) to obtain spatial information.
[0142] In one embodiment, the display (200) may display a shooting guide (610, 620, 630). In one embodiment, the shooting guide (610, 620, 630) may be a guide for providing a user with information about a space in which at least one image was captured, i.e., a space. In one embodiment, the shooting guide (610, 620, 630) may display at least one image translucently. Hereinafter, the images corresponding to the shooting guide (610, 620, 630) may each mean at least one filtered image among a plurality of captured images.
[0143] In one embodiment, if at least one image filtered from a plurality of captured images includes a plurality of images, the capture guide (610, 620, 630) may include a plurality of sub-capture guides corresponding to each of the plurality of images. In one embodiment, each of the plurality of sub-capture guides may include information about the space of each of the plurality of images.
[0144] In one embodiment, FIG. 6 illustrates three sub-capture guides (610, 620, 630). However, the present disclosure is not limited thereto, and the number of sub-capture guides may vary depending on the number of filtered multiple images.
[0145] In one embodiment, the first sub-shooting guide (610), the second sub-shooting guide (620), and the third sub-shooting guide (630) may be displayed on the display (200) included in the electronic device (100), respectively. In one embodiment, FIG. 6 illustrates the first sub-shooting guide (610), the second sub-shooting guide (620), and the third sub-shooting guide (630) displayed on the display (200) to obtain spatial information of three images filtered in at least a part of the surrounding real space (600) displayed on the display (200) as the surrounding real space (600) is photographed through the camera (210) while changing the position of the electronic device (100) or the shooting time point of the camera (210). In one embodiment, the first sub-shooting guide (610), the second sub-shooting guide (620), and the third sub-shooting guide (630) may be shooting guides displayed on the display (200) at different locations of the electronic device (100) or shooting points of the camera (210), respectively.
[0146] In one embodiment, the electronic device (100) may acquire spatial information generated by photographing a real space corresponding to a space included in a filtered image corresponding to each of the first sub-shooting guide (610), the second sub-shooting guide (620), and the third sub-shooting guide (630) among the surrounding real space (600) along the first sub-shooting guide (610), the second sub-shooting guide (620), and the third sub-shooting guide (630) through the camera (210). In one embodiment, the spatial information may include first sub-space information corresponding to a space included in the filtered image corresponding to the first sub-shooting guide (610) (e.g., a space including an air conditioner and the left side of a sofa). The spatial information may include second sub-space information corresponding to a space included in the filtered image corresponding to the second sub-shooting guide (620) (e.g., a space including the middle of a sofa). The spatial information may include third sub-spatial information corresponding to a space (e.g., a space including the right side of the sofa) included in the filtered image corresponding to the third sub-shooting guide (630).
[0147] In one embodiment, the electronic device (100) can generate a 3D space based on acquired spatial information. In one embodiment, the spatial information can include information about a real space corresponding to a space included in a filtered image corresponding to each of the first sub-capture guide (610), the second sub-capture guide (620), and the third sub-capture guide (630) among the surrounding real space (600). The electronic device (100) can generate a 3D space corresponding to a space included in at least one filtered image based on acquired spatial information.
[0148] FIG. 7A is a diagram illustrating an operation of acquiring spatial information by photographing a real space according to a photographing guide displayed on a display, according to one embodiment of the present disclosure. FIG. 7B is a diagram illustrating an operation of acquiring spatial information by photographing a real space according to a photographing guide displayed on a display, according to one embodiment of the present disclosure. FIG. 7C is a diagram illustrating an operation of acquiring spatial information by photographing a real space according to a photographing guide displayed on a display, according to one embodiment of the present disclosure.
[0149] Referring to FIGS. 2, 4, 5A, 6 and 7A, in one embodiment, FIG. 7A illustrates an electronic device (100), three images (500, 510, 520) filtered through an image filtering module (221), a first sub-capture guide (610) based on the first image (500), and a user (700) using the electronic device (100).
[0150] In one embodiment, when the filtered image is a plurality of images, the shooting guide may include a plurality of sub-shooting guides corresponding to each of the plurality of images. At least one processor (230) may sequentially display the plurality of sub-shooting guides on the display (200) by executing commands or program codes of the guide provision module (224). At least one processor (230) may acquire sub-space information generated by shooting a real space corresponding to a space included in an image corresponding to one of the sub-shooting guides among the plurality of sub-shooting guides sequentially displayed on the display (200) through the camera (210), along with one of the sub-shooting guides among the plurality of sub-shooting guides sequentially displayed on the display (200).
[0151] In one embodiment, at least one processor (230) may sequentially display a first sub-capture guide (610), a second sub-capture guide (620), and a third sub-capture guide (630) corresponding to a first image (500), a second image (510), and a third image (520) on the display (200). A user (700) may follow the first sub-capture guide (610), the second sub-capture guide (620), and the third sub-capture guide (630) sequentially displayed on the display (200) to capture a real space corresponding to the first sub-capture guide (610), the second sub-capture guide (620), and the third sub-capture guide (630) among the surrounding real space (600). The electronic device (100) can acquire a plurality of sub-space information pieces generated by shooting corresponding to the first sub-shooting guide (610), the second sub-shooting guide (620), and the third sub-shooting guide (630) among the surrounding real space (600).
[0152] In one embodiment, the user (700) can change the position of the electronic device (100) or the shooting point of the camera (210) so that the real space displayed on the display (200) corresponds to the first sub-shooting guide (610). The user (700) can take a picture at the position of the electronic device (100) or the shooting point of the camera (210) as some real space of the surrounding real space (600) displayed on the display (200) corresponds to the first sub-shooting guide (610). An image captured at the position of the electronic device (100) or the shooting point of the camera (210) can include spatial information about the real space corresponding to the space included in the first sub-shooting guide (610) among the real spaces (600). When the first image (500) is provided as a first sub-shooting guide (610), the image captured at the location of the electronic device (100) or the shooting point of the camera (210) may include spatial information about a real space corresponding to the space included in the first image (500) among the real space (600).
[0153] Referring to FIG. 7B, in one embodiment, a second sub-capturing guide (620) based on a second image (510) is illustrated in FIG. 7B. The user (700) can change the position of the electronic device (100) or the shooting point of the camera (210) so that a portion of the surrounding real space (600) displayed on the display (200) corresponds to the second sub-capturing guide (620). The user (700) can take a picture at the position of the electronic device (100) or the shooting point of the camera (210) as the portion of the surrounding real space (600) displayed on the display (200) corresponds to the second sub-capturing guide (620). An image captured at the position of the electronic device (100) or the shooting point of the camera (210) can include spatial information about a real space corresponding to a space included in the second sub-capturing guide (620) among the surrounding real space (600). When the second image (510) is provided as a second sub-shooting guide (620), the image captured at the location of the electronic device (100) or the shooting point of the camera (210) may include spatial information about a real space corresponding to the space included in the second image (510) among the surrounding real space (600).
[0154] Referring to FIG. 7C, in one embodiment, a third sub-capturing guide (630) based on a third image (520) is illustrated in FIG. 7C. The user (700) can change the position of the electronic device (100) or the shooting point of the camera (210) so that a portion of the surrounding real space (600) displayed on the display (200) corresponds to the third sub-capturing guide (630). The user (700) can take a picture at the position of the electronic device (100) or the shooting point of the camera (210) as the portion of the surrounding real space (600) displayed on the display (200) corresponds to the third sub-capturing guide (630). An image captured at the position of the electronic device (100) or the shooting point of the camera (210) can include spatial information about a real space corresponding to a space included in the third sub-capturing guide (630) among the real spaces (600). When the third image (520) is provided as a third sub-shooting guide (630), the image captured at the location of the electronic device (100) or at the shooting point of the camera (210) may include spatial information about a real space corresponding to the space included in the third image (520) among the surrounding real space (600).
[0155] In one embodiment, the second sub-shooting guide (620) may be displayed on the display (200) after the space corresponding to the first sub-shooting guide (610) is captured. The third sub-shooting guide (630) may be displayed on the display (200) after the space corresponding to the second sub-shooting guide (620) is captured. The user (700) may easily capture a space corresponding to a space included in a plurality of filtered images among the real space (600) according to the first to third sub-shooting guides (610, 620, 630) sequentially displayed on the display (200).
[0156] Referring to FIG. 7D , in one embodiment, at least one processor (230) may display a plurality of sub-shooting guides together on the display (200). In one embodiment, as the plurality of sub-shooting guides are displayed on the display (200), the user (700) may start shooting by following one of the plurality of sub-shooting guides and end shooting by following another of the plurality of sub-shooting guides. In one embodiment, the user (700) may start shooting in a real space corresponding to a space included in an image corresponding to one of the plurality of sub-shooting guides in the real space (600), and end shooting in a real space corresponding to a space included in an image corresponding to another of the plurality of sub-shooting guides. In this case, shooting may mean shooting a video that shoots a real space for a plurality of frames through the camera (210).
[0157] In one embodiment, at least one processor (230) may display a shooting order together in a plurality of sub-shooting guides displayed on the display (200) and provide the display to the user (700). In one embodiment, the user (700) may start shooting in one of the sub-shooting guides in which the shooting order is indicated first among the plurality of sub-shooting guides, and may end shooting in another sub-shooting guide in which the shooting order is indicated last. The at least one processor (230) may obtain spatial information from a real space corresponding to one of the sub-shooting guides in which shooting has started among the surrounding real space (600) through the camera (210) to a real space corresponding to another sub-shooting guide in which shooting has ended. The at least one processor (230) may obtain spatial information from a real space corresponding to a space included in an image corresponding to one of the sub-shooting guides in which shooting has started among the surrounding real space (600) to a real space corresponding to a space included in an image corresponding to another sub-shooting guide in which shooting has ended, through the camera (210).
[0158] In one embodiment, when a plurality of captured images are filtered according to preset conditions, at least one processor (230) may display only some of the plurality of sub-capture guides corresponding to each of the plurality of images through the display (200). In one embodiment, at least one processor (230) may display, on the display (200), two sub-capture guides corresponding to two images whose locations are the most separated from each other based on GPS information of each of the plurality of filtered images. At least one processor (230) may display, on the display (200), two sub-capture guides corresponding to two images that include the most non-overlapping feature points within the images based on feature point information of each of the plurality of filtered images. At least one processor (230) may display an order to take a picture first on one of the two sub-capture guides, and may display an order to take a picture last on the other of the two sub-capture guides. At least one processor (230) can obtain spatial information from a space corresponding to a space included in an image corresponding to one sub-shooting guide in which shooting has started among the surrounding real spaces (600) through a camera (210) to a space corresponding to a space included in an image corresponding to another sub-shooting guide in which shooting has ended.
[0159] In one embodiment, FIG. 7d shows a first sub-capture guide (610) and a third sub-capture guide (630) corresponding to the first image (500) and the third image (520), which contain the most non-overlapping feature points among the filtered first image (500), the second image (510), and the third image (520), respectively, displayed on the display (200). In one embodiment, the first image (500) and the third image (520) are positionally most separated from each other, so that the number of non-overlapping feature points among the feature points included in each image may be large.
[0160] In one embodiment, the user (700) can start shooting at the displayed first sub-shooting guide (610) and end shooting at the displayed third sub-shooting guide (630). Through this, at least one processor (230) can obtain spatial information about the space included in the first image (500) and the space included in the third image (520) among the surrounding real space (600), as well as the space between the first image (500) and the third image (520), for example, the space included in the second image (510).
[0161] In one embodiment, at least one processor (230) may display a shooting guide line (640) connecting the first sub-shooting guide (610) and the third sub-shooting guide (630) on the display (200). In one embodiment, the user (700) may start shooting at the first sub-shooting guide (610), move along the shooting guide line (640), and end shooting at the third sub-shooting guide (630). At this time, the shooting guide line (640) may be a line generated to connect the corresponding edges of each of the first sub-shooting guide (610) and the third sub-shooting guide (630). However, the present disclosure is not limited thereto, and the shooting guide line (640) may be a line generated to pass through a real space corresponding to a sub-shooting guide (for example, the second sub-shooting guide (620)) that is not displayed on the display (200) based on a space included in a plurality of filtered images.
[0162] FIG. 8A is a diagram illustrating content generated based on acquired spatial information and at least one detected object according to one embodiment of the present disclosure. FIG. 8B is a diagram illustrating an operation of determining at least one of the position, shape, or size of at least one detected object within content based on a control signal according to one embodiment of the present disclosure.
[0163] Referring to FIGS. 2, 4, 5B, and 8A, in one embodiment, FIG. 8A illustrates content (140) generated through a content generation module (227). In the content (140), a plurality of 3D objects (801, 802, 803, 811, 821) generated through a 3D object generation module (430) are arranged in a 3D space (800) generated through a 3D background generation module (440).
[0164] In one embodiment, the acquired spatial information may include information about a real space corresponding to a space included in a first image (500), a real space corresponding to a space included in a second image (510), and a real space corresponding to a space included in a third image (520). The 3D space (800) may be a space created by modeling in 3D the real space corresponding to the space included in the first image (500), the real space corresponding to the space included in the second image (510), and the real space corresponding to the space included in the third image (520) based on the acquired spatial information.
[0165] In one embodiment, a first person 3D object (801) corresponding to a first person (501) obtained from a first image (500), a pet 3D object (802) corresponding to a pet (502), and an amusement ride 3D object (803) corresponding to an amusement ride (503) may be placed in a space corresponding to the space of the first image (500) in the 3D space (800). A second person 3D object (811) corresponding to a second person (511) obtained from a second image (510) may be placed in a space corresponding to the space of the second image (510) in the 3D space (800). A third person 3D object (821) corresponding to a third person (521) obtained from a third image (520) may be placed in a space corresponding to the space of the third image (520) in the 3D space (800).
[0166] In one embodiment, the position, size, and shape of each of the plurality of 3D objects (801, 802, 803, 811, 821) included in the content (140) within the 3D space (800) may be determined by the position, size, and shape between objects within the space of each of the first image (500), the second image (510), and the third image (520).
[0167] In one embodiment, by generating content (140) in which a first person 3D object (801), a pet 3D object (802), an amusement ride 3D object (803), a second person 3D object (811), and a third person 3D object (821) corresponding to objects obtained from a first image (500), a second image (510), and a third image (520) are arranged in a 3D space (800), an experience of viewing the first image (500), the second image (510), and the third image (520) together can be provided to a user (1010) through the content (140).
[0168] In one embodiment, if the first image (500), the second image (510), and the third image (520) were all captured on different dates, the user (1010) may be provided with experiences and memories of events that occurred in a space corresponding to the 3D space (800) over multiple periods of time. In one embodiment, if the first person (501), the second person (511), and the third person (521) are all the same person, the content (140) generated by the present disclosure may allow the user (1010) to view activities of the same person in a space corresponding to the 3D space over a long period of time.
[0169] In one embodiment, if the first image (500), the second image (510), and the third image (520) are all captured from different viewpoints, the user (1010) may be able to observe events occurring in a space corresponding to the 3D space (800) from multiple viewpoints.
[0170] Referring to FIGS. 2, 4, 5b, and 8b, in one embodiment, in FIG. 8b, the positions of the first human 3D object (804) and the pet 3D object (805) within the 3D space (800) are changed and arranged compared to FIG. 8a. In addition, the position and size of the amusement equipment (503) within the 3D space (800) are changed and arranged.
[0171] In one embodiment, the position of the first person 3D object (804) and the position of the pet 3D object (805) within the 3D space (800) included in the content (141) may be different from the position of the first person (501) and the position of the pet (502) within the space of the first image (500). The position and size of the amusement ride 3D object (806) within the 3D space (800) included in the content (141) may be different from the position and size of the amusement ride (503) within the space of the first image (500).
[0172] In one embodiment, the content (141) illustrated in FIG. 8B may be content (141) generated by determining at least one of the positions, shapes, or sizes of the plurality of 3D objects (804, 805, 806, 811, 821) within the 3D space (800) based on a control signal including information about at least one of the positions, shapes, or sizes of the plurality of 3D objects (804, 805, 806, 811, 821) within the 3D space (800). In one embodiment, a user (700) provides a user input for adjusting the positions and sizes of a first person 3D object (804), a pet 3D object (805), and an amusement ride 3D object (806) within the 3D space (800) through a user interface, and the control signal may be a signal generated based on the acquired user input.
[0173] In one embodiment, the content (141) may include a date displayed on each of the plurality of 3D objects (804, 805, 806, 811, 821). The date displayed on each of the plurality of 3D objects (804, 805, 806, 811, 821) may indicate a date on which an image including an object corresponding to each of the plurality of 3D objects (804, 805, 806, 811, 821) was captured. In one embodiment, the first person 3D object (804) may display a date on which a first image (500) including a first person (501) was captured (e.g., 2021.07). The second person 3D object (811) may display a date on which a second image (510) including a second person (511) was captured (e.g., 2018.04). The third person 3D object (821) may display the date (e.g., 2017.03) on which the third image (520) containing the third person (521) was captured.
[0174] In one embodiment, a user (1010) can select an object to be acquired from the first image (500), the second image (510), and the third image (520) through a selection signal based on user input input through a user interface, thereby generating content (140) including the desired object.
[0175] In one embodiment, a user (1010) can determine at least one of a position, shape, or size of a 3D object within a 3D space (800) through a control signal based on user input input through a user interface, thereby generating content (140) having the desired position, shape, or size of the 3D object.
[0176] FIG. 9 is a flowchart illustrating an operation for displaying generated content on a display according to one embodiment of the present disclosure. Hereinafter, steps identical to those described in FIG. 3c are assigned the same reference numerals, and redundant descriptions are omitted.
[0177] Referring to FIGS. 2, 3c, and 9, in one embodiment, the method of operating the electronic device (100) may include a step (S500) of displaying generated content through the display (200). In one embodiment, the method of operating the electronic device (100) may include a step (S500) of displaying generated content through the display (200) after the step (S410) of generating content.
[0178] In one embodiment, at least one processor (230) can display content generated by executing instructions or program code of a content providing module on a display (200).
[0179] FIG. 10 is a diagram for explaining an operation of displaying content generated on an electronic device in the form of a mobile device according to one embodiment of the present disclosure.
[0180] Referring to FIGS. 2, 8a, 9, and 10, in one embodiment, an electronic device (1000) in the form of a mobile device can display generated content (140) on a display and provide it to a user (1010). The user (1010) can receive the content (140) through the electronic device (1000). The user (1010) can hold the electronic device (1000) in the form of a mobile device in his / her hand and view the content (140) displayed on the display.
[0181] In one embodiment, when the content (140) includes a 3D background and a 3D object, the user (1010) can change the viewing point of the content (140) displayed on the display by changing the position of the electronic device (1000). In one embodiment, the user (1010) can change the viewing point of the 3D background and the 3D object included in the content (140) displayed on the display by moving the electronic device (1000) up, down, right, or left.
[0182] In one embodiment, the electronic device (1000) described in FIG. 10 may be the same electronic device as the electronic device (100) that generates the content (140) described in FIGS. 1 to 8B, or may be a different electronic device. In one embodiment, if the electronic device (1000) described in FIG. 10 is the same electronic device as the electronic device (100) that generates the content (140), the user (1010) may perform an operation of photographing a real space through the camera (210) during the process of generating the content (140). Thereafter, when the user (1010) is provided with the generated content (140), the user (1010) may view a 3D space and a 3D object through the content (140) without photographing the real space through the camera (210).
[0183] In one embodiment, if the electronic device (1000) described in FIG. 10 is a different electronic device from the electronic device (100) that generates the content (140), the electronic device (1000) described in FIG. 10 can receive the content (140) generated from the electronic device (100) and provide it to the user (1010).
[0184] In one embodiment, when at least one image is filtered from a plurality of captured images based on a feature location, a shooting guide may be displayed on the display (200) when a user using the electronic device (100) is located at a location adjacent to a location (e.g., a specific location) where at least one of the filtered images was captured. In addition, the shooting guide may be displayed on the display (200) when the user executes a program or application for generating content (140) according to the present disclosure.
[0185] Additionally, the shooting guide can be displayed on the display (200) and provided to the user using the electronic device (100) even when the user is located at a different location from the specific location where at least one filtered image was captured. In this case, the shooting guide can enable the user to move to a specific location and capture the surrounding real space according to the shooting guide, and thus can be used to advertise stores, tourist attractions, objects, exhibitions, concerts, etc. located at a specific location.
[0186] FIG. 11 is a drawing for explaining an operation of displaying content generated on an electronic device in the form of a head mounted display (HMD) according to one embodiment of the present disclosure.
[0187] Referring to FIGS. 2, 8a, 9, and 11, in one embodiment, an electronic device (1100) in the form of a head-mounted display can display generated content (140) on a display and provide it to a user (1110). The user (1110) can receive the content (140) through the electronic device (1100). The user (1110) can view the content (140) displayed on the display while wearing the electronic device (1100) in the form of a head-mounted display on his / her head.
[0188] In one embodiment, when the content (140) includes a 3D background and a 3D object, the user (1110) can change the viewing point of the content (140) displayed on the display by changing the position of the head wearing the electronic device (1100). In one embodiment, the user (1010) can change the viewing point of the 3D background and the 3D object included in the content (140) displayed on the display by moving the head up, down, left, or right, or by rotating the head.
[0189] In one embodiment, the electronic device (1100) described in FIG. 11 may be the same electronic device as the electronic device (100) that generates the content (140) described in FIGS. 1 to 8B, or may be a different electronic device. In one embodiment, if the electronic device (1100) described in FIG. 11 is the same electronic device as the electronic device (100) that generates the content (140), the user (1110) may perform an operation of photographing a real space through the camera (210) during the process of generating the content (140). As the content (140) is generated, the user (1100) may view a 3D space and a 3D object through the content (140) without photographing the real space through the camera (210).
[0190] In one embodiment, if the electronic device (1100) described in FIG. 11 is a different electronic device from the electronic device (100) that generates the content (140), the electronic device (1100) described in FIG. 11 can receive the content (140) generated from the electronic device (100) and provide it to the user (1110).
[0191] Although FIGS. 10 and 11 illustrate providing content to a user using electronic devices in the form of a mobile device and a head-mounted display, the present disclosure is not limited thereto. The electronic devices for generating content (140) and providing the generated content (140) according to the present disclosure may include various shapes and are not limited to any one.
[0192] To solve the above-described technical problem, in one embodiment, an electronic device is provided. The electronic device may include a camera. The electronic device may include a display. The electronic device may include a memory storing at least one instruction and at least one processor executing the at least one instruction stored in the memory. The at least one processor may, by executing the at least one instruction, acquire at least one preset object from at least one image stored in the memory. The at least one processor may, by executing the at least one instruction, display a photographing guide based on the at least one image through the display. The at least one processor may, by executing the at least one instruction, capture a real space corresponding to a space included in the at least one image along the displayed photographing guide through the camera to acquire spatial information. The at least one processor may, by executing the at least one instruction, generate content based on the acquired spatial information and the at least one acquired object.
[0193] In one embodiment, at least one processor can generate at least one 3D object corresponding to each of the acquired at least one object by executing at least one command. At least one processor can generate a 3D space based on the acquired spatial information by executing at least one command. At least one processor can generate content in which at least one 3D object generated is placed in the generated 3D space by executing at least one command.
[0194] In one embodiment, at least one processor may obtain a control signal including information about at least one of a position, a shape, or a size of at least one generated 3D object within the generated 3D space by executing at least one command. At least one processor may generate content by determining at least one of a position, a shape, or a size of at least one generated 3D object within the generated 3D space based on the obtained control signal by executing at least one command.
[0195] In one embodiment, at least one processor may obtain a selection signal for selecting at least one object included in at least one image by executing at least one command. At least one processor may obtain at least one object included in the selection signal from at least one image based on the obtained selection signal by executing at least one command.
[0196] In one embodiment, at least one processor may filter at least one image from a plurality of captured images stored in memory according to preset conditions by executing at least one command. At least one processor may obtain at least one preset object from the at least one filtered image by executing at least one command.
[0197] In one embodiment, at least one processor may display a real-world space captured by a camera via a display by executing at least one command. At least one processor may display at least one image as a shooting guide via the display by executing at least one command.
[0198] In one embodiment, at least one processor may execute at least one command to display a shooting guide overlaid on a real space through a display. The shooting guide may be translucent and displayed on the display along with the overlapping real space.
[0199] In one embodiment, at least one image may include a plurality of images. The shooting guide may include a plurality of sub-shooting guides corresponding to the plurality of images, respectively. At least one processor may sequentially display the plurality of sub-shooting guides through a display by executing at least one command. At least one processor may acquire sub-spatial information by executing at least one command to capture a real space corresponding to a space included in an image corresponding to any one of the sub-shooting guides sequentially displayed on the display through a camera, along any one of the plurality of sub-shooting guides. The spatial information may include a plurality of pieces of sub-spatial information generated along each of the plurality of sub-shooting guides.
[0200] In one embodiment, the plurality of sub-space information may be spatial information obtained by starting shooting in a space corresponding to a space included in an image corresponding to one of the plurality of sub-shooting guides through the camera as one of the plurality of sub-shooting guides is displayed on the display, and ending shooting in a space corresponding to a space included in an image corresponding to another sub-shooting guide through the camera as another of the plurality of sub-shooting guides is displayed on the display.
[0201] In one embodiment, at least one processor may display content generated through a display by executing at least one instruction.
[0202] To address the above-described technical challenges, one embodiment of the present disclosure provides a method for operating an electronic device. The method for operating the electronic device may include a step of acquiring at least one preset object from at least one image stored in a memory. The method for operating the electronic device may include a step of displaying a photographing guide based on the at least one image through a display. The method for operating the electronic device may include a step of acquiring spatial information generated by photographing a real space corresponding to a space included in the at least one image along the displayed photographing guide through a camera. The method for operating the electronic device may include a step of generating content based on the acquired spatial information and the at least one acquired object.
[0203] In one embodiment, a method of operating an electronic device may include a step of generating at least one 3D object corresponding to each of the acquired at least one object. The method of operating an electronic device may include a step of generating a 3D space based on the acquired spatial information. In the step of generating content, the content may be generated by arranging the generated at least one 3D object in the generated 3D space.
[0204] In one embodiment, a method of operating an electronic device may include obtaining a selection signal for selecting at least one object included in at least one image. The method of operating the electronic device may include obtaining at least one object included in the selection signal from at least one image based on the obtained selection signal.
[0205] In one embodiment, a method of operating an electronic device may include a step of filtering at least one image from a plurality of captured images stored in a memory according to preset conditions. The method of operating the electronic device may include a step of acquiring at least one preset object from the at least one filtered image.
[0206] In one embodiment, a method of operating an electronic device may include a step of displaying a real space captured by a camera through a display. In the step of displaying a shooting guide through the display, at least one image may be displayed as a shooting guide through the display.
[0207] In one embodiment, the step of displaying a shooting guide through a display may display the shooting guide so as to overlap with the real space through the display. The shooting guide may be translucent and displayed on the display together with the overlapping real space.
[0208] In one embodiment, at least one image may include a plurality of images. The shooting guide may include a plurality of sub-shooting guides corresponding to the plurality of images, respectively. In the step of displaying the shooting guide through the display, the plurality of sub-shooting guides may be sequentially displayed through the display. In the step of obtaining spatial information, the sub-spatial information may be obtained by photographing a real space corresponding to a space included in an image corresponding to one of the sub-shooting guides, following one of the plurality of sub-shooting guides sequentially displayed on the display through the camera. The spatial information may include a plurality of pieces of sub-spatial information generated according to each of the plurality of sub-shooting guides.
[0209] In one embodiment, the step of obtaining spatial information may include a step of starting to photograph in a real space corresponding to a space included in an image corresponding to one of the plurality of sub-shooting guides, through a camera, as one of the plurality of sub-shooting guides is displayed on the display. The step of obtaining spatial information may include a step of terminating to photograph in a real space corresponding to a space included in an image corresponding to another of the plurality of sub-shooting guides, through a camera, as another of the plurality of sub-shooting guides is displayed on the display, thereby obtaining a plurality of pieces of sub-space information.
[0210] In one embodiment, a method of operating an electronic device may include displaying generated content through a display.
[0211] In order to solve the above-described technical problem, a computer-readable recording medium having recorded thereon a program for performing at least one method of the method of operating an electronic device disclosed in one embodiment can be provided.
[0212] The program executed by the electronic device described in this disclosure may be implemented as hardware components, software components, and / or a combination of hardware components and software components. The program may be executed by any system capable of executing computer-readable instructions.
[0213] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to do a desired thing or may independently or collectively command a processing device to do a desired thing.
[0214] Software may be implemented as a computer program containing instructions stored on a computer-readable storage medium. Examples of computer-readable storage media include magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disks, hard disks, etc.) and optical readable media (e.g., CD-ROMs, DVDs (Digital Versatile Discs)). The computer-readable storage media may be distributed across network-connected computer systems, so that computer-readable code may be stored and executed in a distributed manner. The storage media may be readable by a computer, stored in a memory, and executed by a processor.
[0215] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, the term "non-transitory storage media" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage media and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.
[0216] Additionally, programs according to the embodiments disclosed herein may be provided as part of a computer program product. The computer program product may be traded as a commodity between sellers and buyers.
[0217] A computer program product may include a software program, or a computer-readable storage medium storing the software program. For example, the computer program product may include a product in the form of a software program (e.g., a downloadable application) distributed electronically by the manufacturer of the electronic device (100) or through an electronic marketplace (e.g., the Samsung Galaxy Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily generated. In this case, the storage medium may be a storage medium of a server of the manufacturer of the electronic device (100), a server of the electronic marketplace, or a relay server that temporarily stores the software program.
Claims
1. In an electronic device (100), Camera (210); display (200); A memory (220) storing at least one instruction; and At least one processor (230) that executes at least one command stored in the memory (220), The at least one processor (230) executes the at least one instruction, Obtaining at least one preset object from at least one image stored in the above memory (220), Through the above display (200), a shooting guide (131) based on at least one image is displayed, Through the above camera (210), spatial information is acquired by photographing a real space (132) corresponding to a space included in at least one image along the above-mentioned photographing guide (131), An electronic device (100) that generates content (140) based on the acquired spatial information and at least one acquired object.
2. In paragraph 1, The at least one processor (230) executes the at least one instruction, Generating at least one 3D object corresponding to at least one object obtained above, Generate a 3D space based on the acquired spatial information, An electronic device (100) that generates the content (140) in which at least one 3D object generated is placed in the generated 3D space.
3. In paragraph 2, The at least one processor (230) executes the at least one instruction, Obtaining a control signal including information on at least one of the position, shape, or size of at least one generated 3D object within the generated 3D space, An electronic device (100) that generates the content (140) by determining at least one of the position, shape, or size of at least one generated 3D object within the generated 3D space based on the acquired control signal.
4. In any one of clauses 1 to 3, The at least one processor (230) executes the at least one instruction, Obtaining a selection signal for selecting at least one object included in at least one image, An electronic device (100) that acquires at least one object included in the selection signal from at least one image based on the acquired selection signal.
5. In any one of clauses 1 to 4, The at least one processor (230) executes the at least one instruction, Filtering at least one image from a plurality of captured images stored in the above memory (220) according to preset conditions, An electronic device (100) for obtaining at least one preset object from at least one filtered image.
6. In any one of clauses 1 to 5, The at least one processor (230) executes the at least one instruction, Through the above display (200), the real space captured through the camera (210) is displayed, An electronic device (100) that displays at least one image as a shooting guide (131) through the display (200).
7. In paragraph 6, The at least one processor (230) executes the at least one instruction, The shooting guide (131) is displayed so as to overlap with the real space through the display (200). The above shooting guide (131) is an electronic device (100) that is translucent and displayed on the display (200) together with the overlapping real space.
8. In any one of clauses 1 to 7, wherein at least one image comprises a plurality of images, The above shooting guide (131) includes a plurality of sub shooting guides each corresponding to the plurality of images, The at least one processor (230) executes the at least one instruction, The plurality of sub-shooting guides are sequentially displayed through the above display (200), Through the camera (210), the real space corresponding to the space included in the image corresponding to one of the plurality of sub-shooting guides sequentially displayed on the display (200) is photographed to obtain sub-space information, following one of the sub-shooting guides among the plurality of sub-shooting guides. An electronic device (100) wherein the above spatial information includes a plurality of sub-spatial information generated along each of the plurality of sub-shooting guides.
9. In paragraph 8, The above multiple sub-space information is, As one of the plurality of sub-shooting guides is displayed on the display (200), shooting begins in a space corresponding to a space included in an image corresponding to one of the sub-shooting guides through the camera (210). An electronic device (100) that is spatial information obtained by terminating shooting in a space corresponding to a space included in an image corresponding to the other sub-shooting guide through the camera (210) as another sub-shooting guide among the plurality of sub-shooting guides is displayed on the display (200).
10. In any one of clauses 1 to 9, The at least one processor (230) executes the at least one instruction, An electronic device (100) that displays the generated content through the display (200).
11. In the method of operating an electronic device, A step (S100) of obtaining at least one preset object from at least one image stored in memory; A step (S200) of displaying a shooting guide based on at least one image through a display; Step (S300) of obtaining spatial information generated by photographing a real space corresponding to a space included in at least one image along the above-mentioned shooting guide through a camera; and An operating method of an electronic device, comprising a step (S400) of generating content based on the acquired spatial information and at least one object acquired.
12. In paragraph 11, The method of operation of the above electronic device is: Step (S150) of generating at least one 3D object corresponding to at least one object obtained above; and It includes a step (S350) of generating a 3D space based on the acquired spatial information, In the step of generating the content (S410), an operation method of an electronic device for generating the content by arranging at least one 3D object generated in the generated 3D space.
13. In either of paragraphs 11 or 12, The method of operation of the above electronic device is: A step of obtaining a selection signal for selecting at least one object included in at least one image; and An operating method of an electronic device, comprising a step (S110) of acquiring at least one object included in the selection signal from at least one image based on the acquired selection signal.
14. In any one of paragraphs 11 to 13, The method of operation of the above electronic device is: A step (S50) of filtering at least one image from a plurality of captured images stored in the memory according to preset conditions; and An operating method of an electronic device, comprising a step (S100) of obtaining at least one preset object from at least one filtered image.
15. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 11 to 14 on a computer.
Citation Information
Patent Citations
Virtual reality presentation system, virtual reality presentation device, and virtual reality presentation method
JP2016045815A
Electronic device and method for guiding photograph thereof
KR1020170016248A
Large size work spindle
KR1020230124365A
Method for reconstructing three dimension information of object and apparatus for the same
KR102129458B1
Apparatus and Method for Converting Neural Network
KR102591312B1
