Electronic device and operation method for providing 3D virtual object related to product

Abstract

A method for displaying a three-dimensional virtual object performed by an electronic device according to various embodiments of the present disclosure may include the step of generating a three-dimensional virtual object based on user input; and the step of synthesizing the three-dimensional virtual object with a real image to display it as augmented reality.

Description

Technology Field

[0001] The present disclosure relates to an electronic device for providing a three-dimensional virtual object related to a product and a method of operating the same. Background Technology

[0002] Augmented Reality (AR) is a technology that enhances the user experience by adding virtual elements (e.g., 3D virtual objects) to the real world. This technology provides users with rich interactions by overlaying or synthesizing visual information onto the real world and can be applied in various fields. In particular, the retail industry is exploring ways to utilize augmented reality to provide product information and strengthen interactions with consumers. The problem to be solved

[0003] According to various embodiments of the present disclosure, the technical problem is to enable a user to easily recognize the size of a 3D virtual object by providing a 3D virtual object displayed as augmented reality by synthesizing it with a real image based on a real object within the real image.

[0004] According to various embodiments of the present disclosure, the technical problem is that a processor determines the display size of a three-dimensional virtual object based on the actual size of the three-dimensional virtual object and the actual size of the actual object within the actual image.

[0005] According to various embodiments of the present disclosure, the technical challenge is to create a three-dimensional virtual object even when the electronic device does not include a LiDAR sensor.

[0006] According to various embodiments of the present disclosure, the technical problem is for a processor to determine the display size of a three-dimensional virtual object based on a scale determined according to zoom in and zoom out.

[0007] According to various embodiments of the present disclosure, the technical problem is for a processor to display various product information (e.g., delivery type, price discount, etc.) around a three-dimensional virtual object.

[0008] The technical problems of the present disclosure are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by a person skilled in the art of the present disclosure from the description below. means of solving the problem

[0009] A method for displaying a three-dimensional virtual object performed by an electronic device according to one embodiment may include the step of generating a three-dimensional virtual object based on user input; and the step of synthesizing the three-dimensional virtual object with a real image to display it as augmented reality. The display size of the three-dimensional virtual object according to one embodiment may be determined based on the actual size of the three-dimensional virtual object and the actual size of the real object within the real image.

[0010] A step of generating a three-dimensional virtual object according to one embodiment may include: generating at least one of an image set acquired from two or more different angles based on user input, LiDAR sensor data, or the actual size of the three-dimensional virtual object; and generating a three-dimensional virtual object based on at least one of the image set, LiDAR sensor data, or the actual size.

[0011] The step of displaying a three-dimensional virtual object in augmented reality by compositing it with a real image according to one embodiment may include the step of displaying the texture of the three-dimensional virtual object on the surface of the three-dimensional virtual object.

[0012] The step of generating a three-dimensional virtual object according to one embodiment may include the step of determining at least one of the actual size or shape of the three-dimensional virtual object based on user input.

[0013] The step of generating a three-dimensional virtual object according to one embodiment may include: transmitting to a server at least one of a set of images acquired from two or more different angles generated based on user input, LiDAR sensor data, or the actual size of the three-dimensional virtual object; and receiving from the server a three-dimensional virtual object generated based on at least one of the set of images, LiDAR sensor data, or the actual size.

[0014] A method according to one embodiment may further include a step of determining whether a reference object having a predetermined actual size and shape is included in an actual image.

[0015] The step of synthesizing a 3D virtual object with a real image and displaying it as augmented reality according to one embodiment may include, when a reference object is included in the real image, a step of determining the display size of the 3D virtual object based on the actual size of the 3D virtual object and the actual size of the reference object.

[0016] The step of synthesizing a three-dimensional virtual object with a real image and displaying it as augmented reality according to one embodiment may include the step of determining the position to display the three-dimensional virtual object based on the position of a reference object.

[0017] The step of synthesizing a three-dimensional virtual object with a real image and displaying it as augmented reality according to one embodiment may include the step of moving or rotating the position of the three-dimensional virtual object in the real image based on user input.

[0018] The step of synthesizing a three-dimensional virtual object with a real image and displaying it as augmented reality according to one embodiment may include: a step of adjusting the magnification of a camera included in an electronic device based on user input; and a step of changing the display size of the three-dimensional virtual object based on the adjusted magnification.

[0019] A method according to one embodiment may further include the step of displaying at least one of delivery type data of a product, price discount data of a product, review data of a product, or a purchase object corresponding to a three-dimensional virtual object within a certain distance from a three-dimensional virtual object.

[0020] Delivery type data according to one embodiment is data related to a delivery form, and the delivery form may include at least one of general delivery, express delivery, fresh food delivery, or overseas delivery.

[0021] A method according to one embodiment may further include the step of displaying product sales list data of a seller account selling products corresponding to a 3D virtual object in an actual image; or the step of displaying recommended product data in an actual image in which the product and product category are the same. Effects of the invention

[0022] According to various embodiments of the present disclosure, by utilizing augmented reality technology to present information about a product to a customer, consumers can view the product in a virtual environment and make a purchase decision as if they were actually experiencing the product.

[0023] According to various embodiments of the present disclosure, by displaying a three-dimensional virtual object corresponding to a product to be purchased in augmented reality, the user can easily compare the size of the three-dimensional actual object included in the actual image with the size of the product to be purchased.

[0024] According to various embodiments of the present disclosure, even if an electronic device does not have a LiDAR sensor, a three-dimensional virtual object can be generated based on images captured from multiple angles through a camera.

[0025] According to various embodiments of the present disclosure, even when zooming in or out in an actual image, by adjusting the display size of a 3D virtual object to match the display size of actual objects according to the scale, the user can compare the size of the 3D actual object included in the actual image with the size of the product to be purchased.

[0026] The effects according to the technical concept of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description in the specification. Brief explanation of the drawing

[0027] FIG. 1 is a drawing illustrating a system according to various embodiments of the present disclosure. FIG. 2 is a block diagram illustrating an electronic device according to various embodiments of the present disclosure. FIG. 3 is a diagram illustrating a method for displaying a three-dimensional virtual object as augmented reality by compositing it with a real image according to various embodiments of the present disclosure. FIG. 4 is a drawing for illustrating a screen for creating a three-dimensional virtual object according to various embodiments of the present disclosure. FIG. 5 is a diagram illustrating a method for acquiring LiDAR sensor data to generate a three-dimensional virtual object according to various embodiments of the present disclosure. FIG. 6 is a diagram illustrating a method for acquiring multiple compositional images to generate a three-dimensional virtual object according to various embodiments of the present disclosure. FIGS. 7a and 7b are drawings for illustrating a three-dimensional virtual object according to the actual size or shape of the three-dimensional virtual object determined based on user input according to various embodiments of the present disclosure. FIG. 8 is a flowchart of the operation of an electronic device according to various embodiments of the present disclosure. Specific details for implementing the invention

[0028] The embodiments of the present disclosure are illustrative for the purpose of explaining the technical concept of the present disclosure. The scope of rights according to the present disclosure is not limited to the embodiments presented below or the specific description thereof.

[0029] All technical and scientific terms used in this disclosure, unless otherwise defined, have the meaning generally understood by those skilled in the art to which this disclosure pertains. All terms used in this disclosure are selected for the purpose of further clarifying this disclosure and are not selected to limit the scope of the rights under this disclosure.

[0030] Expressions such as “comprising,” “comprising,” “having,” etc. used in this disclosure should be understood as open-ended terms implying the possibility of including other embodiments, unless otherwise stated in the phrase or sentence containing such expressions.

[0031] Unless otherwise stated, singular expressions described in this disclosure may include a plural meaning, and this applies likewise to singular expressions described in the claims.

[0032] Expressions such as "first," "second," etc. used in this disclosure are used to distinguish multiple components from one another and do not limit the order or importance of said components.

[0033] As used in this disclosure, the term “part” refers to software or hardware components such as field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs). However, “part” is not limited to hardware and software. “Part” may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, by example, “part” includes components such as software components, object-oriented software components, class components, and task components, as well as processors, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. Components and functions provided within “part” may be combined into a smaller number of components and “parts” or further separated into additional components and “parts.”

[0034] The expression “based on” as used in this disclosure is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and this expression does not exclude additional factors affecting an act or action of a decision or judgment.

[0035] In the present disclosure, where it is stated that a component is "connected" or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, or connected or connected through a new component.

[0036] Embodiments of the present disclosure will be described below with reference to the attached drawings. In the attached drawings, identical or corresponding components are given the same reference numerals. Furthermore, in the description of the embodiments below, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0037] FIG. 1 illustrates an environment in which an electronic device according to one embodiment of the present disclosure may be applied. An electronic device (110) and a user terminal (120, hereinafter referred to as 'terminal') may be connected via a network and may communicate with each other. The electronic device (110) may be a server or may be another terminal distinct from the terminal (120).

[0038] The terminal (120) may be implemented as a terminal capable of transmitting and receiving various information to and from the electronic device (110) via a network. For example, the terminal (120) may be one of a computer, a laptop, a portable communication terminal (such as a smartphone), a portable multimedia device, a wearable device, or an HMD. However, the type of terminal (120) is not limited thereto, and the terminal (120) may be any device capable of receiving information from a user or outputting information to a user, and communicating with the electronic device (110) or other devices via a network. The terminal (120) may provide information received from the electronic device (110) to the user and may receive input from the user and transmit it to the electronic device (110). Input obtained from the user may include various forms of input, such as a click using a mouse, a touch using a touchpad or touch screen, voice recognition, and other electronic inputs.

[0039] The network can serve to connect the electronic device (110) with the terminal (120) or other external devices. For example, the network can provide a connection path so that the terminal (120) can be connected to the electronic device (110) and transmit and receive packet data with the electronic device (110). The network can be implemented as any kind of wired or wireless network, such as, for example, a Local Area Network (LAN), a Wide Area Network (WAN), a Mobile Radio Communication Network, or a Wibro (Wireless Broadband Internet).

[0040] FIG. 2 is a block diagram of an electronic device according to one embodiment of the present disclosure. In one embodiment, the electronic device (200) may be a server or a terminal. In one embodiment, the electronic device (200) may include a communication circuit (210), one or more processors (220) and / or one or more memories (230) as components. In one embodiment, at least one of the components of the electronic device (200) may be omitted, or another component may be added to the electronic device (200). In one embodiment, additionally or alternatively, some components may be implemented as a single or multiple entities. In the present disclosure, one or more processors (220) may be referred to as processors (220). Unless the context clearly indicates otherwise, the expression processors (220) may mean a set of one or more processors. In the present disclosure, one or more memories (230) may be referred to as memories (230). The expression "memory" (230) may mean a set of one or more memories unless the context clearly indicates otherwise. In one embodiment, at least some of the components inside and outside the electronic device (200) may be connected to each other via a bus, GPIO (General Purpose Input / Output), SPI (Serial Peripheral Interface), or MIPI (Mobile Industry Processor Interface), etc., to exchange information (data, signals, etc.).

[0041] A communication circuit (210) can communicate with a user's terminal and an external device. The communication circuit (210) can perform wireless or wired communication between an electronic device (200) and a terminal. For example, the communication circuit (210) can perform wireless communication according to methods such as eMBB (enhanced Mobile Broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (Long-Term Evolution), LTE-A (LTE Advance), NR (New Radio), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), WiBro (Wireless Broadband), WiFi (Wireless Fidelity), Bluetooth, NFC (Near Field Communication), GPS (Global Positioning System), or GNSS (Global Navigation Satellite System). For example, the communication circuit (210) can perform wired communication according to methods such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), RS-232 (Recommended Standard-232), or POTS (Plain Old Telephone Service). In one embodiment, the electronic device (200) may be implemented by integrating it with another device. In this case, the communication circuit (210) may function as a connection circuit or interface connecting the electronic device (200) and the other device.

[0042] The processor (220) can control at least one component of an electronic device (200) connected to the processor (220) by running software (e.g., instructions, programs, etc.). Additionally, the processor (220) can perform various operations related to the present disclosure, such as computation, processing, data generation, and processing. Furthermore, the processor (220) can load data, etc. from memory (230) or store it in memory (230). Moreover, the processor (220) can transmit and receive various information to and from a user's terminal and an external device through a communication circuit (210). In one embodiment, the processor (220) can control the communication circuit (210) to transmit various information, such as information about various pages, to the user's terminal.

[0043] The memory (230) can store various information (data). The information stored in the memory (230) is information acquired, processed, or used by at least one component of the electronic device (200), and may include software (e.g., instructions, programs, etc.). The memory (230) may include volatile and / or non-volatile memory. In the present disclosure, instructions or programs are software stored in the memory (230) and may include an operating system for controlling the resources of the electronic device (200), an application, and / or middleware that provides various functions to the application so that the application can utilize the resources of the electronic device (200). In one embodiment, the memory (230) may store instructions that cause the processor (220) to perform calculations when executed by the processor (220). The memory (230) may store at least a portion of information received from a terminal through the communication circuit (210) and / or information transmitted to the terminal through the communication circuit (210). The processor (220) can store at least a portion of the information received from the terminal through the communication circuit (210) and / or the information transmitted to the terminal through the communication circuit (210) in the memory (230).

[0044] Hereinafter, the operation described as being performed by the electronic device in FIGS. 3 to 8 can be understood as being performed by the processor (220) of the electronic device (200) described in FIG. 2.

[0045] Although process steps, method steps, algorithms, etc. are described in a sequential order in the configuration and flowcharts illustrated in FIGS. 3 and 8, such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods, and algorithms described in the various embodiments of the present disclosure do not need to be performed in the order described in the present disclosure. Furthermore, even if some steps are described as being performed asynchronously, in other embodiments, such steps may be performed simultaneously. Also, the illustration of a process by the drawings does not imply that the illustrated process excludes other variations and modifications thereof, does not imply that any of the illustrated process or any of its steps is essential to one or more of the various embodiments of the present disclosure, and does not imply that the illustrated process is preferred.

[0046] FIG. 3 is a diagram illustrating a method for displaying a three-dimensional virtual object as augmented reality by compositing it with a real image according to various embodiments of the present disclosure.

[0047] A processor (220) according to one embodiment can generate a three-dimensional virtual object based on user input. If the electronic device (200) according to one embodiment is a terminal, the terminal can generate a three-dimensional virtual object based on user input. The processor (220) can generate at least one of a set of images acquired from two or more different angles, LiDAR sensor data, or the actual size (and / or shape) of a three-dimensional virtual object based on user input. The processor (220) can generate a three-dimensional virtual object based on at least one of the set of images, LiDAR sensor data, or the actual size (and / or shape). In another embodiment, the processor (220) can transmit data generated based on user input to a server. The server can generate a three-dimensional virtual object based on the received data and transmit it to the terminal. The processor (220) included in the terminal can display the three-dimensional virtual object received from the server on a screen.

[0048] According to one embodiment, user input can respond to a request to create a three-dimensional virtual object for a specific product. For example, referring to FIG. 4, when user input is received for at least one of a scan object (430), an image upload object (450), or a size input object (470), the processor (220) can generate data corresponding to the user input and create a three-dimensional virtual object based on the generated data.

[0049] A processor (220) according to one embodiment can generate a three-dimensional virtual object based on user input regarding a scan object (430). The scan object (430) may be an object for generating a three-dimensional virtual object based on LiDAR sensor data. LiDAR sensor data can be obtained from a LiDAR sensor. A LiDAR (Light Detection and Ranging) sensor is a technology that measures distance and detects the environment using the wavelength of light. A LiDAR sensor primarily calculates distance by emitting a laser beam and measuring the time it takes for the beam to reflect back from an object, thereby generating LiDAR sensor data (e.g., spatial information of the surrounding environment, size of the object, distance to the object, shape of the object).

[0050] When a scanned object (430) is selected in the screen (400) of FIG. 4, the screen (500) of FIG. 5 may be displayed. Referring to FIG. 5, the screen (500) is a screen that displays guide lines on an actual image based on LiDAR sensor data. The guide lines may be lines that represent spatial information of the surrounding environment. A grid may be formed at regular intervals as multiple guide lines intersect, and the grid length may correspond to a specific length (e.g., 1 cm, 2 cm). The user can determine the length of a specific object based on the number of grid lines. The processor (220) can calculate the distance between the electronic device (200) and the actual object (510) based on the LiDAR sensor data. Through this, the processor (220) can calculate the actual size (e.g., width, length, height, etc.) of the object (510). Similarly, the aforementioned operation is performed on the actual object (530), so that the processor (220) can calculate the actual size of the actual object (530). The processor (220) can generate a three-dimensional virtual object based on LiDAR sensor data captured at multiple different angles of the actual object (510). For example, the user can capture various sides of the actual object (510), such as the front, back, left side, and right side, while moving the electronic device (200). Through this, the processor (220) can acquire LiDAR sensor data for various sides of the actual object (510).

[0051] As used in this specification, the term "actual size" refers to the size of an object in an actual environment, and "display size" refers to the size of an object displayed on a screen within an actual image displayed on a display unit of an electronic device (200).

[0052] A processor (220) according to one embodiment can display a texture of a 3D virtual object on the surface of a virtual object created based on the actual size and shape of the 3D virtual object included in the LiDAR sensor data. A texture is an image or pattern used in computer graphics and may be data applied to the surface of a 3D model to determine the appearance of an object. A texture can be mapped onto the surface of an object to simulate a specific material, color, uniformity, texture, etc., to provide visual realism or to create a specific design. A texture can be generated based on images acquired through a camera included in the electronic device (200). The processor (220) can generate the size and shape of a 3D virtual object based on the LiDAR sensor data. However, a texture may be required to display the surface of the 3D virtual object identically to the actual product. Therefore, the processor (220) can display the surface of the 3D virtual object on the surface using a texture. Through this, the processor (220) can provide the user with a more realistic 3D virtual object. By displaying a texture on the surface of an object, users can compare the colors of real objects included in the actual image with the colors of the 3D virtual object and determine whether they are harmonious. Therefore, when making a purchase decision, users can consider not only the size of the product but also its color to determine whether the product's color is harmonious with the surrounding environment.

[0053] A processor (220) according to one embodiment can generate a three-dimensional virtual object based on user input to an image upload object (450). The image upload object (450) may be an object for generating a three-dimensional virtual object based on a set of images acquired from two or more different angles. The electronic device (220) may or may not include a LiDAR sensor. Even if the electronic device (200) does not include a LiDAR sensor, there may still be a need to generate a three-dimensional virtual object. Therefore, without LiDAR sensor data, the processor (220) can generate a three-dimensional virtual object based on a set of images acquired from two or more different angles. The processor (220) can generate the actual size of the three-dimensional virtual object based on depth data (e.g., distance between the electronic device and the object) acquired by analyzing the images included in the image set. Additionally, the processor (220) can determine the shape of the three-dimensional virtual object based on the depth information. The processor (220) can form a virtual object based on the actual size and shape of the three-dimensional virtual object, and then display the surface of the virtual object based on a texture. The texture can be generated based on an image set. In another embodiment, the processor (220) can transmit the image set to a server. The server can generate data necessary for creating a three-dimensional virtual object based on the image set. The processor (220) can generate a three-dimensional virtual object using the data necessary for creating a three-dimensional virtual object received from the server. Alternatively, the server can generate a three-dimensional virtual object based on an image set.

[0054] A processor (220) according to one embodiment can generate a three-dimensional virtual object based on user input to a size input object (470). The size input object (470) may be an object for generating a three-dimensional virtual object without LiDAR sensor data and an image set. When the size input object (470) of FIG. 4 is selected, the system may switch to the screen (700) of FIG. 7a. The processor (220) can receive the shape (710) (e.g., cylindrical, rectangular, triangular pyramid, etc.) and actual size (720, 721, 722) of the three-dimensional virtual object from the user through the screen (700). The actual size may include, for example, a width (720), a length (721), and a height (722). The processor (220) can determine the actual size (720, 721, 722) and the shape (710) based on user input. And the processor (220) can generate a three-dimensional virtual object based on the determined actual size (720, 721, 722) and shape (710). In reality, the actual size of the three-dimensional virtual object acquired through a LiDAR sensor or camera may not be accurate. Or, it may be an environment where it is difficult to acquire a LiDAR sensor or image set. Even in such cases, the user can generate a three-dimensional virtual object with an accurate size and a shape similar to the product by inputting the actual size of the product displayed in the product details.

[0055] A processor (220) according to one embodiment can display a three-dimensional virtual object created based on actual size (720, 721, 722) and shape (710) by compositing it with actual image and displaying it as augmented reality. Referring to FIG. 7b, on the screen (701) of the actual image captured through the camera of the electronic device (220), not only is the actual object (750) displayed, but a three-dimensional virtual object (730) can also be composited and displayed simultaneously. Through this, the processor (220) can display a three-dimensional virtual object that can be compared in size with the actual object in the actual image without using a LiDAR sensor or camera. And by comparing the three-dimensional virtual object of accurate size with the actual object, the user can compare the size of the product corresponding to the three-dimensional virtual object with other actual objects. In one embodiment, the user can input texture information of the three-dimensional virtual object so that the color or texture of the three-dimensional virtual object is displayed together.

[0056] The screens of FIGS. 4, 5, 7a, and 7b described above may be screens of a terminal connected to a seller account (or administrator account). The terminal of the seller account may create a 3D virtual object for an e-commerce platform and upload it along with product information. The terminal of the seller account may composite the 3D virtual object with actual video and display it as augmented reality. Through this, the seller can verify within the actual video whether the product sold by the seller is accurately represented as a 3D virtual object. For example, the screen (300) of FIG. 3 may be displayed on the terminal of the seller account. Through this, the seller can verify whether the 3D virtual object (310) corresponding to the product sold by the seller is well represented to indicate the product features.

[0057] According to one embodiment, it is necessary to display a 3D virtual object corresponding to a product to a consumer in augmented reality by synthesizing it with a real image. To this end, a 3D virtual object generated corresponding to a product may be transmitted to a server of an e-commerce platform. The server may receive a request for a 3D virtual object for a specific product from a terminal of a consumer account. The server may transmit the 3D virtual object to the terminal of the consumer account. On the screen of the terminal of the consumer account, a real object and a 3D virtual object may be displayed as shown in the screen (300) of FIG. 3. For example, a terminal of a seller account may generate a 3D virtual object for product A. The terminal of the seller account may transmit the 3D virtual object for product A to the server of the e-commerce platform. The terminal of the consumer account may transmit a request for a 3D virtual object for product A to the server. The server may transmit the 3D virtual object for product A to the terminal of the consumer account. The terminal of the consumer account may display the received 3D virtual object for product A on a screen (e.g., a real image).

[0058] A processor (220) according to one embodiment can determine whether a reference object having a predetermined actual size and shape is included in an actual image. The reference object may refer to an object whose actual size and shape data is stored in memory (or memory of a server). For example, the reference object may include items commonly encountered in real life. For example, cans, canned goods, snack boxes, mobile phones, books, electrical outlets, etc., are commonly encountered in real life. After storing data regarding the actual size and shape of these items in memory in advance, if a reference object is present in the actual image, the processor (220) can display the 3D virtual object more accurately on the screen in augmented reality by comparing the actual size of the reference object with the actual size of the 3D virtual object. The processor (220) may receive the actual size and shape of the reference object by calling it from memory included in an electronic device or by requesting it from a server. For example, referring to FIG. 3, an electrical outlet can be a reference object because it is commonly encountered in real life. Accordingly, the processor (220) can determine the display size of the three-dimensional virtual object (310) based on the actual size of the reference object A (330) corresponding to the outlet and the actual size of the three-dimensional virtual object. The size of the three-dimensional virtual object (310) can be determined based on the display position of the three-dimensional virtual object. For example, if the three-dimensional virtual object (310) is positioned close to the user, the display size of the three-dimensional virtual object (310) may be larger than if the three-dimensional virtual object (310) is positioned far from the user. The display position of the three-dimensional virtual object (310) can be determined and / or changed by the user. As another example, a canned food can also be the reference object.The processor (220) can determine the display size of the 3D virtual object (310) based on the actual size of the reference object B (370) corresponding to the can and the actual size of the 3D virtual object.

[0059] A processor (220) according to one embodiment can determine the display size of a 3D virtual object by comparing the actual size of the reference object with the actual size of the 3D virtual object when the reference object is included in the actual image. When the reference object is not included in the actual image, the processor (220) can determine the display size of the 3D virtual object based on the actual size of the actual object in the actual image and the actual size of the 3D virtual object. The actual size of the actual object can be calculated based on the display size of the actual object in the actual image and the distance from the electronic device (200) to the actual object.

[0060] A processor (220) according to one embodiment can determine a location to display a three-dimensional virtual object based on the location of a reference object. For example, the processor (220) can identify the location of a reference object within an actual image. Based on the identified location of the reference object, the processor (220) can determine a location to display a three-dimensional virtual object within an actual image. According to one embodiment, if the location of the reference object within the actual image is located at the bottom of the actual image, the processor (220) can display a three-dimensional virtual object at a location a certain distance upward from the actual reference object location. Referring to FIG. 3, the reference object B (370) may be located at the bottom of the actual image. Accordingly, the processor (220) can display a three-dimensional virtual object (310) at a location a certain distance upward from the location of the reference object B (370). According to another embodiment, if the location of the reference object within the actual image is located at the top of the actual image, the processor (220) can display a three-dimensional virtual object at a location a certain distance downward from the actual reference object location. Referring to FIG. 3, the reference object A (330) can be positioned at the top of the actual image. Accordingly, the processor (220) can display a three-dimensional virtual object (310) at a position located a certain distance downward from the position of the reference object A (330). Through this, the processor (220) can ensure that both the reference object and the three-dimensional virtual object are displayed within the actual image.

[0061] A processor (220) according to one embodiment can move or rotate the position of a 3D virtual object in a real image based on user input. A processor (220) according to one embodiment can display a 3D virtual object (310) at a predetermined position (e.g., the center within the real image). A processor (220) according to one embodiment can determine the predetermined position by considering the position where the real object is displayed in the real image. For example, the processor (220) can determine the predetermined position so that the real object and the 3D virtual object do not overlap. A processor (220) according to one embodiment can move the position of the 3D virtual object (310) based on user input. Additionally, the processor (220) can rotate the 3D virtual object (310) based on user input. Through this, the user can simulate product placement by placing products (e.g., furniture, electronic products, etc.) corresponding to the 3D virtual object within the real image.

[0062] A processor (220) according to one embodiment can adjust the magnification of a camera included in an electronic device based on user input. For example, it may be zoomed in or zoomed out based on user input. Zooming in means increasing the magnification, and zooming out means decreasing the magnification. The processor (220) can change the display size of a 3D virtual object based on the adjusted magnification. When the magnification is adjusted, the display size of the actual object in the actual image may be changed. Since the display size of the actual object is changed, the display size of the 3D virtual object also needs to be changed. The processor (220) can know the ratio of the actual size of the actual object to the display size. Therefore, the processor (220) can change the display size of the 3D virtual object based on the ratio of the actual size of the actual object to the display size. Through this, even when the magnification is changed, the actual size of the 3D virtual object is changed, so the user can accurately compare the sizes of the actual object and the 3D virtual object.

[0063] A processor (220) according to one embodiment may display at least one of delivery type data, price discount data, review data, or purchase object corresponding to a 3D virtual object within a certain distance from the 3D virtual object. Referring to FIG. 3, delivery type data (381) may be displayed within a certain distance from the 3D virtual object (310). The delivery type is a type of delivery for a product and may include, for example, standard delivery, express delivery, fresh food delivery, overseas delivery, etc. Express delivery may be a delivery type where the average delivery time is shorter than that of Japanese delivery and delivery is completed within a guaranteed time (e.g., 1 day). Fresh food delivery may be a type where the product is delivered through packaging to maintain the freshness of the food for fresh food delivery. Overseas delivery may be a type where goods traded overseas are imported into the country and delivered to customers. Since the delivery type is a factor that directly influences a user's purchase of a product, by displaying the delivery type data (381) together with the 3D virtual object, the user can intuitively check the delivery type and make a purchase decision. In addition, by displaying price discount data (380), review data, etc., on the screen that influence the user's decision to purchase a product, the user can comprehensively grasp the information necessary for purchasing a product through a single screen (300). A processor (220) according to one embodiment may also display a purchase object on the screen (300). When a purchase object is selected, the user can purchase a product corresponding to a three-dimensional virtual object (310). Through this, the user's product purchase process can be simplified. Only some of the aforementioned data may be displayed on the screen (300), or other data may be additionally displayed on the screen (300), and the present disclosure is not limited thereto.

[0064] A processor (220) according to one embodiment may display not only the product corresponding to the 3D virtual object but also other products on the actual image (or screen). A processor (220) according to one embodiment may display product sales list data of a seller account that sells products. For example, there may be a seller account A that sells products corresponding to the 3D virtual object (310). And seller account A may also sell other products. The processor (220) may display other products sold by seller account A on the actual image. For example, referring to FIG. 3, the processor (220) may display other products (391, 392, 393) sold by seller account A in an area where other products (390) are displayed. Through this, the user can compare other products sold by a specific seller account with the product corresponding to the 3D virtual object. A processor (220) according to another embodiment may display recommended product data where the product and product category are the same. The recommended product data may be data determined based on user preference data. User preference data can be generated based on user product purchase log data. For example, if the product corresponding to the 3D virtual data is milk, the recommended product data may include other products included in the milk category. For example, if the user preference data includes a preference tendency for user A to purchase less than 1L of milk, other milks less than 1L may be included in the recommended product data. Referring to FIG. 3, the processor (220) may display products (391, 392, 393) included in the recommended product data in an area where other products (390) are displayed. Through this, the user can view other products within the actual video without having to move to another screen to view other products, thereby simplifying the online shopping process.

[0065] FIG. 4 is a drawing for illustrating a screen for creating a three-dimensional virtual object according to various embodiments of the present disclosure.

[0066] The screen (400) of FIG. 4 may include a three-dimensional virtual object display area (410). A generated three-dimensional virtual object may be displayed in the three-dimensional virtual object display area (410). The three-dimensional virtual object displayed in the three-dimensional virtual object display area (410) may be a three-dimensional virtual object generated based on user input for at least one of a scan object (430), an image upload object (450), or a size input object (470).

[0067] FIG. 5 is a diagram illustrating a method for acquiring LiDAR sensor data to generate a three-dimensional virtual object according to various embodiments of the present disclosure. Since a detailed description of FIG. 5 has been previously described in FIG. 3, a detailed description is omitted in this figure.

[0068] FIG. 6 is a diagram illustrating a method for acquiring multiple compositional images to generate a three-dimensional virtual object according to various embodiments of the present disclosure.

[0069] A processor (220) according to one embodiment can generate a three-dimensional virtual object based on a set of images obtained from two or more different angles. For example, to generate a three-dimensional virtual object for a product (610), the processor (220) can acquire a set of images (611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622) captured from two or more different angles through a camera. The set of images shown in FIG. 6 is merely an example, and the number of images included in the set of images may be fewer or more than that shown in FIG. 6. The processor (220) can generate a three-dimensional virtual object for a product (610) based on a set of images (611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622).

[0070] FIGS. 7a and 7b are drawings for illustrating a three-dimensional virtual object according to the actual size or shape of the three-dimensional virtual object determined based on user input according to various embodiments of the present disclosure. Since a detailed description of FIGS. 7a and 7b has been previously described in FIG. 3, a detailed description is omitted in these drawings.

[0071] FIG. 8 is a flowchart of the operation of an electronic device according to various embodiments of the present disclosure.

[0072] An electronic device (200) according to one embodiment can generate a three-dimensional virtual object (810) based on user input. The user input may be an input to at least one of a scan object (430), an image upload object (450), or a size input object (470). The electronic device (200) can generate a three-dimensional virtual object based on at least one of LiDAR sensor data, an image set, or the actual size (and / or shape) of the three-dimensional virtual object. Some or all of the operations during the process of generating the three-dimensional virtual object may be performed on a server.

[0073] An electronic device (200) according to one embodiment can composite a three-dimensional virtual object with a real image and display it as augmented reality (820). The display size of the three-dimensional virtual object can be determined based on the actual size of the three-dimensional virtual object and the actual size of the real object within the real image. Through this, within the real image, the user can intuitively compare the size of the real object and the size of the three-dimensional virtual object.

[0074] Although the above method has been described through specific embodiments, the above method can also be implemented as computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. Additionally, the computer-readable recording medium may be distributed across networked computer systems so that computer-readable code can be stored and executed in a distributed manner. Furthermore, functional programs, codes, and code segments for implementing the above embodiments can be easily inferred by programmers skilled in the art to which this disclosure pertains.

Claims

Claim 1 A method for displaying a three-dimensional virtual object performed by an electronic device, comprising: a step of generating a three-dimensional virtual object based on user input; a step of determining whether a reference object having a predetermined actual size and shape is included in an actual image; and a step of synthesizing the three-dimensional virtual object with the actual image to display it as augmented reality; wherein, if the reference object is included in the actual image, the step of determining the display size of the three-dimensional virtual object based on the actual size of the three-dimensional virtual object and the actual size of the reference object; and a step of determining the position to display the three-dimensional virtual object based on the position of the reference object. Claim 2 The method of claim 1, wherein the step of generating the three-dimensional virtual object comprises: generating at least one of an image set acquired from two or more different angles based on the user input, LiDAR sensor data, or the actual size of the three-dimensional virtual object; and generating the three-dimensional virtual object based on at least one of the image set, the LiDAR sensor data, or the actual size of the three-dimensional virtual object. Claim 3 A method according to claim 1, wherein the step of synthesizing the three-dimensional virtual object with a real image to display it as augmented reality comprises the step of displaying the texture of the three-dimensional virtual object on the surface of the three-dimensional virtual object. Claim 4 The method according to claim 1, wherein the step of generating the three-dimensional virtual object comprises the step of determining at least one of the actual size or shape of the three-dimensional virtual object based on user input. Claim 5 The method of claim 1, wherein the step of generating the three-dimensional virtual object comprises: transmitting to a server at least one of an image set acquired in two or more different angles generated based on the user input, LiDAR sensor data, or the actual size of the three-dimensional virtual object; and receiving from the server a three-dimensional virtual object generated based on at least one of the image set, LiDAR sensor data, or the actual size of the three-dimensional virtual object. Claim 6 A method according to claim 1, wherein the step of synthesizing the three-dimensional virtual object with a real image to display it as augmented reality includes the step of moving or rotating the position of the three-dimensional virtual object in the real image based on user input. Claim 7 A method according to claim 1, wherein the step of synthesizing the three-dimensional virtual object with a real image to display it as augmented reality comprises: a step of adjusting the magnification of a camera included in the electronic device based on user input; and a step of changing the display size of the three-dimensional virtual object based on the adjusted magnification. Claim 8 A method according to claim 1, further comprising the step of displaying at least one of delivery type data of a product corresponding to the 3D virtual object, price discount data of the product, review data of the product, or a purchase object within a certain distance from the 3D virtual object. Claim 9 In paragraph 8, the above delivery type data is data related to the delivery form, wherein the delivery form is at least one of general delivery, express delivery, fresh food delivery, or overseas delivery. Claim 10 A method according to claim 1, further comprising: a step of displaying product sales list data of a seller account selling products corresponding to the 3D virtual object on the actual image; or a step of displaying recommended product data in the actual image in which the product and product category are the same. Claim 11 An electronic device comprising: a processor; and a memory in which instructions configured to be executed by said processor are stored, said processor being configured to execute a method according to any one of claims 1 to 10. Claim 12 A non-transient computer-readable recording medium having a computer program to be executed by a processor, wherein the computer program is configured to cause the processor to execute a method according to any one of claims 1 to 10.

Images