Electronic device and three-dimensional model construction method using same

Abstract

The electronic device comprises a processor and a three-dimensional model builder. The processor is used for generating a depth map of each two-dimensional original image according to a plurality of two-dimensional original images, and responding to a user instruction to generate a plurality of two-dimensional updated images according to the two-dimensional original images, the depth maps and the user instruction. The three-dimensional model constructor is used for constructing a three-dimensional model according to the two-dimensional update graphs.

Description

Technical Field

[0001] The embodiments of the present invention relate to an electronic device and a three-dimensional model construction method using the same. Background Art

[0002] If consumers want to learn about products, they can go to physical stores to play with them or look at online photos or videos. However, modern people are busy and may not have time to go to physical stores to play with them. Or, products are only sold overseas and cannot be obtained, so they can only be viewed online. However, when looking at online photos or videos, they may not have a precise imagination of the material, size, and specifications of the product, which may lead to returns. Or, during an online press conference, the product introduction is only a few minutes long. How to discuss the details after the introduction and quickly understand the product in a short period of time and solve the misunderstanding of the product is a problem worth discussing. Summary of the invention

[0003] The embodiments of the present invention relate to an electronic device and a three-dimensional model construction method using the same, which can improve the above-mentioned conventional problems.

[0004] According to one embodiment of the present invention, an electronic device is provided. The electronic device includes a processor and a 3D model builder. The processor is used to generate a depth map of each 2D original image based on a plurality of 2D original images, and, in response to a user instruction, generate a plurality of 2D update images based on the 2D original images, the depth maps and the user instruction. The 3D model builder is used to construct a 3D model based on the 2D update images.

[0005] According to another embodiment of the present invention, a 3D model construction method is provided. The 3D model construction method includes the following steps: generating a depth map of each 2D original image based on a plurality of 2D original images; generating a plurality of 2D update images based on the 2D original images, the depth maps and the user instructions in response to a user instruction; and constructing a 3D model based on the 2D update images.

[0006] In order to better understand the above and other aspects of the present invention, the following embodiments are specifically described in detail with reference to the accompanying drawings as follows: BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 FIG. 1 is a functional block diagram of an electronic device 100 according to an embodiment of the present invention.

[0008] Figure 2 Draw Figure 1 The two-dimensional original graph P 2D Schematic diagram of .

[0009] Figure 3 Draw Figure 2 The two-dimensional original graph P 2DThe depth map P DF Schematic diagram of.

[0010] Figure 4 Draw Figure 2 The two-dimensional original graph P 2D The two-dimensional update graph P' 2D Schematic diagram of.

[0011] Figure 5 Draw Figure 2 The two-dimensional original graph P 2D Another two-dimensional update graph P' 2D Schematic diagram of.

[0012] Figure 6 Draw Figure 1 Flow chart of a method for constructing a three-dimensional model of the electronic device 100.

[0013] in,

[0014] 100: electronic device;

[0015] 110: processor;

[0016] 120: 3D model builder;

[0017] 130: 3D model renderer;

[0018] 140: command receiving interface;

[0019] M 3D :3D model;

[0020] P 2D : 2D original image;

[0021] P DF :Depth map;

[0022] P' 2D :2D update graph;

[0023] P11, P21: drum;

[0024] P12, P22: cymbals;

[0025] S110~S130: steps;

[0026] T1: User command. DETAILED DESCRIPTION

[0027] Please refer to Figure 1 to Figure 5 , Figure 1 FIG. 1 is a functional block diagram of an electronic device 100 according to an embodiment of the present invention. Figure 2 Draw Figure 1 The two-dimensional original graph P 2D Schematic diagram of Figure 3 Draw Figure 2 The two-dimensional original graph P 2D The depth map P DF Schematic diagram, Figure 4 Draw Figure 2 The two-dimensional original graph P 2D The two-dimensional update graph P' 2D Schematic diagram, and Figure 5 Draw Figure 2 The two-dimensional original graph P 2D Another two-dimensional update graph P' 2D Schematic diagram of.

[0028] like Figure 1 As shown, the electronic device 100 is, for example, a desktop computer, a notebook computer, a tablet computer, a smart phone, etc. The electronic device 100 includes a processor 110, a 3D model builder 120, a 3D model renderer 130, and a command receiving interface 140. The 3D model builder 120 and the 3D model renderer 130 are, for example, physical circuits or software supporting the OpenXR architecture. When the 3D model builder 120 and the 3D model renderer 130 are physical circuits, the 3D model builder 120 and the 3D model renderer 130 can be formed using semiconductor processes.

[0029] like Figure 1 As shown, the processor 110 is used to: 2D , generate each two-dimensional original graph P 2D The depth map P DF ; and, in response to a user instruction T1, according to these two-dimensional original images P 2D , these depth maps P DF and user instruction T1, generate several two-dimensional update graphs P' 2D The three-dimensional model builder 120 is used to update the two-dimensional graphs P′ according to these two-dimensional graphs. 2D , construct a three-dimensional model M 3D In this way, the user can input the corresponding user command T1 according to the demand for the three-dimensional model to change the three-dimensional model M in real time. 3D In this way, the three-dimensional model M 3D Replacing physical products helps to change the 3D model M in real time online (e.g., through wired or wireless communication, etc.) 3D And it is presented to at least one electronic device (or participant) online in real time (for example, through wired or wireless communication, etc.).

[0030] These two-dimensional images are, for example, a plurality of images captured by a camera surrounding an object or a scene, or images captured by screen capture technology. 2DFor example, it is a plurality of static pictures of an animation or a static picture. The animation is captured by a camera (not shown) of the electronic device 100 or a camera (not shown) of another electronic device. When the animation or static picture is captured by a camera (not shown) of another electronic device, the animation or static picture can be transmitted to the electronic device 100 via wired or wireless communication technology. 2D Any image content can be used, as long as it is a two-dimensional image that can be electronically transmitted, regardless of the content, it is the so-called two-dimensional original image P in the embodiment of the present invention. 2D .

[0031] like Figure 1 As shown, the three-dimensional model builder 120 can analyze each two-dimensional image (two-dimensional original image P 2D Or the two-dimensional update graph P' 2D ), with at least one feature point matching each two-dimensional image marker. 3D The construction of is achieved by matching the matching feature points of each two-dimensional image. In addition, as long as the three-dimensional model M can be constructed 3D That is, the embodiments of the present invention do not limit the construction technology.

[0032] In one embodiment, the command receiving interface 140 is, for example, a user interface, and the 3D model renderer 130 is, for example, a display that can display the aforementioned user interface. In another embodiment, the 3D model renderer 130 is, for example, a touch display that can provide both a display function and a command input function. In addition, the 3D model renderer 130 can display the 3D model M 3D In response to a user operation instruction (not shown), the 3D model builder 120 may rotate and / or move the 3D model renderer 130, and the 3D model renderer 130 may display the 3D model M to be changed in real time. 3D In addition, the aforementioned user operation command is inputted through the command receiving interface 140 , or inputted through hardware such as a mouse (not shown), a keyboard (not shown) or other wired / wireless external devices (such as a tablet computer, a smart phone) of the electronic device 100 .

[0033] 2D original image P 2D is an original image, not an image processed by the processor 110 (the processed image is different from the two-dimensional original image P 2D Compared with the two-dimensional updated image obtained based on the processed image, the processor 110 of the embodiment of the present invention is based on the two-dimensional original image P 2D Get the two-dimensional update graph P' 2D , so the two-dimensional update graph P' 2D The accuracy is higher (the distortion is smaller).

[0034] In one embodiment, the two-dimensional update map P' 2D The number of two-dimensional original graph P 2D Specifically, the processor 110 may, according to the user instruction T1, 2D Based on this, the corresponding two-dimensional update graph P' is generated. 2D In addition, the depth map P DF The number of two-dimensional original graph P 2D In other words, the processor 110 can generate each two-dimensional original image P 2D Depth map P DF .

[0035] like Figure 2 As shown, the two-dimensional original graph P 2D The user instruction T1, for example, indicates changing the two-dimensional original image P 2D The processor 110 is, for example, an artificial intelligence (AI) processor. In addition, a component judgment model can be trained using machine learning technology or artificial intelligence learning technology. After the processor 110 loads this component judgment model, it can analyze and judge the two-dimensional original image P 2D The aforementioned machine learning technology or artificial intelligence learning technology is, for example, any suitable or known technology, such as ControlNet, and the embodiment of the present invention is not limited thereto.

[0036] like Figure 1 As shown, the command receiving interface 140 includes, for example, a user interface, which can display an input field, and the input field is used to receive the user command T1, such as text input by the user through a keyboard. In another embodiment, the command receiving interface 140 includes, for example, a voice receiver, such as a microphone, which is used to receive the user command T1, such as voice input by the user. In addition, the command receiving interface 140 can also receive corresponding wireless radio frequency signals through other wired or wireless external devices (such as tablets, smart phones, etc.), and correspondingly receive the user command T1. The command receiving interface 140 transmits the user command T1 to the processor 140, and the processor 110 can analyze the user command T1 to obtain the content of the user command T1.

[0037] like Figure 3 As shown, the depth map P DF The processor 110 is based on the two-dimensional original image P 2D Depth map P DF Contains at least one element, whose shape and relative position correspond to the two-dimensional original image P 2DFor example, the depth map P DF The drum P21 and cymbal P22 are included, and their shapes and relative positions correspond to the two-dimensional original image P 2D The drum P11 and cymbal P12. Depth map P DF The drum P21 and cymbal P22 have a two-dimensional original image P 2D The outer contours of the drum P11 and cymbal P12 are shown, but there are no detailed structures surrounded by the outer contours. DF The distance of the element from a reference point (e.g., the photographer's position, the camera coordinate point, etc.) is expressed in grayscale values. DF For any of these depth maps, the processor 110 sets different grayscale values ​​for each component according to the different distances between the multiple components and the reference points.

[0038] For example, the closer the component is to the reference point, the smaller (brighter) or larger (darker) the grayscale value is, and vice versa. Figure 3 As shown, the drum P21' is closer to the reference point than the drum P21", so the grayscale value of the drum P21' is smaller than that of the drum P21" (brighter, represented by a sparser cross section); or, the cymbal P22' is closer to the reference point than the cymbal P22", so the grayscale value of the cymbal P22' is smaller than that of the cymbal P22" (brighter, represented by a sparser cross section). Depth map P DF Provide 2D original image P 2D The relative position relationship of several components can generate a two-dimensional update graph P' 2D More accurate.

[0039] like Figure 4 As shown, when the user instruction T1 is "change the color of the drum", the processor 110 responds to the user instruction T1 according to the two-dimensional original image P 2D , depth map P DF and user instruction T1, generate several two-dimensional update graphs P' 2D . Figure 4 Present one of the two-dimensional update graphs P' 2D The color of drum P11 (indicated by denser hatching) is Figure 2 The color of drum P11 (indicated by the thinner hatching) is different. Then, processor 110 updates the two-dimensional graph P' according to these two-dimensional graphs. 2D , construct a new three-dimensional model M 3D .

[0040] like Figure 5 As shown, when the user instruction T1 is "add a drum", the processor 110 responds to the user instruction T1 and according to the two-dimensional original image P 2D , depth map P DFand user instruction T1, generate several two-dimensional update graphs P' 2D . Figure 5 Present one of the two-dimensional update graphs P' 2D , which increases a drum P11'. Then, the processor 110 updates the two-dimensional graph P' 2D , construct a new three-dimensional model M 3D .

[0041] The user command T1 can also be a two-dimensional original image P 2D Instructions on the shape, position or other parameters of a component.

[0042] Please refer to Figure 6 , which shows Figure 1 Flow chart of a method for constructing a three-dimensional model of the electronic device 100.

[0043] In step S110, the processor 110 generates a plurality of two-dimensional original images P 2D , generate each two-dimensional original graph P 2D A depth map P DF .

[0044] In step S120, the processor 110 responds to the user instruction T1 and generates the two-dimensional original images P 2D , these depth maps P DF and user instruction T1, generate several two-dimensional update graphs P' 2D .

[0045] In step S130, the processor 110 updates the two-dimensional graphs P′ according to the two-dimensional graphs P′. 2D , construct a three-dimensional model M 3D .

[0046] In summary, the embodiments of the present invention provide an electronic device and a 3D model construction method using the same. Based on a plurality of 2D original images, the parameters of each 2D original image are changed through user instructions to construct a 3D model corresponding to the user instructions.

[0047] In summary, although the present invention has been disclosed as above by the embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various changes and modifications without departing from the spirit and scope of the present invention, and the changes and modifications are not limited by the embodiments of the present invention, and are still within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. An electronic device, comprising: A processor for: Generating a depth map of each of the two-dimensional original images according to a plurality of two-dimensional original images; In response to a user instruction, generating a plurality of two-dimensional updated images according to the two-dimensional original images, the depth images and the user instruction; A 3D model builder for: A three-dimensional model is constructed according to the two-dimensional updated graphs.

2. The electronic device according to claim 1, characterized in that Each of the two-dimensional original images includes an element, and the user instruction indicates changing the color, quantity, shape or position of the element.

3. The electronic device according to claim 1, characterized in that Also includes: A user interface is used to display an input field, and the input field is used to accept the user's instruction.

4. The electronic device according to claim 1, characterized in that: Also includes: A voice receiver is used to receive the user's instructions and transmit them to the processor.

5. The electronic device according to claim 1, characterized in that: Each of the depth maps includes a plurality of elements, and the grayscale value of each of the elements has a different value corresponding to the distance from a reference point.

6. A three-dimensional model construction method, comprising: Generating a depth map of each of the two-dimensional original images according to a plurality of two-dimensional original images; In response to a user instruction, generating a plurality of two-dimensional updated images according to the two-dimensional original images, the depth images and the user instruction; and A three-dimensional model is constructed according to the two-dimensional updated graphs.

7. The three-dimensional model construction method according to claim 6, characterized in that: Each of the two-dimensional original images includes an element, and the three-dimensional model construction method further includes: The color, quantity, shape or position of the component is changed according to the user's instruction.

8. The three-dimensional model construction method according to claim 6, characterized in that: Also includes: An input field is displayed through a user interface, wherein the input field is used to accept the user's instruction.

9. The three-dimensional model construction method according to claim 6, characterized in that: Also includes: The user command is received through a voice receiver and transmitted to a processor.

10. The three-dimensional model construction method according to claim 6, characterized in that: Each of the depth maps includes a plurality of elements; the steps of generating the depth map of each of the two-dimensional original images according to the two-dimensional original images include: For any of the depth maps, different grayscale values ​​are set for each of the components according to different distances of the components corresponding to a reference point.

Images