Method and system for generating three-dimensional digital model

Abstract

Methods and systems for generating a three-dimensional digital model are described. The method includes: receiving, by at least one computing device, scan data obtained by scanning, by a scanner disposed outside the package, a physical object contained inside the package; generating, by the at least one computing device, a three-dimensional digital model of the physical object based on the received scan data; identifying, by the at least one computing device, one or more points of a portion of the physical object based on the received scan data, the one or more points being movable via at least one of a bending movement and an articulating movement; modifying, by the at least one computing device, the three-dimensional digital model, the modifying including relocating a portion of the three-dimensional digital model of the physical object based on the identified one or more movable points; and outputting, by the at least one computing device, the three-dimensional digital model to support user interaction via the user interface, the user interaction involving changing the viewing angle with respect to the three-dimensional digital model.

Description

[0001] This application is a divisional application of the Chinese invention patent application with Chinese application number 201980036853.1, application date May 20, 2019, and invention name “Colored Three-Dimensional Digital Model Generation”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Application No. 15 / 996,289, filed on June 1, 2018, entitled “GENERATION OF SHADED 3D DIGITAL MODELS,” the entire disclosure of which is incorporated herein by reference. Background Art

[0004] Three-dimensional digital models have been developed to increase the ways in which users can interact with digital content via computing devices. For example, a three-dimensional digital model can be configured to support user interaction to change the perspective at which the model is viewed by a computing device. As a result, user interaction with the three-dimensional digital model via a computing device can be mimicked in the way that user interaction would occur with respect to an actual physical object. This serves to increase the richness of user interaction over two-dimensional digital images, as opposed to navigating between different views supported by different two-dimensional digital images, because the user can effectively navigate between these perspectives in a non-modal manner.

[0005] However, conventional techniques for creating three-dimensional digital models are complex, computationally intensive, and typically involve a significant amount of user interaction and expertise, thus making these conventional techniques unavailable to the average user. Furthermore, in some cases, the formation of the model may have a detrimental effect on the user's intent to create the model. For example, suppose a user wishes to sell a vintage toy in its original packaging. Including the toy in the packaging typically increases the monetary value of the entire toy. Therefore, using conventional modeling techniques that involve removing the toy from the packaging may adversely affect that monetary value and run counter to the user's intent to sell the toy. Therefore, conventional modeling techniques may not be available to the user, such as for providing a three-dimensional digital model as part of digital content that provides the toy for sale. Summary of the invention

[0006] Colored three-dimensional digital model generation techniques and systems are described. In one example, a scanning system uses the following scanning techniques: a physical object is scanned while it is placed in a package to form a three-dimensional digital model. A model coloring system is used to color the three-dimensional digital model. To this end, a two-dimensional digital image of the same or similar physical object is taken. The color of the digital image can be applied to the model using various techniques. In one example, features of the model are matched to the image. This is then used to align the viewing perspective relative to the model with the viewing perspective of the object within the digital image, for example, to find which "perspective" of the model corresponds to the image. The color is then applied from the digital image to the model, for example, from the pixels of the image to the corresponding points in the model. In one example, a distortion technique is used to stretch and compress corresponding portions of the digital image to provide the color. Other examples include stereo matching of an uncolored portion of an initial version of the model with a colored portion of the initial version, patch matching techniques, and the like.

[0007] This summary introduces a series of concepts in a simplified form that will be further described in the following detailed description. As such, this summary is neither intended to identify essential features of the claimed subject matter nor to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawings will be provided by the Office upon request and payment of the necessary fee. The detailed description is described with reference to the drawings. Entities represented in the drawings may refer to one or more entities, and thus, singular or plural entities may be referred to interchangeably in the discussion.

[0009] Figure 1 is an illustration of an environment in an example implementation that is operable to employ the shaded three-dimensional digital model generation techniques described herein.

[0010] Figure 2 Describes in more detail Figure 1 Operation of the scanning system and model coloring system.

[0011] Figure 3 In more detail Figure 2 The operation of the scanning system is described as generating a three-dimensional digital model by scanning a physical object through a packaging.

[0012] Figure 4 The operation of the model shading system is described as Figure 3 Feature matching is used between the three-dimensional digital model generated in the process and the two-dimensional digital image.

[0013] Figure 5 The operation of the model coloring system to color the three-dimensional digital model is described.

[0014] Figure 6 Describes the generation of digital content as including Figure 5 3D digital model.

[0015] Fig. 7A Depicted Figure 6 Examples of digital content including three-dimensional digital models.

[0016] Figure 7B Another example implementation of outputting digital content in virtual reality scenes and augmented reality scenes is depicted.

[0017] Figure 8 is a flow chart depicting a procedure in an example implementation of shaded three-dimensional digital model generation.

[0018] Fig. 9 An example system is shown that includes various components of an example device that can be implemented as a reference Figures 1 to 8 Any type of computing device is described and / or utilized to implement implementations of the techniques described herein. DETAILED DESCRIPTION

[0019] Overview

[0020] Conventional techniques for creating 3D digital models are complex, computationally intensive, and often involve significant user interaction and expertise. This makes these conventional techniques unavailable to the average user. Furthermore, in some cases, the formation of a 3D digital model may have a detrimental effect on the user's intent to create the model, for example, it may need to be removed from its packaging, which may adversely affect the value of the physical object being modeled.

[0021] Thus, colored three-dimensional digital model generation techniques and systems are described. In one example, a scanning system employs a scanning technique that scans a physical object while it is placed within a package. Examples of such scanning techniques include X-ray techniques that generate scan data as a series of "slices" that are then formed together to directly generate a three-dimensional digital model of the physical object. Indirect examples of using the scan data to identify a previously generated three-dimensional digital model are also contemplated.

[0022] After the three-dimensional digital model is obtained, the three-dimensional digital model is colored using a model coloring system. To this end, a two-dimensional digital image is used. In one example, the two-dimensional digital image is provided to the system by a user who initiates the model generation process. For example, the digital image can capture a view of the same or similar physical object as the subject of the model. In another example, the two-dimensional digital image is obtained via a search, such as by shape matching, based at least in part on a search query generated from the three-dimensional digital model.

[0023] After obtaining the two-dimensional digital image, a variety of techniques can be used to apply color from the digital image to the model. In one example, features of the model are matched to the image. This is then used to align the viewing perspective relative to the model with the viewing perspective of the object within the digital image, for example, to find which "perspective" of the model corresponds to the image. Color is then applied from the digital image to the model, for example from the pixels of the image to the corresponding points in the model.

[0024] In some cases, some parts of the model are not visible in the digital image and are therefore not colored in the initial version of the model. Therefore, techniques can also be used to color these parts. In one example, deformation techniques are used to stretch and compress corresponding parts of the digital image to provide the color. Other examples include stereo matching the uncolored parts of the initial version of the model with the colored parts of the initial version, block matching techniques, etc. In this way, even though the digital image may capture a single perspective of the object, the color generation system can still use that perspective to provide color for the "invisible" parts of the model. Other examples are also contemplated, such as linear blend skinning.

[0025] The painted three-dimensional digital model of the physical object can then be used to support various functions. For example, a seller may wish to offer the physical object for sale within a package. To this end, the seller interacts with a service provider system to initiate the generation of digital content indicating that the physical object is available for sale, a two-dimensional digital image of the same or similar physical object, a purchase price, and the digital content has a function that can be selected by a buyer to initiate the purchase of the physical object.

[0026] The seller then delivers the physical object in the package to a fulfillment center. The fulfillment center includes a scanning system including a three-dimensional scanning device that can be used to generate a three-dimensional digital model. The three-dimensional scanning device is configured to generate scanning data for forming a model without removing the physical object from the package, thereby saving manpower and operating resources and preserving packaging that may be needed.

[0027] The three-dimensional digital model is then colored using a model coloring system at the fulfillment center based at least in part on the two-dimensional digital image provided by the seller. In this way, the colored three-dimensional digital model can be easily generated to be included as part of digital content (e.g., an offer to sell a physical object). As a result, the digital content can include the colored three-dimensional digital model without removing the physical object from the packaging, which is not possible using traditional techniques. Other examples are also contemplated, further discussion of which can be found in the following sections and shown in the corresponding figures.

[0028] In the following discussion, an example environment in which the techniques described herein may be employed is first described. Example procedures are then described, and the example procedures may be performed in the example environment as well as other environments. Therefore, the execution of the example procedures is not limited to the example environment, and the example environment is not limited to the execution of the example procedures.

[0029] Example Environment

[0030] Figure 1 1 is an illustration of a digital media environment 100 in an example implementation that is operable to employ the model generation techniques described herein. The illustrated environment 100 includes a service provider system 102 and a user's client device 104 that are communicatively coupled to each other via a network 106. For example, the client device 104 may include a communication module 108 that is configured to access functionality of the service provider system 102 via the network 106. The communication module 108 may be configured in various ways, such as a browser, a mobile application, etc. The computing devices that implement the service provider system 102 and the client device 104 may also be configured in various ways.

[0031] For example, the computing device may be configured as a desktop computer, a laptop computer, a mobile device (e.g., assuming a handheld configuration such as a tablet or mobile phone as shown for client device 104), etc. Thus, the computing device may range from a full-resource device with a large amount of memory and processor resources (e.g., a personal computer, a game console) to a low-resource device with limited memory and / or processing resources (e.g., a mobile device). In addition, the computing device may represent a plurality of different devices, such as those shown for service provider system 102 and in combination with Fig. 9 Describes the multiple servers used by an enterprise to perform operations "in the cloud."

[0032] The service provider system 102 is configured in this example to process the physical object 110 and the two-dimensional digital image 114 in the packaging 112 to form a colored three-dimensional digital model 116. To this end, a scanning system 118 is used to generate a three-dimensional digital model 120, for example by using X-ray technology. Although the scanning system 118 is implemented by the service provider system 102 in this example, the scanning system 118 can also be configured as a "stand alone" unit for direct use by a user. The three-dimensional digital model 120 is colored using the two-dimensional digital image 114 using a model coloring system 122, thereby forming a colored three-dimensional digital model 116. In this way, a model can be formed based on the physical object 110 without removing it from the packaging 112, thereby improving efficiency because the physical object 110 does not need to be repackaged, and it can also be used to preserve potentially valuable packaging.

[0033] In general, the functions, features, and concepts described with respect to the examples above and below can be employed in the context of the example processes described in this section. In addition, the functions, features, and concepts described with respect to the different figures and examples herein can be interchangeable with each other and are not limited to implementation in the context of a particular figure or process. In addition, the boxes associated with the different representative processes and corresponding figures herein can be applied together and / or combined in different ways. Therefore, the various functions, features, and concepts described with respect to the different example environments, devices, components, figures, and processes herein can be used in any suitable combination and are not limited to the specific combinations represented by the examples listed in this specification.

[0034] Colored 3D digital model generation

[0035] Figure 2 The system 200 is depicted, showing in more detail Figure 1 The scanning system 118 and the model rendering system 122 are operated. Figure 3 Figure 7 depicts Figure 2

[0046] Further examples of the operation of the scanning system and the model coloring system. Figure 8 Depicted is a process 800 in an example implementation in which a three-dimensional digital model formed by scanning a physical object through packaging is rendered using a two-dimensional digital image.

[0036] The following discussion describes techniques that can be implemented using the previously described systems and devices. Various aspects of the process can be implemented in hardware, firmware, software, or a combination thereof. The process is shown as a set of blocks that specify operations performed by one or more devices and is not necessarily limited to the order shown in which the operations are performed by the various blocks. In portions of the following discussion, reference will be made interchangeably to Figures 2 to 8 .

[0037] First, the service provider system 102 receives the physical object 110 contained in the packaging 112. Figure 1 As shown, packaging 112 may partially obstruct the ability to view physical object 110, or even Figure 2 As shown, the ability to view object 110 is completely obstructed. Therefore, 3D scanning device 202 including scan control module 204 is configured to generate scan data 206 that passes through packaging 112 and describes physical object 110 .

[0038] For example, Figure 3 As shown in the example implementation 300 of , the scan data 206 can capture various views / perspectives of the physical object 110 through the packaging 112. For example, the 3D scanning device 202 can be configured to generate the scan data 206 as a collection of two-dimensional X-ray images. The scan data 206 is able to capture views of the physical object 110 through the packaging 112 because the material density of the physical object 110 is greater than the material density of the packaging. In one example, the density of the packaging material (e.g., vinyl, Styrofoam, paper, etc.) is estimated and then can be removed from the scan data 206 automatically by the system and without user intervention. Other examples are also contemplated, including computed tomography, magnetic resonance imaging, etc.

[0039] Then, the three-dimensional digital model generation module 208 receives the scan data 206 obtained by scanning the physical object 110 in the packaging 112 (block 802). The three-dimensional digital model generation module 208 is configured to generate a three-dimensional digital model 120 of the physical object based on the received scan data (block 804). For example, various techniques (e.g., isosurface extraction algorithms) can be used to form the three-dimensional digital model 120 as a discrete 3D volume representation of the physical object 110 based on the scan data 206 captured from different perspectives.

[0040] Thus, at this point, the scanning system 118 outputs a three-dimensional digital model 120 of the physical object 110. Because the three-dimensional digital model 120 is formed from the physical object 110 through the packaging 112, the model is not colored. Therefore, the three-dimensional digital model 120 is then colored using a model coloring system 122. This can be performed in a variety of ways.

[0041] exist Figure 2In the example shown, the model rendering system 122 receives the two-dimensional digital image 114 of the same or similar physical object 110 and the three-dimensional model 120 from the scanning system 118 (block 806). For example, a user who provides the physical object 110 and the packaging 112 to the service provider system 102 may also provide the two-dimensional digital image 114. In another example, the two-dimensional digital image 114 is obtained by the model rendering system 122 based on an image search, such as finding the two-dimensional digital image 114 based on the shape of the three-dimensional digital model 120.

[0042] Then, the feature matching module 210 is used to detect features of the two-dimensional digital image 114 corresponding to the features of the three-dimensional digital model 120 (block 808), and the features of the two-dimensional digital image 114 are output as mapped feature data 212. Figure 4 As shown in the example implementation 400 of , the feature matching module 210 can generate the mapping feature data 212 in various ways. For example, the feature matching module 210 can form an abstraction of the information in the two-dimensional digital image 114 and the three-dimensional digital model 120 to make a local decision (e.g., at the pixel level) about whether to include a feature at that location, and if the feature is included at that location, whether the feature can also be found in the model, and vice versa. This can be performed using edge detection, corners and points of interest, through blob detection, etc. An edge is a point that forms a boundary between two image regions, such as through gradient detection, such as the boundary between different parts of the toy robot shown. Corners and points of interest refer to point-like features in the image that have local two-dimensional structure, such as buttons on the chest of a toy robot. In contrast to the corners in the previous example, blobs provide a description of the structure included in the image as a region (e.g., the feet of a toy robot).

[0043] This can also be used to modify the position of the scanned image and / or the three-dimensional digital model 120. For example, points in a physical object that can move (e.g., bend, rotate, articulate, etc.) can be identified. By identifying the relevant points of these different parts (e.g., through object recognition), the three-dimensional digital model 120 can be mapped to a two-dimensional digital image, even in different poses, and vice versa. For example, the robot shown can be included in a package and the arms and legs are placed straight, and then it is mapped to the three-dimensional digital model 120 with the arms extended. This can include the ability of the system and / or user to modify the model after it is formed (e.g., reposition an accessory). This can also include the ability to include accessories as part of the model (e.g., so that a sword included in the package is held in the robot's hand). This functionality can be implemented manually, for example, based on an image repository of other digital images and / or models, through user interaction using a user interface, and / or automatically by the system without user intervention.

[0044] Colors are then generated for the three-dimensional digital model based on the detected features by the model colorization system 122 (block 810) to generate the colored three-dimensional digital model 116. This may be performed in a variety of ways. Figure 5 An example 500 of applying color from a two-dimensional digital image 114 to a three-dimensional digital model 120 is depicted. First, the color generation module first generates an initial shaded three-dimensional digital model 502 by aligning the perspective of the three-dimensional digital model 120 with the perspective of the physical object in the two-dimensional digital image 114. Then, the mapping feature data 212 is used to map color from the digital image to the model for portions of the physical object visible in the two-dimensional digital image 114.

[0045] As shown, this may result in parts of the initially colored three-dimensional digital model 502 lacking color. In the two-dimensional digital image 114 used as the color basis, the right earphone 504, arm joints 506, 508, and the right side 510 of the right leg of the toy robot may not be visible. This may be due to occlusion by other parts of the model, perspective, etc. Therefore, the color generation module 214 utilizes the color mixing module 512 to color these missing parts, which can be performed in various ways.

[0046] In a first example, a perspective correction module 514 is employed to stretch 516 and compress 518 corresponding portions of color obtained from a two-dimensional digital image to apply color to portions that are not colored due to a change in perspective relative to the model.

[0047] In another example, feature coloring module 520 is employed to color uncolored features in initial colored three-dimensional digital model 502 based on similarity to colored features in the model, for example, coloring right earphone 504 based on similarity to left earphone 522 of the toy robot. In the example shown, feature similarity can also utilize stereo correspondence, for example, coloring the right side of the toy robot based on bilateral similarity of the right side of the toy robot to the colored left side.

[0048] In another example, a block matching module 524 is used. In block matching, the nearest neighbors of the offsets of the blocks for image similarity are used as a measure of distance to "fill holes" in the missing parts of the initial colored three-dimensional digital model 502. This can be based on blocks taken from the colored parts of the initial colored three-dimensional digital model 502 and / or the two-dimensional digital image. Therefore, the color generation module 214 can use a variety of techniques to generate the colors used to color the three-dimensional digital model 116.

[0049] The rendered three-dimensional digital model 116 is output by the model rendering system 122 to support user interaction via a user interface that involves changing viewing angles relative to the rendered three-dimensional digital model 116 (block 812). The rendered three-dimensional digital model 116 may thus support richer user interactions than conventional modal techniques that require user navigation through different two-dimensional digital images to view representations of the physical object 110 from different angles.

[0050] Figure 6 An example implementation 600 of incorporating a rendered three-dimensional digital model 116 as part of digital content is depicted. For example, a user may wish to offer a physical object 110 for sale. To do so, the seller interacts with the service provider system 102 over the network 106 using a client device 104 to initiate the generation of digital content 602, which is shown as being stored in a storage device 604. In this example, the digital content 602 is a web page indicating that the physical object 110 is available for sale, a two-dimensional digital image 114 of the same or similar physical object, a purchase price, and includes functionality that a buyer can select in order to initiate the purchase of the physical object 110.

[0051] In this example, a user delivers a physical object 110 in a package 112 to a fulfillment center of a service provider system 102. The fulfillment center includes a scanning system 118, which includes a three-dimensional scanning device 202 that can be used to generate a three-dimensional digital model. The three-dimensional scanning device 202 is configured to generate scan data 206. The three-dimensional data model generation module 208 uses the scan data 206 to form a model without removing the physical object 110 from the package 112, thereby saving manpower and operating resources and preserving packaging that may be required.

[0052] The model rendering system 122 is then used to render the three-dimensional digital model 120 based at least in part on the user-provided two-dimensional digital image 114. In this manner, the rendered three-dimensional digital model 116 can be easily generated to be included as part of the digital content 602 (e.g., an offer to sell a physical object). As a result, the digital content 602 can include the rendered three-dimensional digital model without removing the physical object from the packaging, which is not possible using conventional techniques.

[0053] For example, the service provider system 102 may include a digital content control module 606 configured to manage the output of the digital content 602. This may include using a search module 608 that is configured to perform a search for the digital content 602, whether based on keywords or images, such as to find prospective items to purchase. The digital content control module 606 may also include a commerce module 610 that represents functionality for processing payment information to initiate and complete the purchase of the physical object 110.

[0054] Fig. 7A An example implementation 700 of output of digital content 602 is depicted, the example implementation 700 including a rendered three-dimensional digital model 116. In the example shown, the client device 702 outputs a user interface including the digital content 602, such as by executing a browser, a mobile application, etc. The digital content 602 includes the rendered three-dimensional digital model 116, which supports user interaction via gestures (e.g., selecting and dragging) to change the perspective (i.e., viewing angle) of the modeled physical object. The digital content 602 also includes functionality 704 for initiating a purchase of the physical object. In this way, the functionality and richness of the user's interaction with the digital content 602 can be increased to overcome the limitations of conventional techniques.

[0055] Figure 7BAnother example implementation 750 of digital content output in a virtual reality scene and an augmented reality scene is depicted. In a first example 752, the rendered three-dimensional digital model 116 is output as virtual reality digital content as part of a virtual store. In this way, the user can navigate through the store in a natural and intuitive manner and interact with the rendered three-dimensional digital model 116. In a second example 752, the rendered three-dimensional digital model 116 is output as augmented reality content that can be viewed as placed on a surface in the user's environment through the client device 104. Various other examples are also contemplated.

[0056] Now return again Figure 6 As previously described, the digital content control module 606 is configured to maintain the rendered three-dimensional digital models 116. In one example, this includes generating a library of models that can be used without rescanning the physical object 110. For example, the search module 608 can be used to find the rendered three-dimensional digital models 116 based on the keyword description or two-dimensional digital image 114 provided by the user to generate the digital content 602. These models can also be used to support output in virtual reality scenes and augmented reality scenes, such as as part of a virtual shopping service. Other examples are also contemplated.

[0057] Example Systems and Devices

[0058] Fig. 9 An example system, generally at 900, is shown including an example computing device 902, which represents one or more computing systems and / or devices that may implement the various techniques described herein. This is illustrated by the inclusion of a rendered three-dimensional digital model 116. Computing device 902 may be, for example, a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and / or any other suitable computing device or computing system.

[0059] The illustrated example computing device 902 includes a processing system 904, one or more computer-readable media 906, and one or more I / O interfaces 908 coupled to each other in communication. Although not shown, the computing device 902 may also include a system bus or other data and command transmission system coupling various components to each other. The system bus may include any one of different bus structures or any combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any one of various bus architectures. Various other examples are also contemplated, such as control lines and data lines.

[0060] Processing system 904 represents the function of performing one or more operations using hardware. Therefore, processing system 904 is shown as including hardware elements 910 that can be configured as processors, functional blocks, etc. This can include hardware implementations as application-specific integrated circuits or other logic devices formed using one or more semiconductors. Hardware elements 910 are not limited by the materials from which they are formed or the processing mechanisms used therein. For example, a processor may include semiconductors and / or transistors (e.g., electronic integrated circuits (ICs)). In this context, processor executable instructions may be electronic executable instructions.

[0061] Computer-readable storage media 906 is shown to include memory / storage 912. Memory / storage 912 represents memory / storage capacity associated with one or more computer-readable media. Memory / storage component 912 may include volatile media (e.g., random access memory (RAM)) and / or non-volatile media (e.g., read-only memory (ROM), flash memory, optical disk, magnetic disk, etc.). Memory / storage component 912 may include fixed media (e.g., RAM, ROM, fixed hard drive, etc.) and removable media (e.g., flash memory, removable hard drive, optical disk, etc.). As further described below, computer-readable media 906 may be configured in various other ways.

[0062] Input / output interface 908 represents functionality that allows a user to input commands and information to computing device 902, and also allows information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, a touch function (e.g., a capacitive or other sensor configured to detect physical touch), a camera device (e.g., which may employ visible light wavelengths or invisible light wavelengths such as infrared frequencies to identify movement as gestures that do not involve touch), etc. Examples of output devices include a display device (e.g., a monitor or projector), a speaker, a printer, a network card, a tactile response device, etc. Thus, as further described below, computing device 902 may be configured in various ways to support user interaction.

[0063] Various techniques may be described herein in the general context of software, hardware elements, or program modules. Typically, such modules include routines, programs, objects, elements, parts, data structures, etc. that perform specific tasks or implement specific abstract data types. The terms "module," "function," and "part" as used herein typically represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques may be implemented on various commercial computing platforms with various processors.

[0064] The implementation of the described modules and techniques may be stored on or transmitted on some form of computer-readable media. Computer-readable media may include various media that can be accessed by computing device 902. By way of example and not limitation, computer-readable media may include “computer-readable storage media” and “computer-readable signal media”.

[0065] "Computer-readable storage media" may refer to media and / or devices that enable persistent and / or non-transient storage of information, as compared to simple signal transmission, carrier waves, or signals themselves. Therefore, computer-readable storage media refers to non-signal bearing media. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media, and / or storage devices implemented in the following methods or technologies: the method or technology is suitable for storing information such as computer-readable instructions, data structures, program modules, logical elements / circuits, or other data. Examples of computer-readable storage media may include, but are not limited to: RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, hard disk, cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or other storage device, tangible media, or an article suitable for storing the desired information and accessible by a computer.

[0066] "Computer-readable signal media" may refer to signal-bearing media that are configured to transmit instructions to the hardware of the computing device 902, for example, via a network. Signal media may typically embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, a data signal, or other transmission mechanism. Signal media also include any information delivery media. The term "modulated data signal" means a signal in which one or more of its characteristics are set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media include wired media (such as a wired network or a direct wired connection) and wireless media (such as acoustic, RF, infrared, and other wireless media).

[0067] As previously described, hardware elements 910 and computer-readable media 906 represent modules, programmable device logic, and / or fixed device logic implemented in hardware that can be employed in some embodiments to implement at least some aspects of the technology described herein (e.g., execute one or more instructions). Hardware can include components of integrated circuits or components of systems on a chip, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon or other hardware. In this context, the hardware can operate as a processing device that performs program tasks defined by instructions and / or logic embodied by the hardware and hardware for storing execution instructions (e.g., computer-readable storage media described previously).

[0068] The aforementioned combination can also be adopted to implement the various technologies described herein. Therefore, software, hardware, or executable modules can be implemented as one or more instructions and / or logic implemented on a certain form of computer-readable storage medium, and / or implemented by one or more hardware elements 910. Computing device 902 can be configured to implement specific instructions and / or functions corresponding to software modules and / or hardware modules. Therefore, the implementation of the module that can be executed as software by computing device 902 can be implemented at least partially in hardware, for example, by using a computer-readable storage medium and / or a hardware element 910 of a processing system 904. Instructions and / or functions can be performed / operated by one or more products (e.g., one or more computing devices 902 and / or processing systems 904) to implement the technology, modules and examples described herein.

[0069] The technology described herein can be supported by various configurations of computing device 902 and is not limited to the specific examples of the technology described herein. As described below, the functionality can also be implemented in whole or in part by using a distributed system, such as on a "cloud" 914 via a platform 916.

[0070] Cloud 914 includes and / or represents a platform 916 for resources 918. Platform 916 abstracts the underlying functionality of the hardware (e.g., servers) and software resources of cloud 914. Resources 918 may include applications and / or data that may be utilized when performing computer processing on a server remote from computing device 902. Resources 918 may also include services provided over the Internet and / or through a subscriber network such as a cellular network or a Wi-Fi network.

[0071] The platform 916 can abstract resources and functionality to connect the computing device 902 with other computing devices. The platform 916 can also be used to abstract the scale of resources to provide a corresponding level of scale to the demand encountered for resources 918 (which are implemented via the platform 916). Therefore, in an interconnected device embodiment, the implementation of the functionality described herein can be distributed throughout the system 900. For example, the functionality can be implemented partially on the computing device 902 and implemented via the platform 916 that abstracts the functionality of the cloud 914.

[0072] In addition, the techniques described herein also include the following examples.

[0073] In one example, the scanning system uses the following scanning technology: The physical object is scanned while it is placed in the packaging to form a three-dimensional digital model. The three-dimensional digital model is colored using a model coloring system. A two-dimensional digital image of the same or similar physical object is taken. In one example, features of the model are matched to the image. This is then used to align the viewing perspective on the model with the viewing perspective of the object in the digital image, for example, to find which "perspective" of the model corresponds to the image. Color is then applied from the digital image to the model, for example, from the pixels of the image to corresponding points in the model.

[0074] Example 1. A method implemented by at least one computing device, the method comprising:

[0075] receiving, by the at least one computing device, scan data obtained by scanning a physical object within a package, the scan being performed through the package;

[0076] generating, by the at least one computing device, a three-dimensional digital model of the physical object based on the received scan data;

[0077] receiving, by the at least one computing device, a two-dimensional digital image;

[0078] detecting, by the at least one computing device, features of the two-dimensional digital image that correspond to features of the three-dimensional digital model;

[0079] generating, by the at least one computing device, a color for the three-dimensional digital model based on the detecting; and

[0080] The rendered three-dimensional digital model is output by the at least one computing device to support user interaction via a user interface, the user interaction involving changing a viewing perspective with respect to the rendered three-dimensional digital model.

[0081] Example 2. The method of Example 1, wherein the scan is an X-ray scan.

[0082] Example 3. The method of Example 1, wherein the three-dimensional digital model does not include the packaging.

[0083] Example 4. The method of Example 1, wherein the density of the physical object is greater than the density of the packaging.

[0084] Example 5. The method of Example 1, wherein the generating comprises:

[0085] generating an initial rendered three-dimensional digital model as rendered having features corresponding to features of the two-dimensional digital image; and

[0086] A color is generated for at least a portion of the three-dimensional digital model that is not colored in the initial colored three-dimensional digital model to form a generated colored three-dimensional digital model.

[0087] Example 6. The method of Example 5, wherein generating a color for the at least one portion that is not colored is based on detecting another portion of the initial colored three-dimensional digital model as corresponding to the at least one portion.

[0088] Example 7. A method according to Example 6, wherein the correspondence is based on a bilateral correspondence between the at least one part and the other part.

[0089] Example 8. The method of Example 5, wherein generating a color for the at least one portion that is not colored is based on stretching a color from another portion of the initial colored three-dimensional digital model to cover the at least one portion.

[0090] Example 9. The method of Example 1, wherein the generating comprises: linearly blending and skinning the two-dimensional digital image to the three-dimensional digital model.

[0091] Example 10. A method according to Example 1, wherein the generating includes: when the perspective of the three-dimensional digital model is aligned with the perspective of the physical object in the two-dimensional digital image, stretching a portion of the two-dimensional digital image and compressing another portion of the two-dimensional digital image.

[0092] Example 11. The method of Example 1, further comprising: generating a search query based on the three-dimensional digital model, and wherein receiving the two-dimensional digital image is caused by performing a search based on the search query.

[0093] Example 12. The method according to Example 1 also includes: generating digital content to include the colored three-dimensional digital model and a function that a user can select to initiate a purchase of the physical object from the user.

[0094] Example 13. The method of Example 12, wherein the two-dimensional digital image is received from the user via a network, and the user also initiates the generation of the digital content.

[0095] Example 14. The method according to Example 1 also includes: generating the digital content as a web page including the colored three-dimensional digital model.

[0096] Example 15. A system comprising:

[0097] a three-dimensional scanning device configured to generate a three-dimensional digital model by scanning a physical object within the package, the scanning being performed through the package; and

[0098] A model rendering system implemented at least in part in hardware of a computing device to render the three-dimensional digital model of the physical object using the two-dimensional digital image of the physical object by:

[0099] aligning features of the two-dimensional digital image with corresponding features of the three-dimensional digital model;

[0100] generating an initial shaded three-dimensional digital model based on the alignment; and

[0101] Color is generated for at least a portion of the three-dimensional digital model that is not colored in the initial colored three-dimensional digital model to form a generated colored three-dimensional digital model.

[0102] Example 16. The system of Example 15, wherein generating a color for the at least one portion that is not colored is based on detecting another portion of the initial colored three-dimensional digital model as corresponding to the at least one portion.

[0103] Example 17. A system according to Example 16, wherein the correspondence is based on a stereo correspondence between the at least one part and the other part.

[0104] Example 18. The system of Example 15, wherein generating a color for the at least one portion that is not colored is based on stretching a color from another portion of the initial colored three-dimensional digital model to cover the at least one portion.

[0105] Example 19. A system comprising:

[0106] a three-dimensional scanning device configured to generate a three-dimensional digital model by scanning a physical object within the package, the scanning being performed through the package; and

[0107] A model rendering system implemented at least in part in hardware of a computing device to render the three-dimensional digital model of the physical object using a two-dimensional digital image of the physical object received from a user via a network by:

[0108] aligning features of the two-dimensional digital image with corresponding features of the three-dimensional digital model;

[0109] generating an initial shaded three-dimensional digital model based on the alignment; and

[0110] generating a color for at least a portion of the three-dimensional digital model that is not colored in the initial colored three-dimensional digital model to form a generated colored three-dimensional digital model; and

[0111] A digital content control module implemented at least in part in hardware of a computing device to generate digital content to include the rendered three-dimensional digital model and a function selectable by a user to purchase the physical object from the user, the generation of the digital content being initiated by user input received from the user via a network.

[0112] Example 20. The system of Example 19, wherein generating a color for the at least one portion that is not colored is based on detecting another portion of the initial colored three-dimensional digital model as corresponding to the at least one portion.

[0113] in conclusion

[0114] Although the invention has been described in language specific to structural features and / or methodological acts, it should be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed invention.

Claims

1. A method implemented by at least one computing device, the method comprising: receiving, by the at least one computing device, scan data obtained by scanning a physical object contained within a package with a scanner, the scanner being disposed outside the package; generating, by the at least one computing device, a three-dimensional digital model of the physical object based on the received scan data; identifying, by the at least one computing device based on the received scan data, one or more points of a portion of the physical object, the one or more points being movable via at least one of a bending movement and an articulating movement; modifying, by the at least one computing device, the three-dimensional digital model, the modifying comprising repositioning a portion of the three-dimensional digital model of the physical object based on the identified one or more points capable of movement; as well as The three-dimensional digital model is output by the at least one computing device to support user interaction via a user interface, the user interaction involving changing a viewing perspective with respect to the three-dimensional digital model.

2. The method according to claim 1, wherein: The scan is an X-ray scan.

3. The method according to claim 1, wherein: The three-dimensional digital model does not include the packaging.

4. The method according to claim 1, wherein: The density of the physical object is greater than the density of the packaging.

5. The method according to claim 1, further comprising: detecting, by the at least one computing device, features of the digital image corresponding to features of the three-dimensional digital model; generating, by the at least one computing device, a color for the three-dimensional digital model based on the detecting; as well as The rendered three-dimensional digital model is output by the at least one computing device to support user interaction via a user interface, the user interaction involving changing a viewing perspective with respect to the rendered three-dimensional digital model.

6. The method according to claim 5, wherein: Generating color for the three-dimensional digital model includes: generating an initial rendered three-dimensional digital model as rendered having features corresponding to features of the digital image; and A color is generated for at least a portion of the three-dimensional digital model that is not colored in the initial colored three-dimensional digital model to form a generated colored three-dimensional digital model.

7. The method according to claim 6, wherein: Generating a color for the at least one portion of the three-dimensional digital model that is not rendered in the initial rendered three-dimensional digital model is based on detecting a different portion of the initial rendered three-dimensional digital model as corresponding to the at least one portion.

8. The method according to claim 6, wherein: Generating a color for the at least one portion of the three-dimensional digital model that is not colored in the initial colored three-dimensional digital model is based on stretching a color from another portion of the initial colored three-dimensional digital model to cover the at least one portion.

9. The method according to claim 5, wherein: Generating color for the three-dimensional digital model includes stretching a portion of the digital image and compressing another portion of the digital image when the perspective of the three-dimensional digital model is aligned with the perspective of the physical object in the digital image.

10. The method according to claim 1, further comprising: generating a search query based on the three-dimensional digital model, and A digital image resulting from a search performed based on the search query is received.

11. The method according to claim 1, further comprising: Digital content is generated to include the three-dimensional digital model and a function that a user can select to initiate a purchase of the physical object from the user.

12. The method according to claim 11, wherein: The digital image is received via a network from the user who also initiated the generation of the digital content.

13. The method according to claim 1, wherein: The modification of the three-dimensional digital model of the physical object comprises adding the three-dimensional digital model of the accessory to the three-dimensional digital model of the physical object.

14. The method according to claim 1, wherein: The scan data includes the packaging of the physical object.

15. The method according to claim 1, wherein: The generation of the three-dimensional digital model of the physical object is also based on an isosurface extraction algorithm.

16. A system comprising: one or more processors; as well as A computer-readable storage medium comprising instructions executable by the one or more processors to perform operations comprising: receiving scan data obtained by scanning a physical object by a scanner, the scanning being performed by the scanner disposed externally through a package on the physical object disposed inside the package; generating a three-dimensional digital model of the physical object based on the received scan data; identifying one or more points of a portion of the three-dimensional digital model of the physical object based on the received scan data, the one or more points being movable via at least one of bending movement and articulating movement; modifying the three-dimensional digital model, the modifying comprising repositioning the portion of the three-dimensional digital model of the physical object based on the identified one or more points capable of movement; and The three-dimensional digital model is output to support user interaction via a user interface, the user interaction involving changing a viewing perspective with respect to the three-dimensional digital model.

17. The system of claim 16, the operations further comprising: generating a search query based on the three-dimensional digital model, and A digital image obtained by performing a search based on the search query is received.

18. A system comprising: a three-dimensional scanning device disposed outside the package and the physical object, the physical object being disposed inside the package, the three-dimensional scanning device being configured to generate a three-dimensional digital model of the physical object based on scanning data obtained by scanning the physical object by the three-dimensional scanning device, the scanning of the physical object being performed externally through the package; a feature matching module implemented at least in part in hardware of a computing device to identify, based on the scan data, one or more points of a portion of the physical object, the one or more points being movable via at least one of bending movement and articulating movement, and the feature matching module to modify the three-dimensional digital model, the modification comprising repositioning a portion of the three-dimensional digital model of the physical object based on the identified one or more points being movable; and a digital content control module implemented at least in part in the hardware of the computing device to generate digital content to include the three-dimensional digital model and a function that a user can select to purchase the physical object from the user, the generation of the digital content being initiated by user input received from the user via a network.

19. The system of claim 18, wherein: The digital content control module generates the digital content as a web page including the three-dimensional digital model.

20. The system of claim 18, wherein: The generation of the three-dimensional digital model of the physical object is also based on an isosurface extraction algorithm.

Images