3D model optimization

Abstract

Methods, systems, and apparatus for optimizing three-dimensional models, including computer programs encoded on computer storage media. The method includes determining, for a three-dimensional model of an object to be optimized, a plurality of points on the object, each point having at least a threshold likelihood of being in focus, the three-dimensional model having two or more regions, wherein each region includes one or more textures, one or more meshes, or both; identifying one or more non-focus regions from the two or more regions, wherein each non-focus region does not include any of the plurality of points; generating an optimized three-dimensional model for the object having a smaller size than a larger size of the three-dimensional model using the non-focus regions; and storing the optimized three-dimensional model in a non-volatile memory.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 16 / 553,925, filed on August 28, 2019, entitled “Three-Dimensional Model Optimization,” which claims the benefit of U.S. Patent Application No. 16 / 395,722, filed on April 26, 2019, entitled “Three-Dimensional Model Optimization.” The disclosures of the above applications are incorporated herein by reference in their entirety for all purposes. Background Art

[0003] Systems can use three-dimensional ("3D") models to represent objects. For example, an augmented reality ("AR") system, a virtual reality ("VR") system, or a web browser can use 3D models to represent objects in the corresponding environment. The model may be outside the field of view and affect the appearance of objects within the field of view (e.g., a ball bouncing on a chair). Summary of the Invention

[0004] This specification describes techniques, methods, systems, and other approaches for optimizing the creation, rendering, or both of 3D models. For example, a 3D model can be a model of a product displayed through an application such as a web browser or a dedicated application. A computing device such as a mobile device, an AR device, a VR device, or another type of computer can display the 3D model on a screen such as a mobile device screen, a computer screen, AR goggles, or VR goggles. A modeling system can collect viewing data of the 3D model image, analyze the collected data, and optimize the 3D model based on the analysis of the collected data.

[0005] When viewing a 3D model, the viewer may have the ability to view various aspects of the model. The viewer may adjust the model's appearance by, for example, interacting with various aspects of the model using a computer mouse, a finger, or a stylus, or using another input mode to adjust the model's presentation. The viewer may be able to click and drag the model or press keyboard arrow keys to rotate the model. When using an AR or VR device, the viewer may adjust the model's appearance by, for example, turning their head to view various aspects of the model.

[0006] For example, a viewer viewing a 3D model of a backpack may initially be presented with a side view of the backpack. The viewer may click and drag an input device (e.g., a computer mouse) in a vertical direction on the backpack to view the bottom of the backpack. The viewer may click and drag left or right to view the side of the backpack. The viewer may click and drag in another vertical direction to view the top of the backpack.

[0007] When viewing a 3D model, a viewer may have the ability to zoom in or out of the model. The viewer may zoom in on the model, for example, by clicking or double-clicking various aspects of the model with a computer mouse, or by using any other suitable method to zoom in on the model or a portion of the model. The viewer may be able to scroll the mouse wheel or click an icon, such as a magnifying glass, to zoom in or out. For example, a viewer viewing a 3D model of a backpack may be able to zoom in on the backpack's texture or view features of the backpack, such as closures, such as zippers or buttons.

[0008] Different viewers of the 3D model may have different viewing modes when viewing the 3D model. For example, one viewer may rotate the 3D backpack model to a left-side view and then zoom in on the left-side view. Another viewer may rotate the 3D backpack model to a top view and, for example, zoom in on the top view before performing other interactions with the 3D backpack model.

[0009] For example, the modeling system may receive data regarding the viewer's interaction with the 3D model when the viewer chooses to provide this data to the modeling system. For example, the modeling system may receive data regarding the viewer's interaction with the 3D model. The data may represent the viewpoint of the 3D model from the viewer's perspective, the portion of the model presented to the viewer on a display, the order in which portions of the model are presented on the display, or a combination of two or more of the above.

[0010] After receiving the data, such as when the viewer's session of interacting with the 3D model ends, the modeling system can analyze the viewing data. For example, the data may indicate that the viewer spent 10 seconds looking at the front view of the backpack, then spent 20 seconds looking at the left side view of the backpack, zoomed in on the left side view including the water bottle pocket for 10 seconds, and then spent 30 seconds looking at the back view including the backpack's straps.

[0011] The modeling system can receive data for each viewer of the 3D model or a subset of viewers of the 3D model, aggregated data, or both. The modeling system can use this data to optimize the 3D model. For example, the modeling system can use the aggregated data to identify areas of the 3D model that viewers focus on more than other areas of the 3D model. Some examples of areas include a mesh, texture, quadrant, other component of the 3D model, or a combination of two or more of these.

[0012] A region can be, for example, a perspective view of a 3D model, such as a left side view or a right side view. For example, when a perspective view is a region, the perspective view can include a texture, mesh, or both of the 3D model displayed in a particular perspective view, which a viewer may focus on more than other textures, meshes, or both of the 3D model. Each region can be included in one or more perspective views. Each perspective view can include one or more regions, which can, for example, be included in other perspective views.

[0013] For the example 3D model of a backpack, for example, an image of the 3D model presented to a viewer may depict a left side view that includes the water bottle pocket. The image may also depict the textures and meshes of the zipper sleeve and straps. The image may not depict any textures and meshes of the umbrella pocket on the right side of the 3D model, the logo on the left side of the backpack, or both. Because, for example, the logo is located behind the water bottle pocket, the actual textures and meshes of the logo may or may not be loaded into the 3D model, or both, so the logo may not be presented in the left side view. In this example, the perspective view (e.g., as a region) may include at least a portion of the water bottle pocket, at least a portion of the straps, and at least a portion of the zipper sleeve, and may not include the umbrella pocket, the logo, or both.

[0014] In some examples, when the perspective view is a region, the perspective view can include discontinuous portions of the 3D model. For example, the perspective view can include a water bottle bag and a zippered envelope, even if the water bottle bag and the zippered envelope are separated by other regions of the 3D model that are not shown in the perspective view, for example.

[0015] To identify areas of the 3D model that viewers are more interested in than other areas, the modeling system can perform cluster analysis on data from viewer interactions. The cluster analysis can be performed on aggregated data, can generate aggregated data, or both. For example, the modeling system can perform cluster analysis by creating a heat map of the 3D model. The heat map can represent areas of the 3D model using various patterns, shading, colors, or a combination of these to distinguish areas that viewers are more interested in than other areas.

[0016] The modeling system can use the heat map to optimize the 3D model. For example, the modeling system (e.g., a model optimization device) can use the identified regions that the viewer is more interested in than other regions of the 3D model to optimize one or more regions of the 3D model. For regions of the 3D model that the viewer is less interested in than other regions, the modeling system can generate a lower resolution image from a higher resolution image.

[0017] In some implementations, the modeling system can generate two or more versions of the 3D model. Each version of the 3D model can be of a different data size, for example, in megabytes, resolution, or both. The smaller-sized model can include, at high resolution, areas of the 3D model that the viewer is more interested in than other areas of the 3D model, for example, as determined by a heat map. The smaller-sized model can include, at low resolution, areas of the 3D model that the viewer is less interested in than other areas of the 3D model. The larger-sized model can include high-resolution data for areas of the 3D model that the viewer is less interested in than other areas of the 3D model, as well as areas of the 3D model that the viewer is more interested in than other areas of the 3D model (e.g., all areas of the 3D model).

[0018] By optimizing the 3D model, the modeling system can optimize the presentation of the 3D model. For example, when a 3D model of a backpack has a left side view showing a water bottle holder that is of greater interest to viewers than other areas of the 3D model, the modeling system can optimize the 3D model so that the left side view has a higher resolution than other areas of the 3D model, loads first during 3D model presentation, or both. This optimization of the 3D model can enable a device presenting the 3D model (e.g., on a display) to present the optimized 3D model more quickly than an unoptimized 3D model. The modeling system can program high-resolution images of areas of the 3D model that are of less interest to viewers than other areas of the 3D model to load after the initial version has loaded.

[0019] In some implementations, the system can render different quality regions of the 3D model based on the starting orientation of the 3D model. For example, the system can load a first, higher quality (e.g., fidelity) region with reference to what will initially be rendered on the display based on the starting orientation of the 3D model. For example, based on the starting orientation, the system can load a second, lower quality region that will not initially be rendered. For example, for a web page with a camera that initially depicts the front region of the 3D model when the page loads, the system (e.g., a rendering system) can determine to load the front region of the 3D model at a higher resolution while loading the unseen back region at a lower resolution.

[0020] In some implementations, the modeling system can optimize 3D models for categories, such as product categories. For example, viewing data can indicate that for a threshold number of viewers of a particular hiking shoe model, the sole is the area of the 3D model that viewers pay more attention to than other areas of the 3D model. The modeling system can use this data to optimize 3D models for product categories. For example, the modeling system can use data from a particular hiking shoe model to optimize 3D models for product categories, such as all hiking shoes from a particular manufacturer, all hiking boots, and / or all shoes. The modeling system can optimize the 3D model of the shoe so that the sole has a higher resolution than the rest of the 3D model, is loaded first during rendering of the 3D model, or both.

[0021] In some examples, the data may indicate that at least a threshold amount of viewers of collared shirts zoom in on the collar of the shirt. The modeling system may use the data to optimize 3D models for product categories, such as 3D models for multi-collared shirts (e.g., full-collared shirts) or potentially all tops.

[0022] In some implementations, the modeling system may optimize the category of the 3D model based on a category of viewers. The category of viewers may include a single user (e.g., John) or multiple viewers. For example, viewing data for a 3D model of a pillow may indicate that for a threshold number of viewers with a particular demographic (e.g., a physical geographic location), the back of the pillow is an area of the 3D model that the viewers focus on more than other areas of the 3D model. The modeling system may use this data to optimize the 3D model for the category of viewers, for example, when the viewer chooses to share category information such as a physical geographic location. For example, the modeling system may use data from the pillow model to optimize the 3D model for a viewer category, such as all viewers in a particular physical geographic location. The modeling system may optimize the 3D model of the pillow so that the back of the pillow has a higher resolution than the rest of the 3D model, is loaded first during rendering of the 3D model, or both.

[0023] The modeling system can generate different 3D models of the pillow for different viewer categories (e.g., physical geographic locations) and provide one of the different 3D models to the device using data about the viewer category. For example, when the modeling system receives a request for a 3D model of the pillow from a first physical location, the modeling system can provide the first 3D model. When the modeling system receives a request for a 3D model of the pillow from a second, different physical location, the modeling system can provide a second, different 3D model.

[0024] In general, one innovative aspect of the subject matter described herein can be embodied in a method comprising the following acts: for a three-dimensional model of an object to be optimized, determining a plurality of points on the object, each point having at least a threshold likelihood of being in focus, the three-dimensional model having two or more regions, each region comprising one or more textures, one or more meshes, or both; identifying one or more out-of-focus regions from the two or more regions, wherein i) each out-of-focus region does not include any of the plurality of points, and ii) is a proper subset of the two or more regions; using the one or more out-of-focus regions to generate an optimized three-dimensional model for the object having a smaller size than a larger size of the three-dimensional model; and storing the optimized three-dimensional model in a non-volatile memory. Other embodiments of this aspect include corresponding computer systems, apparatus, computer program products, and computer programs recorded on one or more computer storage devices, each configured to perform the acts of the method. The one or more computer systems can be configured to perform specific operations or acts by installing software, firmware, hardware, or a combination thereof on the system, the software, firmware, hardware, or a combination thereof, which, in operation, causes or causes the system to perform the acts. One or more computer programs may be configured to perform certain operations or actions by including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

[0025] In general, one innovative aspect of the subject matter described herein can be embodied in a method comprising the following acts: for a three-dimensional model of an object to be optimized, determining a plurality of points on the three-dimensional model, each point having at least a threshold likelihood of being a focus, the three-dimensional model having two or more regions, each region comprising one or more textures, one or more meshes, or both; identifying one or more focus regions from the two or more regions, wherein i) each includes at least one point from the plurality of points and ii) is a proper subset of the two or more regions; using the one or more focus regions to generate an optimized three-dimensional model for an object having a smaller size than a larger size of the three-dimensional model; and storing the optimized three-dimensional model in a non-volatile memory. Other embodiments of this aspect include corresponding computer systems, apparatus, computer program products, and computer programs recorded on one or more computer storage devices, each configured to perform the acts of the method. The one or more computer systems can be configured to perform specific operations or acts by installing software, firmware, hardware, or a combination thereof on the system, the software, firmware, hardware, or a combination thereof, which, in operation, causes or causes the system to perform the acts. One or more computer programs may be configured to perform certain operations or actions by including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

[0026] The foregoing and other embodiments may each optionally include one or more of the following features, alone or in combination. The method may include receiving a request for an object model across a network; and in response to receiving the request for the object model, transmitting an optimized three-dimensional model of the object to a device and using the network. The method may include, after sending the optimized three-dimensional model having a smaller size to the device, determining to send a three-dimensional model having a larger size to the device; and in response to determining to send the three-dimensional model having the larger size to the device, sending the three-dimensional model to the device.

[0027] In some implementations, determining to send the three-dimensional model having a larger size to the device can include determining that network connections across the network and between the system and the device have a usage less than a threshold. Sending the three-dimensional model to the device can be responsive to determining that network connections across the network and between the system and the device have a usage less than a threshold. Determining to send the three-dimensional model having a larger size to the device can include receiving a request for the three-dimensional model having a larger size. Sending the three-dimensional model to the device can be responsive to receiving the request for the three-dimensional model having a larger size.

[0028] In some implementations, generating the optimized three-dimensional model may include, for each of the one or more non-focus regions, reducing the quality of the one or more textures, one or more meshes, or both included in the three-dimensional model from the quality of the corresponding one or more textures, one or more meshes, or both. For each of the one or more non-focus regions, reducing the quality of the one or more textures, one or more meshes, or both included in the three-dimensional model from the quality of the corresponding one or more textures, one or more meshes, or both may include, for each of the one or more non-focus regions, reducing the resolution of each of the one or more textures, one or more meshes, or both included in the three-dimensional model from a higher resolution of the corresponding one or more textures, one or more meshes, or both.

[0029] In some implementations, identifying one or more non-focus regions from the two or more regions may include identifying one or more textures, one or more meshes, or one or more quadrants as the one or more non-focus regions; and generating the optimized three-dimensional model may include generating an optimized three-dimensional model for an object having a smaller size that is smaller than a larger size of the three-dimensional model using the identified one or more textures or the identified one or more meshes or the identified one or more quadrants.

[0030] In some implementations, determining a plurality of points on an object, each having at least a threshold likelihood of being in focus, can include: retrieving data for a plurality of images of the object from a memory, each image depicting at least a portion of a view of the object generated on a display for presentation to a viewer; determining one or more potential focal points for each image from the plurality of images; and selecting each potential focal point having at least a threshold likelihood of being in focus as the plurality of points from the one or more potential focal points for the plurality of images.

[0031] In some implementations, selecting potential focus points each having at least a threshold likelihood of being in focus can include: i) selecting a first subset of potential focus points from a plurality of potential focus points depicted in a first image of the plurality of images, ii) each potential focus point in the first subset having at least the threshold likelihood of being in focus; and a) determining to skip selection of a second subset of potential focus points from the plurality of potential focus points depicted in the first image of the plurality of images, b) each potential focus point in the second subset not having at least the threshold likelihood of being in focus.

[0032] In some implementations, each of the potential focal points may include an estimated point in a corresponding image from the plurality of images at which a viewer viewing the presentation of an object on the display is likely to focus. Selecting each potential focal point having at least a threshold likelihood of being in focus from the one or more potential focal points in the plurality of images may include weighting at least one of the one or more potential focal points using a distance of the potential focal point from a center of the corresponding image. A first potential focal point that is closer to the center of the corresponding image has a higher weight than a second potential focal point that is farther from the center of the corresponding image.

[0033] In some implementations, the method may include receiving data for one or more of the plurality of images from a device that presents a model of an object on a display and across a network; and storing the data for one or more of the plurality of images in a memory.

[0034] In some implementations, determining one or more potential focal points for each image from the plurality of images may include: for each image from the plurality of images and from a direction indicated by a camera that will generate the corresponding image, casting one or more rays onto the object; and for each of the one or more rays, selecting a point where the ray intersects the object as the corresponding focal point. Casting one or more rays onto the object for each image from the plurality of images and from a direction indicated by the camera that will generate the corresponding image may include: for each image from the plurality of images: determining one or more regions within which to generate the rays; and for each of the one or more regions, randomly generating a ray that is cast onto the object.

[0035] In some implementations, determining, for each image from a plurality of images, one or more regions within which to generate a ray may include determining, for each of the one or more regions, an angular deviation range from a reference point within which to generate the ray. Randomly generating, for each of the one or more regions, a ray projected onto the object may include: randomly selecting, for each of the one or more regions, an angular deviation within the angular deviation range; and generating the ray at the randomly selected angular deviation. The reference point may include the position of a camera that will generate the corresponding image. Determining, for each of the one or more regions, the angular deviation range may include: determining, for each of the one or more regions, a size of the angular deviation range based on the distance of the region from the center of the corresponding image.

[0036] In some implementations, selecting, from the one or more potential focal points of the plurality of images, each potential focal point having at least a threshold likelihood of being in focus can include: for one or more points on the object, determining a number of times the point is a potential focal point for the corresponding image; and selecting, as the plurality of points, the potential focal points whose corresponding quantities satisfy the threshold quantity. Selecting, as the plurality of points, the potential focal points whose corresponding quantities satisfy the threshold quantity can include: for the one or more points on the object, determining a normalized quantity using a highest number of times the point on the object is a potential focal point; and selecting, as the plurality of points, the potential focal points whose corresponding normalized quantities satisfy the threshold quantity. Determining, as the plurality of points on the object, each having at least a threshold likelihood of being in focus can include determining, as the plurality of points on the object, each having a weight that satisfies a threshold weight.

[0037] In some implementations, the first region and the second region may each have a first texture; a third region from the two or more regions may have a second texture different from the first texture; and identifying one or more focus regions from the two or more regions may include identifying the first region as a focus region that includes points from a plurality of points having at least a threshold likelihood of being in focus. The second region may not include any point from each of the plurality of points having at least the threshold likelihood of being in focus. Generating an optimized three-dimensional model of the object using the one or more focus regions may include generating an optimized three-dimensional model of the object using the one or more focus regions, wherein the first region has a higher quality and the second region has a higher quality than the third region, which has a lower quality.

[0038] In some implementations, the method may include determining that the second region does not include any point from the plurality of points that each has at least a threshold likelihood of being a focus. Identifying one or more focus regions from the two or more regions may include identifying one or more textures, one or more meshes, or one or more quadrants as the one or more focus regions; and generating the optimized three-dimensional model may include generating an optimized three-dimensional model for the object having a smaller size that is smaller than a larger size of the three-dimensional model using the identified one or more textures, the identified one or more meshes, or the identified one or more quadrants.

[0039] The subject matter described herein can be implemented in various embodiments and can yield one or more of the following advantages. In some implementations, generating a smaller, optimized 3D model can enable a modeling system to reduce storage requirements for the 3D model, reduce network usage (e.g., when transferring the 3D model to another device or system), or both. By optimizing the 3D model, the time it takes for the 3D model to load onto a device screen can be reduced. The device can initially load a smaller, optimized 3D model containing higher-resolution data for only non-optimized areas and lower-resolution data for optimized areas. The device can then optionally load the higher-resolution data for the 3D model, e.g., when the device later receives a higher-quality 3D model from the modeling system. This can improve loading time and enhance 3D model quality by, for example, maintaining certain areas of greater focus at a higher resolution rather than using a 3D model containing only lower-resolution content, thereby reducing the bandwidth required to transmit the 3D model data, e.g., from the modeling system to the device (or a combination of two or more of these). In some implementations, the modeling system can determine to use a smaller 3D model when low network quality is detected on the device accessing the 3D model. For example, the modeling system may determine to provide a lower quality 3D model to a device accessing a web page on a 3G network compared to a device accessing a web page on a Wi-Fi connection.

[0040] In some implementations, the modeling system can determine to use a combination of lower-quality 3D data and higher-quality 3D data. For example, the modeling system can selectively load a combination of lower-quality 3D data and higher-quality 3D data before a viewing session begins, based on previous data collection. The modeling system can selectively load and / or update the combination of lower-quality 3D data and higher-quality 3D data during a viewing session, for example based on viewer interaction. This technique can improve loading times on devices, such as when accessing web pages on a lower-quality network.

[0041] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is an example of an environment for generating optimized 3D models.

[0043] Figure 2 is a flow chart of a process for generating an optimized 3D model.

[0044] Figure 3 is a diagram of an example environment including a 3D model with two rays cast from an image.

[0045] Figure 4 is an example graph depicting a heat map of aggregated viewing data for a processed 3D model.

[0046] Figure 5 An example image of an optimized 3D model with higher and lower resolution areas.

[0047] Figure 6 is a block diagram of a computing system that can be used in conjunction with the computer-implemented methods described in this document.

[0048] Like reference numbers and designations throughout the drawings indicate like elements. DETAILED DESCRIPTION

[0049] Figure 1 is an example of an environment 100 for generating an optimized 3D model. A 3D model is a three-dimensional representation of an object in 3D space. Environment 100 has two components: a modeling system 105 and a device 115. Modeling system 105 can generate a 3D model, optimize the 3D model, or both. Device 115 can display the 3D model, the optimized 3D model, or a combination of the two on a display 120 at different times. Environment 100 can include a network 110 that can transmit 3D models between modeling system 105 and device 115.

[0050] exist Figure 1 In the example of FIG, modeling system 105 may receive model request 125 from device 115 via network 110. Model request 125 is a request for modeling system 105 to send 3D model 130 to device 115 so that device 115 can display 3D model 130 on display 120. In this example, modeling system 105 receives model request 125 to send 3D model 130 of a camera to device 115.

[0051] Modeling system 105 can store the 3D model in a database, such as any suitable type of database. The 3D model can be 3D model 135, such as a non-optimized 3D model, optimized 3D model 140, or both. An example of optimized 3D model 140 is camera 3D model 130. 3D model 135, optimized 3D model 140, or both can include regions.

[0052] To name a few examples, a region of a 3D model can be one or more textures, one or more meshes, quadrants, or other components of the 3D model. A region can be, for example, a perspective view of the 3D model, such as a left side view or a right side view. For example, when the perspective view is a region, the perspective view can include textures and meshes of the 3D model displayed in a particular perspective that a viewer may focus on more than other textures and meshes of the 3D model. For a camera's 3D model 130, the regions can be, for example, the camera's front view, right side view, left side view, top view, bottom view, and back view.

[0053] In some examples, a perspective view may include one or more regions. Each region may be included in one or more perspective views. Each perspective view may include, for example, one or more regions that may be included in other perspective views.

[0054] For the example 3D model of a camera, for example, an image of the 3D model presented to a viewer may depict a left side view including a shutter release button. The image may also include a texture for the lens. The image may not include any textures for a zoom operator on the right side of the 3D model, for example, if such textures are located behind other areas of the 3D model (such as the lens). In this example, the perspective view (e.g., as a region) may include the shutter release button and the lens, but not the zoom operator, because the zoom operator is not depicted in the image.

[0055] Each region of the 3D model can include one or more textures. The 3D models 130, 135 can include any suitable number of textures, such as those determined during the generation of the 3D models 130, 135. A texture can be an image file containing visual details applied to polygons within the 3D model 135. For example, the texture on the front region of the 3D model 130 of the camera can be an image of the camera lens. The texture on the top region of the 3D model 130 of the camera can be an image of the camera's flash mount.

[0056] Each region of the 3D model 130, 135 may include one or more meshes. A mesh may be a polygon, such as a triangle, defined by a collection of points in a 3D space defined by 3D axes. One or more textures may be applied to each mesh, i.e., wrapped around each mesh. A greater number of meshes may improve image resolution.

[0057] The modeling system 105 may include a focus device 145. The focus device 145 may determine a focus of the 3D model 135, as discussed in more detail below. The focus may be an estimated point on the 3D model at which one or more viewers viewing the 3D model may focus, e.g., based on an image displayed to the viewer. For example, when a viewer of the 3D model 130 of a camera is shown a top area of the camera including the viewfinder and, separately, a bottom area of the camera including the battery compartment, the modeling system 105 may determine that, for the viewer, the 3D model has two focus points. The first focus point may be on the viewfinder, while the second focus point may be on the battery compartment. The modeling system 105 may determine the focus point using, for example, a center point of an image presented on a display. In some implementations, there may not be any eye tracking of the viewer.

[0058] The modeling system 105 (e.g., focus device 145) can store the identified focal points in a focus database 150. Each of the focal points corresponds to (e.g., is included in) an area of the 3D model 135. The focus database 150 can include a mapping that indicates, for each of the identified focal points, the area of the 3D model 135 to which the focal point corresponds. The focus database 150 can include data for each potential focal point of the 3D model 135 or a suitable subset of the potential focal points of the 3D model 135, such as when the modeling system 105 determines that only some of the potential focal points are possible focal points for the viewer.

[0059] Modeling system 105 may classify regions of 3D model 135 based on the focus corresponding to the region, for example, by storing data in focus database 150 indicating whether the region is a focus region 155 or a non-focus region 160. Database 150 may include data for only focus regions 155, only non-focus regions 160, or both.

[0060] Modeling system 105 can use any suitable method to determine focus regions 155, non-focus regions 160, or both. In some implementations, modeling system 105 can determine a weight for each focus point and use the weights of the focus points in the region to categorize the regions, as discussed in more detail below. In some examples, modeling system 105 (e.g., focus device 145 or model optimization device 165) can categorize regions of 3D model 135 that contain more than a threshold number of focus points as focus regions 155. Modeling system 105 (e.g., focus device 145 or model optimization device 165) can categorize regions of 3D model 135 that contain less than a threshold number of focus points as non-focus regions 160. For example, focus region 155 of 3D model 130 of a camera may be the front of the camera, which may include the focus points of a lens and a zoom ring. Each of the lens and zoom ring may have multiple possible focus points, for example, based on their size, number of textures, or both. If the bottom of the camera contains less than a threshold number of focus points, the out-of-focus area 160 of the 3D model 130 of the camera may be the bottom of the camera.

[0061] The threshold number of focal points can be any suitable threshold value. For example, the modeling system 105 can use the size of the 3D model (e.g., in bytes or dimensions, the number of textures or meshes in the 3D model, or both, or a combination of two or more of these) to determine the threshold number of focal points. The threshold number of focal points can be determined as a function of the total number of focal points in the 3D model. For example, a 3D model with more focal points will have a higher threshold number of focal points than a 3D model with fewer focal points. The function can be linear or nonlinear. Additional details regarding the focal point analysis are described below.

[0062] In some implementations, the modeling system 105 can determine a potential focus. As described in more detail below, the modeling system 105 can use the potential focus to determine whether an area should be classified as a focus area 155 or a non-focus area 160. For example, the non-focus area 160 can have a value that is less than or greater than a threshold minimum value for potential focus. The focus area 155 can have a value that is not less than or greater than a threshold minimum value for potential focus.

[0063] Model optimization device 165 uses the focus within focus database 150, the area corresponding to the focus, or both to generate an optimized 3D model 140 having a smaller size than 3D model 135. When measured in data size (e.g., in megabytes, resolution, or both), optimized 3D model 140 may be smaller than 3D model 135.

[0064] To generate optimized 3D model 140, model optimization device 165 may reduce the quality of textures within non-focus region 160. To reduce the quality of textures within non-focus region 160, model optimization device 165 reduces the resolution of each of the textures in non-focus region 160 from the higher resolution of the corresponding texture in 3D model 135. Model optimization device 165 may use any suitable method to reduce the resolution of the textures. By reducing the quality of textures in non-focus region 160, model optimization device 165 generates optimized region 170.

[0065] Model optimization device 165 may process the data for focus region 155 or determine to skip processing data for focus region 155 of 3D model 135. For example, model optimization device 165 may not degrade the quality of textures within focus region 155; for example, model optimization device 165 may determine not to process the textures for focus region 155. In some examples, model optimization device 165 may degrade the quality of textures for focus region 155 by a smaller amount than that for non-focus regions 160. This ensures that the textures for focus region 155 are likely to be of higher quality than the textures for non-focus regions 160. Focus region 155 becomes non-optimized region 175 within optimized 3D model 140.

[0066] Optimized 3D model 140 includes optimized region 170 and non-optimized region 175. Model optimization device 165 stores optimized 3D model 140 in non-volatile memory within modeling system 105. Because the texture within optimized region 170 has reduced quality, such as resolution, optimized 3D model 140 is smaller than 3D model 135.

[0067] Modeling system 105 can generate optimized 3D model 140 from 3D model 135 at any suitable time. For example, modeling system 105 can generate optimized 3D model 140 upon receiving viewer data, such as for determining focus. In some examples, modeling system 105 can generate optimized 3D model 140 upon receiving model request 125.

[0068] In response to receiving model request 125, modeling system 105 may send optimized 3D model 140 (e.g., as 3D model 130) to device 115 using network 110. Device 115 may then display 3D model 130 on display 120. The rendering of 3D model 130 on display 120 shows the 3D model 130 for the camera with a higher resolution texture in non-optimized area 175 and a lower resolution texture in optimized area 170. For example, the front view of the camera may correspond to non-optimized area 175 and have a higher resolution, while the bottom view of the camera may correspond to optimized area 170 and have a lower resolution.

[0069] In some implementations, non-optimized regions 175 can be loaded before optimized regions 170 during rendering on display 120. For example, 3D model 130 (as optimized 3D model 140) can include loading data that indicates the order in which a rendering device should load regions of 3D model 130 for rendering. The loading data can indicate that a rendering device (e.g., device 115) should first load non-optimized regions 175, such as the front of the camera, and then load optimized regions 170, such as the bottom of the camera.

[0070] After sending the optimized 3D model 140 to the device 115, the modeling system 105 can determine whether to send a higher quality 3D model 135 to the device 115. The determination to send the 3D model 135 to the device 115 can be based on the size of the 3D model 135, the usage of the network 110 between the modeling system 105 and the device 115 (e.g., the network 110 bandwidth), or both. If the network 110 has less than a threshold usage and there is available bandwidth, the modeling system 105 can send the larger 3D model 135 to the device 115.

[0071] By optimizing the 3D model, the time to load the 3D model onto the display 120 of the device 115 can be reduced. The device 115 can first load a smaller optimized 3D model 140 that contains high-resolution data for only the non-optimized areas 175 and low-resolution data for the optimized areas 170. Then, for example, when the device 115 later receives a higher-quality 3D model 135 from the modeling system 105, the device 115 can optionally load the high-resolution data for the 3D model 135. This can improve loading time, improve 3D model quality, for example, by keeping certain areas of higher focus at a higher resolution rather than using a 3D model that only includes lower-resolution content, and reduce the bandwidth required to transmit 3D model data (e.g., from the modeling system 105 to the device 115, or a combination of two or more of these).

[0072] In some implementations, regions of 3D model 130 may include a combination of high-resolution data and low-resolution data. For example, 3D model 130 may include a combination of high-resolution data and low-resolution data for some regions. In some examples, 3D model 130 may include a combination of high-resolution data and low-resolution data for each region.

[0073] The high-resolution data may include more and / or smaller meshes for an area, while the low-resolution data may include fewer and / or larger meshes for an area. Based on viewer interaction, available network bandwidth, and available processing resources, or a combination of two or more of these, the system (e.g., a rendering system on device 115) may select and use high-resolution data or low-resolution data for various areas of 3D model 130. The system may dynamically change the quality of an area before or during a viewing session, for example, based on using high-resolution data or low-resolution data.

[0074] In some implementations, the modeling system 105 may send low-resolution and high-resolution data for some areas simultaneously, may send low-resolution and high-resolution data for some areas as needed, may send some low-resolution data for some areas and then some high-resolution data, or a combination of two or more of these. The subsequent high-resolution data may be sent all at once or in separate messages. For example, the 3D model 130 may initially load a certain area at low resolution. When the device 115 renders the 3D model 130, the device 115 may replace the low-resolution data with the high-resolution data, for example, to improve the quality of the content rendered on the display 120. In this way, the device 115 can dynamically update the 3D model 130 during rendering.

[0075] In some implementations, the modeling system 105 may determine focus weights, for example, using viewer data, session browsing behavior, or both for a viewer session. The determination of focus weights specific to a viewer session may enable the modeling system 105 to dynamically configure areas of the 3D model 130 that are specific to that viewer session, such as the capabilities of the device 115, the network connection to the requesting device 115, the specific content presented by the requesting device 115, or a combination of two or more of these. For example, for some areas, the modeling system may dynamically determine low-resolution data, medium-resolution data, high-resolution data, or a combination of two or more of these based on the focus weights. In some implementations, the modeling system may generate and cache pre-customized versions of the 3D model 130 based on the focus weights, and select one of the pre-customized versions of the 3D model 130 for an individual viewer and / or individual viewing session.

[0076] For example, if device 115 presents multiple different 3D models within a category, such as a shirt, during a browsing session, modeling system 105 can dynamically configure regions of 3D model 130 of the shirt based on the content presented during the browsing session, such as the specific perspectives of the multiple different 3D models presented during the browsing session. Modeling system 105 can send low-resolution data for regions of the shirt that include low-weighted focal points for the session. Modeling system 105 can send high-resolution data for regions of the shirt that include high-weighted focal points for the session.

[0077] In some implementations, the 3D model 130 may include multiple regions having the same texture or textures. For example, when the 3D model 130 of a camera includes a lens, the 3D model 130 may include a first region having a particular texture at the top of the lens and a second region having a particular texture at the bottom of the lens. The modeling system 105 may determine to optimize a repeating texture (e.g., a particular texture) in a specific, individual region (e.g., the bottom of the lens) or in all regions including a repeating texture. In some examples, the modeling system 105 may optimize the repeating texture only if the repeating texture is not included in the focus region 155. In some implementations, the modeling system 105 may determine not to optimize a repeating texture, such as a particular texture, if the particular texture is located in one or more focus regions 155 (e.g., the top of the lens).

[0078] The modeling system 105 is an example of a system implemented as a computer program on one or more computers in one or more locations in which the systems, components, and techniques described in this document are implemented. The devices 115 may include personal computers, mobile communication devices, augmented reality ("AR") goggles, virtual reality ("VR") goggles, and other devices that can send and receive data over the network 110. The network 110, such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, connects the devices 115 and the modeling system 105.

[0079] Modeling system 105 may utilize a single server computer or multiple server computers operating in conjunction with one another, including, for example, a set of remote computers deployed as a cloud computing service. Modeling system 105 may include several different functional components, including focus device 145 and model optimization device 165. The various functional components of modeling system 105 may be installed on one or more computers as separate functional components or as different modules of the same functional component. For example, focus device 145 and model optimization device 165 of modeling system 105 may be implemented as computer programs installed on one or more computers in one or more locations, coupled to one another via a network. For example, in a cloud-based system, these components may be implemented by individual computing nodes of a distributed computing system.

[0080] Figure 2 is a flow chart of a process 200 for generating an optimized 3D model. For example, modeling system 105 from environment 100 can use process 200. In some examples, some steps of process 200 can be performed in modeling system 105 by focus device 145, model optimization device 165, or both.

[0081] The modeling system retrieves data for images of the object from a memory, each of the images depicting at least a portion of a view of the object to be generated on a display for presentation to a viewer (202). For example, the image data may be the position, orientation, focal length, or a combination of two or more of these of a virtual camera. In some examples, the image data may be, for example, the position and orientation of the object relative to the virtual camera.

[0082] An object may have two or more regions, each of which may include one or more textures. An image of an object may depict portions of a view of the object displayed to a viewer, portions of a view of the object displayed to multiple viewers, or both. For example, a modeling system or another system may provide a 3D model of the object to a device. The device may use the 3D model to render one or more views of the object. The device may acquire data representing these views, for example, as images.

[0083] For a 3D model of a camera, the regions may be, for example, the front region, right region, left region, top region, bottom region, and back region of the camera. An example texture on the top region of the 3D model of the camera may be an image of the camera flash mount, while an example texture on the bottom region of the 3D model of the camera may be an image of the battery compartment.

[0084] During an example viewer interaction, a first viewer may view a top area including a flash mounting texture and a bottom area including a battery compartment texture. A second viewer may view only the top area, including the flash mounting texture, from a different angle than the first viewer. The modeling system may obtain first data for the top view and the bottom view from the first viewer interaction, and second data for the top view from the second viewer interaction.

[0085] In some examples, the images can be from multiple viewer sessions. For example, a first viewer can view the 3D model of the camera during a first session. A second viewer, or the first viewer, can view the 3D model of the camera during a second session that is different from the first session.

[0086] In some implementations, the data for the images may come from a viewer session within a studio, such as a physical VR studio. For example, a viewer may participate in a VR experience in which they view and interact with a 3D model of a product, such as at a retail store.

[0087] In some implementations, the system can randomly generate various combinations of meshes and textures, such as potentially optimized meshes, textures, or both. The system can display these combinations to a viewer and receive feedback. In some examples, the system can collect load time data for some combinations of meshes and textures. The modeling system can use this data to determine which mesh and texture combinations to optimize.

[0088] For each image, the modeling system determines one or more potential focal points (204). Each of the potential focal points can be an estimated point in the corresponding image from the image on which a viewer viewing the object presented on the display might focus. The estimated point can be a point in the image, a point on the 3D model, or both. The modeling system can use, for example, focus device 145 from environment 100 to identify the potential focal points.

[0089] To identify potential focal points, the modeling system may cast one or more rays from an image of the 3D model of the object onto the 3D model. The modeling system may use the position, orientation, focal length of the virtual camera, or a combination of two or more of these to cast the rays. The modeling system may use the position, orientation, and / or focal length of the virtual camera; the position and / or orientation of the 3D model; or both to determine potential focal points. The image of the 3D model of the object may depict a field of view of the 3D model, such as a portion of the 3D model. For example, in a virtual world including the 3D model, the modeling system casts rays from the image from a direction represented by the virtual camera, which will generate a corresponding image for a viewer (e.g., for the field of view of the corresponding image).

[0090] Figure 3 is a diagram of an example environment 300 including a 3D model 302 with two rays 304 cast from an image 305. A modeling system may use environment 300 of one or more of images 305 to determine potential focal points.

[0091] For example, for each image, the modeling system can determine one or more regions 306, 308 within which to generate a ray 304 or multiple rays. To determine the regions 306, 308 within which to generate the rays 304, the modeling system can identify one or more reference points, distances from the reference points, or both to define the corresponding regions. The modeling system can use the center of the image 305 as a reference point. In some examples, the modeling system can use a point in the left third of the image 305 as a reference point, such as a point centered on the left third of the image. In some examples, the modeling system can use the position of a virtual camera, which can be used to create the image 305, to determine the reference point. For example, the modeling system can use the position of the virtual camera relative to the image 305, the 3D model 302, or both to determine the reference point.

[0092] The modeling system can determine any suitable number of regions. For example, first region 306 can be circular and centered in image 305. Second region 308 can be circular and centered in image 305, e.g., having the same center 310 as first region 306. Second region 308 can be non-overlapping with first region 306, e.g., second region 308 can be annular with a hole defined by first region 306.

[0093] The modeling system can project rays from image 305 toward camera model 302 at any suitable angle. For example, the modeling system can determine an angular deviation range within which to generate ray 304 based on the regions 306, 308 from which ray 304 is projected. The modeling system can use the region's distance from center 310 of image 305 to determine the size of the angular deviation range. In some examples, regions closer to center 310 can have a smaller angular deviation range than regions farther from center 310. The modeling system can randomly select an angular deviation 312 within the angular deviation range and generate ray 304 at the randomly selected angular deviation 312. For each ray 304, the modeling system can select the point where the ray intersects 3D model 302 as the corresponding potential focus 314.

[0094] Potential focal point 314 can be any suitable size. The modeling system can select the size of the potential focal point based on the resolution of 3D model 302, the size of the 3D model, or any other suitable data. For example, the potential focal point can be smaller or larger than Figure 3 Potential focus depicted in 314.

[0095] Return Reference Figure 2 The modeling system determines, for each of the potential focus points, whether the potential focus point has at least a threshold likelihood of being a focus point (206). The modeling system may classify the potential focus points that have at least the threshold likelihood of being a focus point as a focus point. For example, the modeling system may store data classifying the potential focus points as a focus point in a memory, such as a database implemented on the memory.

[0096] To select a potential focus 314 that has at least a threshold likelihood of being a focus, the modeling system may weight the potential focus 314 using a distance 316 of the potential focus 314 from a ray cast from the center 310 of the image 305 (e.g., where the ray does not have any angular deviation), or a distance 316 of the ray 304 used to determine the potential focus 314 from the center 310 of the image 305. For example, a potential focus 314 that is closer to the center 310 of the corresponding image 305 may have a higher weight than a potential focus 314 that is farther away from the center 310 of the corresponding image 305.

[0097] For example, when a viewer interacts with the camera's 3D model, the system can predict that the viewer is focused on the center 310 of the image 305 displayed on the screen. The system can also predict that the viewer will focus on a point displayed toward the edge of the screen. The system can determine that a first point closer to the center 310 of the corresponding image is more likely to be the point on which the viewer is focused than a second point farther away from the center 310 of the image. Therefore, the system can assign a higher weight to the first point than to the second point.

[0098] The likelihood that a point is a focus may be based on the number of images in which the point is a potential focus 314 during viewer interaction, the distance 316 between the potential focus 314 and the center 310 of the image during viewer interaction, the weight of the potential focus 314, or a combination of two or more of these.

[0099] For example, a first point may be displayed in the center 310 of ten images during viewer interaction. A second point located at a distance 316 from the center 310 of the ten images may have a lower likelihood of being the focus than the first point. Similarly, a third point that is the center 310 of only five images may have a lower likelihood of being the focus than the first point. Alternatively, a fourth point that is located in the center 310 of twenty images may have a higher likelihood of being the focus than the first point.

[0100] In some implementations, the modeling system can generate weights for potential focal points. For example, the modeling system can generate a weighted focal point chart, a weighted area chart, or both. The modeling system can generate weights, such as described above, and create a chart including the weights. The chart can represent a 3D model and include a weight for each potential focal point in the 3D model. For example, the chart can include weights for meshes, vertices, edges, or a combination of two or more of these.

[0101] The modeling system can use a weight threshold to determine the likelihood that a potential focus, an area including a potential focus, or both is a focus or a focus area. When the modeling system uses weights between 0 and 1 (inclusive), the weight threshold can be 0.25. A weight can satisfy the threshold weight when it is greater than or equal to the threshold weight, or when it is greater than the threshold weight. In some examples, a weight can satisfy the threshold weight when it is less than or equal to the threshold weight, or when it is less than the threshold weight, e.g., when a lower value indicates a higher likelihood of being a focus.

[0102] For example, when the weight of a potential focus satisfies a weight threshold, the modeling system may determine that the potential focus is a focus. When the weights of potential focuses included in a region satisfy a weight threshold, the modeling system may determine that the region is a focus region. In some examples, when a region includes multiple potential focuses, the modeling system may compare the combination of the weights of the multiple potential focuses to the weight threshold. For example, the modeling system may compare the average weight or the sum of the weights to the weight threshold. The modeling system may use multiple weight thresholds when applying different levels of optimization to each of the weight thresholds.

[0103] In some implementations, the modeling system can cluster the average weights to determine a focal point. For example, the modeling system can determine a set of weights that each meet a weight threshold and are within a threshold distance of each other or other points in the set. The modeling system can determine the center of the set of weights, such as the point closest to the center, and classify the center as a focal point. The modeling system can use the focal point as a focal region, classify a larger area surrounding the focal point as a focal region, or both.

[0104] In some implementations, the modeling system can use network bandwidth, device-specific capabilities, or both to generate weights, determine weight thresholds, or both. For example, when the available network bandwidth of the device requesting the 3D model is low compared to a higher available network bandwidth, the modeling system can select a higher weight threshold to determine the focus and focus area. Similarly, the modeling system can select a higher weight threshold for a device with less processing power than for a device with more processing power. This will result in fewer focus points and more optimized areas.

[0105] In some implementations, the modeling system can cluster potential focal points to determine whether a point meets a threshold for being a focal point. For example, the modeling system can generate aggregated viewing data using the potential focal points. The modeling system can use the aggregated viewing data to determine whether a potential focal point has at least a threshold likelihood of being a focal point. For example, the modeling system can perform cluster analysis by creating a heat map of the 3D model. The heat map can represent areas of the 3D model using various patterns, shading, colors, or a combination thereof to distinguish areas that viewers are more focused on than other areas.

[0106] In some implementations, the modeling system may perform clustering of 3D coordinate paths on or around the 3D model. A 3D coordinate path is a path between one or more points in a 3D space defined by 3D axes (e.g., x, y, and z axes). The modeling system may determine one or more coordinate paths based on an image presented on a display. For example, the data of an image may indicate that the device presents an image centered on coordinate point p1 = (x1, y1, z1), point p2 = (x2, y2, z2), and then point p3 = (x3, y3, z3). Using this data, the modeling system may determine a coordinate path that includes point p1, point p2, and then point p3. The modeling system may determine multiple coordinate paths for different viewer sessions.

[0107] The modeling system can use the coordinate paths to classify regions of the 3D model. For example, the modeling system can identify regions of the 3D model with fewer clustered coordinate paths as non-focus regions. The modeling system can identify regions of the 3D model with more clustered coordinate paths as focus regions.

[0108] The modeling system can aggregate multiple coordinate paths, such as from multiple viewers or multiple viewing sessions, to create a path for a 3D focal density cone. The 3D focal density cone can represent an optimal viewing angle for rendering an image of a 3D model. The path can include multiple 3D focal density cones, each representing an optimal viewing angle for a different portion of the 3D model. The modeling system can determine regions included in the 3D focal density cone. The modeling system can classify regions included in the 3D focal density cone as focal regions, determine to skip optimization for these focal regions, or both. The modeling system can classify regions not included in the 3D focal density cone as non-focal regions, optimize these non-focal regions, or both.

[0109] Figure 4 4 is an example graph depicting a heat map 400 of aggregated viewing data for a processed 3D model. Heat map 400 may indicate the number of times each point in the 3D model is a potential focus point, the weight of a potential focus point, or both. For example, each point in the darkest shaded region 402 of the 3D model may meet at least a first threshold likelihood of being a potential focus point. The threshold likelihood can be a quantity (e.g., a large number of images have the point as a potential focus point) or a percentage (e.g., 10.25% of images have the point as a potential focus point), to name a few examples. Each point in the 3D model in the medium shaded darkness region 404 does not meet the first threshold likelihood but does meet a second threshold likelihood of being a potential focus point. For example, each point in the medium shaded darkness region 404 may have a percentage greater than a lower threshold but less than a higher threshold, such as being a potential focus point in more than 15% but less than 25% of images. Each point in the lightest shaded region 406 of the 3D model does not meet either the second threshold likelihood or the first threshold likelihood. In some examples, heat map 400 may include only one threshold likelihood or more than two threshold likelihoods.

[0110] Because points in lightest shaded area 406 do not meet the second threshold likelihood of potential focus, the modeling system can classify lightest shaded area 406 as a non-focus area. Because areas 402 and 404 meet one or both of the first threshold likelihood or the second threshold likelihood, the modeling system can classify darkest shaded area 402 and medium shaded darkness area 404 as focus areas.

[0111] When the modeling system has multiple threshold probabilities, the modeling system may include multiple “focus area” classifications. For example, the modeling system may use a medium focus area classification for the medium shadow darkness area 404 and a high focus area classification for the darkest shadow area 402.

[0112] The modeling system can classify areas, points, or both. For example, the modeling system can classify points in the lightest shaded area 406 as out of focus, e.g., along with or in addition to classifying the lightest shaded area 406 as an out of focus area. The modeling system can classify points in the darkest shaded area 402 as in focus, e.g., along with or in addition to classifying the darkest shaded area 402 as an in focus area.

[0113] The modeling system can normalize the value of each of the points in the 3D model. For example, the modeling system can determine the highest value of a point in the 3D image, such as the highest number of times a point is a potential focus. The modeling system can use the highest number to normalize the value of each of the points in the 3D model. For example, when the first point in the 3D model is a potential focus in 100 images, and the second point in the 3D model is a potential focus in 63 images, the modeling system can assign the first point a normalized value of 1.0 and the second point a normalized value of 0.63. The modeling system can use the normalized values to generate a heat map.

[0114] In some examples, from a plurality of potential foci, the modeling system can select or determine to skip selecting a first subset of potential foci, each of which has at least a normalized threshold likelihood of being a focus. The first subset of potential foci can include points in darkest shaded area 402. The modeling system can select or determine to skip selecting a second subset of potential foci 314, each of which does not have at least a threshold likelihood of being a focus. The second subset of potential foci can include points in lightest shaded area 406. The modeling system can store the selected foci, for example, in focus database 150 of environment 100.

[0115] return Figure 2 In response to determining that the potential focus does not have at least a threshold likelihood of being a focus, the modeling system classifies the potential focus as not being a focus (208). For example, referring to Figure 4 , the modeling system determines for each of the points in the lightest shaded area 406 that the point is not in focus. To classify a potential focus as not in focus, the modeling system may select or determine to skip selecting the potential focus. When selecting a potential focus, the modeling system may select a set of potential points, each of which is not in focus.

[0116] The modeling system determines to optimize a non-focus area including points that are not in focus by reducing the resolution of the texture included in the non-focus area (210). The modeling system may use, for example, the model optimization device 165 of the environment 100 to optimize the non-focus area. The modeling system may determine to optimize the non-focus area by reducing the resolution of the texture included in the non-focus area. For example, Figure 4, the modeling system can optimize region 406 by reducing the resolution of the texture included in region 406. By reducing the resolution of the texture included in region 406, the modeling system can reduce the required storage space, the bandwidth required to transmit data, or both for points in region 406.

[0117] In response to determining that the potential focus has at least a threshold likelihood of being a focus, the modeling system classifies the potential focus as a focus (212). Figure 4 , the modeling system determines that one or more points in the darkest shadow region 402 are in focus. To classify a potential focus point as in focus, the modeling system can select the potential focus point as in focus, or determine to skip selecting a potential focus point, for example, and only select points that are not in focus. When selecting a potential focus point, the modeling system can select a set of potential focus points, each of which is in focus.

[0118] The modeling system determines to skip the optimization of the focus region including the point that is the focus (214). Figure 4 , the modeling system can determine to skip optimization of regions 402 and 404 because regions 402 and 404 are focus regions that include points that are in focus. By skipping optimization of regions 402 and 404, the modeling system maintains a high resolution of textures in regions 402 and 404. Maintaining a high resolution of content in regions 402, 404, or both can improve the visual presentation of the content in regions 402, 404, or both.

[0119] In some implementations, the modeling system can use a hierarchical approach to optimize regions of the 3D model. For example, region 404 has fewer focal points than region 402, but does include at least one focal point. The modeling system can reduce the resolution of the texture in region 404 to a resolution that is lower than the resolution of the texture in region 402 but higher than the resolution of the texture in region 406.

[0120] In some implementations, the modeling system can reduce the resolution of textures in all regions to varying degrees. For example, if available storage space, network bandwidth, or both are limited, the modeling system can determine to optimize or partially optimize both the focal region and the non-focal region. For example, the resolution of the texture in region 406 can be reduced to a resolution that is lower than the original 3D model resolution but higher than the optimized resolution of regions 404 and 406.

[0121] For either branch of the above determination at step 206, the modeling system can generate an optimized 3D model using the in-focus and out-of-focus areas (216). For example, the modeling system can generate a 3D model having a higher resolution texture for the in-focus area and a lower resolution texture (e.g., a reduced resolution texture) for the out-of-focus area. The modeling system can generate the 3D model using data for the in-focus area, data for the out-of-focus area, or both. The combined in-focus and out-of-focus areas form an optimized 3D model having a smaller size than the original 3D model. In some examples, instead of or in addition to generating the optimized 3D model, the modeling system can store the data for the in-focus and out-of-focus areas in a memory.

[0122] Figure 5 is an example diagram of an optimized 3D model 500 having higher and lower resolution regions. Region 502 is a higher resolution region, such as corresponding to a focus region. In some examples, the modeling system has not optimized region 502 or has optimized region 502 to a smaller extent than the lower resolution regions. Thus, region 502 may be the region with the highest resolution texture, or may be one of multiple regions with the highest resolution texture. Region 504 is a region with a lower texture resolution than region 502 due to partial optimization of the content depicted in region 504. Region 504 may correspond to a focus region that does not have as high a resolution as region 502. Region 506 is a lowest resolution region, such as corresponding to a non-focus region. The modeling system has optimized region 506 to provide 3D model 500 with the lowest texture resolution. In the reduced resolution regions, the quality of one or more textures or other content is reduced compared to the quality of the corresponding textures or other content in the original 3D model. The resulting optimized 3D model 500 is smaller than the original 3D model.

[0123] A 3D model can have one or more of each type of region. For example, a 3D model can have two in-focus regions and three out-of-focus regions. The two in-focus regions can be non-contiguous, for example, separated by one or more out-of-focus regions. Some or all of the three out-of-focus regions can be non-contiguous, for example, separated by one or more of the two in-focus regions.

[0124] The modeling system stores the optimized 3D model in non-volatile memory (218). Generating a lower quality optimized 3D model can enable the modeling system to reduce storage requirements for the 3D model, reduce network usage (e.g., when transmitting the 3D model to another device or system), or both.

[0125] The order of the steps in the above process 200 is illustrative only, and the 3D model optimization can be performed in a different order. For example, when the modeling system stores the optimized 3D model in non-volatile memory (218), the modeling system can determine that the size of the optimized 3D model is still too large for transmission to the device based on, for example, the available network bandwidth of the device requesting the 3D model. The modeling system can then determine additional areas of the 3D model to optimize by reducing the resolution of the texture included in the area (210). The modeling system can then regenerate the optimized 3D model, which includes the focus and non-focus areas (216). In this example, the modeling system can select the threshold number, the degree of optimization, or both based on the maximum size of the optimized 3D model.

[0126] In some implementations, process 200 may include additional steps, fewer steps, or may split some steps into multiple steps. For example, the modeling system may perform steps 204, 208, 212, 216, and 218 without performing other steps in process 200. In some examples, the modeling system may perform steps 204, 208, and 216 without performing other steps in process 200.

[0127] In some implementations, the modeling system can retrieve an image of the object, the image depicting at least one view of the object generated on a display, for presentation to a viewer after one viewer interaction, several viewer interactions, or many viewer interactions (202). In some implementations, the modeling system can retrieve the image of the object at specified time intervals. In some implementations, the modeling system can retrieve the image of the object when triggered by an event. For example, the modeling system can retrieve the image of the object upon receiving a model request (e.g., model request 125 from environment 100).

[0128] In some implementations, the determination to skip optimizing the focus region (214) can include determining to optimize one or more focus regions to varying degrees. For example, the modeling system can determine to partially optimize a focus region that includes a small number of focus points. The modeling system can reduce the resolution of the texture of the focus region to a resolution that is lower than the texture resolution of focus regions with more focus points but higher than the texture resolution of non-focus regions.

[0129] The modeling system can use any suitable data indicating a portion of a view of an object generated on a display for presentation to a viewer, e.g., instead of or in addition to an image. In some implementations, the device can acquire coordinates of a 3D model indicating the view of the model presented to the viewer. The modeling system can receive coordinate data, e.g., xyz coordinate data, from the device. The modeling system can use the coordinate data for process 200.

[0130] In some implementations, the modeling system may use a weight threshold with a weighted focus chart, a weighted region chart, or both to determine what to optimize. For example, the modeling system may overlay focus weights, regions, or both onto the 3D model to determine a hierarchy of meshes, textures, or both in a region of the 3D model. The modeling system may use the hierarchy of meshes, textures, or both to determine the meshes and textures for an optimized region of the 3D model. For example, when the overlay on the 3D model indicates that a region does not include any focus weights that meet a threshold weight, the modeling system may classify the region as a non-focus region, such as associated with steps 208 and 210. When a region includes a focus weight that meets a threshold weight, the modeling system may classify the region as a focus region, such as associated with steps 212 and 214.

[0131] When overlaying a region onto a 3D model, the modeling system can first determine whether the region is a focus region or a non-focus region. The modeling system can then select a mesh, texture, or both for optimization from the overlaid 3D model included in the non-focus region, such as those associated with steps 208 and 210. The modeling system can determine to skip selection or perform another appropriate process for the mesh, texture, or both in the overlaid 3D model included in the focus region, such as those associated with steps 212 and 214.

[0132] In some implementations, the modeling system can be part of a device that renders the 3D model. For example, an application on the device (e.g., the modeling system) can dynamically render optimized or non-optimized areas of the 3D model. The application can include the modeling system and the rendering system. In some examples, when the application is a web browser, the application can use JavaScript. The modeling system can use weighted focus (e.g., a weighted focus chart) to intelligently pre-cache data for areas of the 3D model, render data for areas of the 3D model, or both.

[0133] For example, the modeling system can retrieve a 3D model from a server. The 3D model can include higher-quality data for a focus area that will initially be presented on a display. The 3D model can also include lower-quality data for non-focus areas that will not initially be presented on the display. The model can include higher-quality data for the non-focus areas. In some examples, the modeling system can receive the higher-quality data for the non-focus areas separately from the model, for example, upon request or after receiving the model.

[0134] When a device first renders a 3D model, it can use an optimized 3D model with higher-quality data for generating the image for the display and lower-quality data for generating portions of the 3D model that are not initially rendered. The lower-quality data may include only a mesh without any textures. As the device receives user input, it can dynamically determine which areas have at least a threshold likelihood of being displayed and render higher-quality data for those areas. The device can retrieve the higher-quality data for these areas from a cache, dynamically request the higher-quality data from a server, or both.

[0135] Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their structural equivalents), or a combination of one or more thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more computer program instruction modules encoded on a tangible, non-transitory program carrier, for execution by a data processing device or for controlling the operation of the data processing device. Alternatively or in addition, program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more thereof.

[0136] The term "data processing apparatus" refers to data processing hardware and includes all kinds of devices, equipment, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. The apparatus may also be or further include special-purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the apparatus may optionally include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0137] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script, or code) may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and the computer program may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file containing other programs or data, such as one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, such as files storing one or more modules, subroutines, or code portions. A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0138] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0139] The computer that is suitable for executing computer programs includes, for example, a general or special microprocessor or both, or a central processing unit of any other kind. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for fulfilling or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include or be operably coupled to receive data from one or more large-capacity storage devices (for example, magnetic disks, magneto-optical disks or optical disks) for storing data or transfer data to these devices or both. However, a computer does not need such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a smart phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0140] Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and storage devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0141] To provide for interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., an LCD (liquid crystal display), an OLED (organic light emitting diode), or other monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide for interaction with the user; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, a computer may interact with a user by sending files to and receiving files from a device used by the user, for example, by sending a web page to a web browser on a user's device in response to a request received from the web browser.

[0142] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with implementations of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0143] A computing system may include a client and a server. The client and the server are typically remote from each other and typically interact via a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., a Hypertext Markup Language (HTML) page) to a user device, for example, for the purpose of displaying data to a user interacting with the user device and receiving user input from the user device, the user device acting as a client. For example, as a result of user interaction, data generated at the user device may be received at the server from the user device.

[0144] Figure 66 is a block diagram of computing devices 600, 650 that can be used to implement the systems and methods described in this document as a client or server or multiple servers. Computing device 600 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Computing device 650 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, smart watches, head-mounted devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions are exemplary only and are not meant to limit the implementations described and / or claimed in this document.

[0145] The computing device 600 includes a processor 602, a memory 604, a storage device 606, a high-speed interface 608 connected to the memory 604 and a high-speed expansion port 610, and a low-speed interface 612 connected to a low-speed bus 614 and the storage device 606. Each of the components 602, 604, 606, 608, 610, and 612 is interconnected using various buses and can be mounted on a common motherboard or otherwise as appropriate. The processor 602 can process instructions for execution within the computing device 600, including instructions stored in the memory 604 or on the storage device 606 to display graphical information of a GUI on an external input / output device (such as a display 616 coupled to the high-speed interface 608). In other implementations, multiple processors and / or multiple buses and multiple memories and memory types can be used as appropriate. In addition, multiple computing devices 600 can be connected, each device providing a portion of the necessary operations (e.g., as a server group, a group of blade servers, or a multi-processor system).

[0146] The memory 604 stores information within the computing device 600. In one implementation, the memory 604 is a computer-readable medium. In one implementation, the memory 604 is a volatile memory unit or units. In another implementation, the memory 604 is a non-volatile memory unit or units.

[0147] Storage device 606 can provide mass storage for computing device 600. In one implementation, storage device 606 is a computer-readable medium. In various implementations, storage device 606 can be a floppy disk device; a hard disk device; an optical disk device; or a magnetic tape device; a flash memory or other similar solid-state memory device, or an array of devices, including a storage area network or other configured devices. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as the methods described above. The information carrier is a computer- or machine-readable medium, such as memory 604, storage device 606, or memory on processor 602.

[0148] The high-speed controller 608 manages bandwidth-intensive operations of the computing device 600, while the low-speed controller 612 manages less bandwidth-intensive operations. This division of responsibilities is exemplary only. In one implementation, the high-speed controller 608 is coupled to the memory 604, the display 616 (e.g., via a graphics processor or accelerator), and the high-speed expansion port 610, which can accept various expansion cards (not shown). In one implementation, the low-speed controller 612 is coupled to the storage device 606 and the low-speed expansion port 614. The low-speed expansion port (which can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)) can be coupled to one or more input / output devices, such as a keyboard, pointing device, scanner, or network device, such as a switch or router, for example, via a network adapter.

[0149] As shown, computing device 600 can be implemented in a variety of different forms. For example, computing device 600 can be implemented as a standard server 620, or multiple implementations in a group of such servers. Computing device 600 can also be implemented as a part of rack server system 624. In addition, computing device 600 can be implemented in a personal computer (such as laptop computer 622). Alternatively, components from computing device 600 can be combined with other components in a mobile device (not shown) (such as device 650). Each of such devices can include one or more of computing devices 600, 650, and the entire system can be composed of multiple computing devices 600, 650 that communicate with each other.

[0150] Computing device 650 includes a processor 652, a memory 664, an input / output device such as a display 654, a communication interface 666, and a transceiver 668, among other components. Device 650 may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of components 650, 652, 664, 654, 666, and 668 is interconnected using various buses, and several of the components may be mounted on a common motherboard or in other ways as appropriate.

[0151] The processor 652 can process instructions for execution within the computing device 650, including instructions stored in the memory 664. The processor can also include separate analog and digital processors. For example, the processor can provide coordination of other components of the device 650, such as control of the user interface, execution of applications on the device 650, and wireless communication of the device 650.

[0152] The processor 652 can communicate with the user via a display interface 656 and a control interface 658 connected to a display 654. The display 654 can be, for example, a TFT LCD display or an OLED display, or other suitable display technology. The display interface 656 may include appropriate circuitry for driving the display 654 to present graphics and other information to the user. The control interface 658 may receive instructions from the user and convert the instructions for submission to the processor 652. In addition, an external interface 662 in communication with the processor 652 may be provided to enable near-area communication between the device 650 and other devices. The external interface 662 may provide, for example, wired communication (e.g., via a docking program) or wireless communication (e.g., via Bluetooth or other such technologies).

[0153] Memory 664 stores information within computing device 650. In one implementation, memory 664 is a computer-readable medium. In one implementation, memory 664 is a volatile memory unit or units. In another implementation, memory 664 is a non-volatile memory unit or units. Expansion memory 674 may also be provided and connected to device 650 via expansion interface 672, which may include, for example, a SIMM card interface. Expansion memory 674 may provide additional storage space for device 650 or store applications or other information for device 650. Specifically, expansion memory 674 may include instructions for executing or supplementing the processes described above and may also include security information. Thus, for example, expansion memory 674 may be provided as a security module for device 650 and may be programmed with instructions that enable secure use of device 650. Furthermore, secure applications and additional information may be provided via a SIMM card, such as identifying information placed on the SIMM card in an unhackable manner.

[0154] As described below, the memory may include, for example, flash memory and / or MRAM memory. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods (such as those described above). The information carrier is a computer or machine-readable medium, such as memory 664, expansion memory 674, or memory on processor 652.

[0155] Device 650 can communicate wirelessly via a communication interface 666, which may include digital signal processing circuitry, if necessary. Communication interface 666 can provide communication in various modes or protocols, such as GSM voice calls, SMS, EMS or MMS messaging, CDMA, TDMA, PDC, WGDMA, CDMA2000, or GPRS. This communication can occur, for example, via a radio frequency transceiver 668. In addition, short-range communication can occur, such as using Bluetooth, WiFi, or other such transceivers (not shown). In addition, a GPS receiver module 670 can provide additional wireless data to device 650, which can be used as appropriate by applications running on device 650.

[0156] Device 650 may also communicate audibly using audio codec 660, which may receive spoken information from a user and convert it into usable digital information. Audio codec 660 may similarly generate audible sounds for the user, such as through a speaker, for example, in a handset of device 650. This sound may include sounds from voice phone calls, may include recorded sounds (e.g., voice messages, music files, etc.), and may also include sounds generated by applications running on device 650.

[0157] As shown, computing device 650 can be implemented in a variety of different forms. For example, computing device 650 can be implemented as a cellular phone 680. Computing device 650 can also be implemented as part of a smart phone 682, a personal digital assistant, or other similar mobile device.

[0158] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that are executable and / or interpretable on a programmable system that includes at least one programmable processor, which can be either special purpose or general purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0159] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages and / or in assembly / machine language. As used herein, the terms "machine-readable medium," "computer-readable medium," and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0160] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of what may be claimed, but rather as descriptions of features that may be specific to a particular embodiment. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. Furthermore, although the features described above may be described in certain combinations as acting on certain combinations and even initially claimed, in some cases, one or more features from the claimed combination may be deleted from the combination, and the claimed combination may involve a subcombination or a variant of the subcombination.

[0161] Similarly, although the operations in the accompanying drawings are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in the sequential order shown, or that all illustrated operations be performed, in order to achieve the desired results. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated into a single software product or packaged into multiple software products.

[0162] Specific embodiments of the present subject matter have been described. Other embodiments are also within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method comprising: For a three-dimensional model of an object to be optimized, determining a plurality of points on the object, each point having at least a threshold likelihood of being a focus, the three-dimensional model having two or more regions, each region including one or more textures, one or more meshes, or both; identifying one or more non-focus regions from the two or more regions, wherein i) each non-focus region does not include any of the plurality of points, and ii) the one or more non-focus regions are a proper subset of the two or more regions; generating an optimized three-dimensional model for the object having a smaller size that is smaller than a larger size of the three-dimensional model using the one or more non-focal areas; and storing the optimized three-dimensional model in a non-volatile memory, Wherein, for each of the one or more non-focal regions, reducing the quality of the one or more textures, the one or more meshes, or both included in the corresponding non-focal region from the quality of the corresponding one or more textures, the one or more meshes, or both in the three-dimensional model comprises: For each of the one or more non-focal regions, a resolution of each of the one or more textures, one or more meshes, or both included in the corresponding non-focal region is reduced from a higher resolution of the corresponding one or more textures, one or more meshes, or both in the three-dimensional model.

2. The method according to claim 1, comprising: receiving, by the system and across the network, a request for a model of the object; as well as In response to receiving the request for the model of the object, the optimized three-dimensional model of the object is transmitted to a device and using the network.

3. The method according to claim 2, comprising: After sending the optimized three-dimensional model having the smaller size to the device, determining to send the three-dimensional model having the larger size to the device; as well as In response to determining to send the three-dimensional model having the larger size to the device, the three-dimensional model is sent to the device.

4. The method according to claim 3, wherein: Determining to send the three-dimensional model having the larger size to the device includes determining that a network connection across the network and between the system and the device has a usage less than a threshold; as well as The three-dimensional model is sent to the device in response to determining that the network connection across the network and between the system and the device has a usage less than the threshold.

5. The method according to claim 3 or claim 4, wherein: Determining to send the three-dimensional model having the larger size to the device includes receiving a request for the three-dimensional model having the larger size; and The three-dimensional model is sent to the device in response to receiving a request for the three-dimensional model having the larger size.

6. A method according to any one of the preceding claims, wherein: identifying the one or more non-focus regions from the two or more regions comprises identifying one or more textures, one or more grids, or one or more quadrants as the one or more non-focus regions; as well as Generating the optimized three-dimensional model includes generating the optimized three-dimensional model for the object having a smaller size than a larger size of the three-dimensional model using the identified one or more textures or the identified one or more grids or the identified one or more quadrants.

7. A method according to any one of the preceding claims, wherein Determining the plurality of points on the object that each have at least a threshold likelihood of being a focus comprises: fetching data for a plurality of images of the object from a memory, each image depicting at least a portion of a view of the object generated on a display for presentation to a viewer, determining one or more potential focal points for each image from the plurality of images; and The potential focus points each having at least the threshold likelihood of being in focus are selected from the one or more potential focus points for the plurality of images and are used as the plurality of points.

8. The method according to claim 7, wherein: Selecting each of the potential focal points having at least the threshold likelihood of being a focal point comprises: i) selecting a first subset of potential focal points from the one or more potential focal points depicted in a first image of the plurality of images, ii) each potential focal point in the first subset having at least the threshold likelihood of being in focus; and a) determining to skip selection of a second subset of potential foci from the one or more potential foci depicted in the first of the plurality of images, b) each potential foci in the second subset not having at least the threshold likelihood of being in focus.

9. The method according to claim 7 or claim 8, wherein: Each of the potential focal points comprises an estimated point in the corresponding image from the plurality of images on which a viewer viewing the presentation of the object on the display is likely to focus.

10. The method according to any one of claims 7 to 9, wherein Selecting, from the one or more potential focal points of the plurality of images, each potential focal point having at least the threshold likelihood of being in focus comprises: At least one of the one or more potential focal points is weighted using a distance of the potential focal point from a center of the corresponding image, wherein a first potential focal point that is closer to the center of the corresponding image has a higher weight than a second potential focal point that is farther from the center of the corresponding image.

11. The method according to any one of claims 7 to 10, comprising: receiving data for one or more of the plurality of images from a device that presents a model of the object on the display and across a network; as well as The data for one or more images of the plurality of images is stored in the memory.

12. The method according to any one of claims 7 to 11, wherein Determining the one or more potential focal points for each image from the plurality of images comprises: For each image from the plurality of images and from a direction indicated by a camera that would generate the corresponding image, casting one or more rays onto the object; and For each of the one or more rays, a point where the ray intersects the object is selected as the corresponding focus.

13. The method according to claim 12, wherein: Casting the one or more rays onto the object for each image from the plurality of images and from a direction indicated by a camera that would generate the corresponding image includes: For each image from the plurality of images: determining one or more regions within which to generate rays; and For each of the one or more regions, a ray is randomly generated that is cast onto the object.

14. The method according to claim 13, wherein For each image from the plurality of images: Determining the one or more regions within which the ray is generated includes determining, for each of the one or more regions, a range of angular deviations from a reference point within which the ray is generated; as well as For each of the one or more regions, randomly generating the ray projected onto the object comprises: for each of the one or more regions, Randomly selecting an angle deviation within the angle deviation range; and Generates a ray at the randomly selected angular deviation.

15. The method according to claim 14, wherein The reference point includes the position of the camera that will generate the corresponding image.

16. The method according to claim 14 or 15, wherein: Determining the angular deviation range for each of the one or more regions includes: For each of the one or more regions, a size of the angular deviation range is determined using a distance of the region from a center of the corresponding image.

17. The method according to any one of claims 7 to 16, wherein Selecting, from the one or more potential focal points of the plurality of images, each potential focal point having at least the threshold likelihood of being in focus comprises: For one or more points on the object, determining a number of times the point is a potential focus of a corresponding image; and The potential focal points for which the correspondence amount satisfies a threshold amount are selected as the plurality of points.

18. The method according to claim 17, wherein Selecting the potential focal points where the corresponding amount satisfies a threshold amount as the plurality of points comprises: for the one or more points on the object, determining a normalized amount using the highest number of times a point on the object is a potential focus; and The potential focal points whose corresponding normalized quantities meet a threshold are selected as the multiple points.

19. A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform the method of any preceding claim.

20. A system comprising one or more computers and one or more storage devices storing instructions which, when executed by the one or more computers, are operable to cause the one or more computers to perform the method according to any one of claims 1 to 18.

Images