Digital image display method, enhanced two-dimensional image creation method and computing system

Abstract

The invention relates to a digital image display method, an enhanced two-dimensional image creation method and a computing system. A method for digital image display and interaction includes receiving an enhanced two-dimensional (2D) image including an image dataset derived from a digital object representation including spatial data representing a three-dimensional (3D) model of an object. A 2D display image included in the image dataset is displayed that depicts the object from the virtual camera location. A user input associated with selected pixels in the 2D display image having image space coordinates within the enhanced 2D image is received. 3D object spatial coordinates of a selected point on the object corresponding to the selected pixel are calculated based at least in part on the image spatial coordinates of the selected pixel, the depth value of the selected pixel, the model view transformation matrix, and the projection matrix included in the image dataset. And displaying the 3D object space coordinates of the selected points.

Description

TECHNICAL FIELD

[0001] The present invention relates generally to image rendering techniques, and more specifically to techniques for computing three-dimensional (3D) coordinates corresponding to selected pixels of an augmented two-dimensional (2D) image of an object. BACKGROUND

[0002] Three-dimensional (3D) spatial data - e.g., in the form of a digital model corresponding to a physical object - can be used to digitally represent a physical object. Further, digital objects can be generated that do not correspond to existing real-world physical objects. Such data can be rendered, viewed, and manipulated by suitable 3D rendering applications. SUMMARY

[0003] This summary is not an extensive overview of the specification. It is neither intended to identify key or critical elements of the specification nor to delineate any scope of any of the embodiments or claims. Its sole purpose is to present some concepts of the specification in a simplified form as a prelude to the more detailed description that is presented in the body of the disclosure.

[0004] A method for digital image rendering includes receiving, at a computing device, an augmented two-dimensional (2D) image, the augmented 2D image including an image data set derived from a digital object representation, the digital object representation including spatial data representing a three-dimensional (3D) model of an object. Displaying a 2D display image included in the image data set, the 2D display image depicting the object from a virtual camera position. Receiving user input for a selected pixel in the 2D display image having image space coordinates within the augmented 2D image. Computing 3D object space coordinates of a selected point on the object corresponding to the selected pixel based at least in part on the image space coordinates of the selected pixel, a depth value of the selected pixel, a model view transformation matrix, and a projection matrix included in the image data set. Displaying the 3D object space coordinates of the selected point.

[0005] The features, functions, and advantages that have been discussed can be implemented independently in various embodiments or can be combined in other embodiments, further details of which can be seen with reference to the following description and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 Different two-dimensional (2D) display images of a three-dimensional (3D) digital object are illustratively described.

[0007] Figure 2 An exemplary method for augmented 2D image creation is shown.

[0008] Figure 3 Generating an image data set based on a digital object representation including spatial data of an object is illustratively shown.

[0009] Figure 4 An exemplary method for displaying digital images is shown.

[0010] Figure 5 The illustration schematically shows a client computing device accessing an enhanced 2D image from a server computing device.

[0011] Figures 6A to 6C The illustration shows how to interact with an enhanced 2D image and calculate and display the 3D coordinates corresponding to a selected pixel in the 2D display image.

[0012] Figure 7 A schematic diagram of the component identification diagram corresponding to the 2D display image of the object is shown.

[0013] Figure 8 The illustration depicts the display of the 3D distance between two selected points in a 2D display image.

[0014] Figure 9 An example of an enhanced 2D image file is illustrated schematically.

[0015] Figure 10 An exemplary method is shown for displaying metadata of selected pixels and measuring 3D object spatial distances between selected points.

[0016] Figure 11 An example computing system is illustrated schematically. Detailed Implementation

[0017] In various industries such as aerospace, automotive, architecture, and manufacturing, examining three-dimensional (3D) spatial data corresponding to digital object models can be useful. For example, 3D object models can be rendered and viewed in suitable 3D rendering applications. However, traditional methods of accessing 3D data often involve complex software setups, significant hardware requirements, and potential security risks associated with the transfer of sensitive 3D model data. For instance, current solutions may include the use of 3D visualization applications that require high-performance hardware, limiting accessibility for users with low-end devices. Furthermore, these solutions can be cumbersome due to long loading times and performance degradation as the complexity of 3D scenes increases. These types of 3D visualization systems also require users to learn how to use 3D spatial navigation controls, which can be difficult for some users, especially those unfamiliar with the application. Additionally, other existing solutions for deriving 3D data from two-dimensional (2D) images may require the use of isometric or isometric plots, where the same scale factor is used for each principal axis. Other methods utilize significant amounts of storage to store the 3D coordinate data or metadata for each pixel.

[0018] Therefore, this disclosure relates to a method for deriving 3D information from an enhanced 2D image representing a 3D object. For example, a 2D display image of an object can be viewed in a web browser application or other suitable application for image viewing, advantageously without requiring high-performance hardware. However, when guiding user input at a specific selected pixel on the 2D display image, a computing system can calculate the 3D coordinates of the corresponding selected point on the surface of the 3D object. In other words, by saving the visualization data (e.g., an enhanced 2D image) in a form that allows for subsequent viewing and interaction, alternative forms of viewing and interacting with 3D data are possible using the visualization data created during the initial 3D rendering process, without requiring a 3D rendering application.

[0019] More specifically, according to this disclosure, an image dataset is generated based on spatial data representing a 3D model of a digital object. The image dataset includes a 2D display image of the object rendered from a specific virtual camera position, a depth map, a projection matrix, and a model-view transformation matrix derived from the 3D model and the virtual camera position. The different components of the image dataset can be collectively referred to as an augmented 2D image. The model-view transformation matrix is ​​a combination of a model transformation matrix used to locate the 3D model in a 3D environment and a camera-view transformation matrix. When a user inputs a selected pixel in a 2D display image of the augmented 2D image, having image spatial coordinates within the 2D display image (e.g., pixel coordinates within a 2D pixel grid of the display image), the associated depth values ​​of the pixels in the depth map, the model-view transformation matrix, and the projection matrix are used to calculate the 3D coordinates of the corresponding point on the object's surface. These 3D coordinates can then be displayed for viewing. In some examples, the image dataset may additionally include other suitable types of information—e.g., metadata about object-specific information associated with the selected object (such as part name, part number), additional model-view transformations (e.g., for alternative coordinate systems), and / or additional images of objects from other locations, etc.

[0020] In this way, the technique described herein advantageously enables users to access 3D information related to digital objects, even when viewing 2D images depicting those objects. This advantageously reduces the need for users to use specialized rendering applications to locally render detailed 3D models. Instead, the enhanced 2D images can be displayed in lightweight applications such as web browsers or other suitable applications for image viewing. Furthermore, the technique described herein advantageously enables the use of lower-end hardware devices and improves data security by eliminating the need to transfer 3D models over a network to client computing devices for rendering, and reduces the training required for users to use the application.

[0021] In some examples, the techniques discussed herein can be used to generate isorectangular images that can be used to form a panoramic display (e.g., a 360-degree representation of a 3D scene). In some examples, the techniques discussed herein can be used to implement a front-end interface (e.g., accessible via an application programming interface) to a downstream service that performs additional operations on 3D coordinates and / or metadata. The techniques discussed herein can advantageously enable the computation of 3D data from 2D images with projection (e.g., perspective contraction), such as images generated by a 3D visualization application using a projection virtual camera setup and / or digital photographs taken with a depth camera.

[0022] Figure 1 The illustration schematically demonstrates the concept of different 2D display images for displaying 3D digital models. Specifically, Figure 1 A 3D dataset 100 representing a digital model 102 is schematically shown. It will be understood that a “3D dataset” or “3D model” as described herein can include any suitable data relating to a 3D model of an object. Such data can be formatted in any suitable manner and can be expressed at any suitable level of detail. In some examples, a 3D dataset represents a digital object modeled using a suitable 3D modeling or sculpting application. Additionally or alternatively, a 3D dataset can correspond to a digital representation of a physical object. For example, in some examples, a 3D dataset can include or can be derived from data generated by a 3D scanning operation applied to a physical object in the real world.

[0023] Generally speaking, a 3D dataset refers to a digital representation of an object or scene in three dimensions, including width, height, and depth. It can be constructed using vertices, edges, and faces within a coordinate system, allowing for detailed and interactive visualization from any camera viewpoint (e.g., including camera position and orientation within a 3D environment). 3D models are commonly used in applications such as computer-aided design (CAD), virtual reality, games, and simulations.

[0024] In contrast, 2D images of a 3D model are projections of the 3D model onto a two-dimensional surface. These images capture a single view of the 3D model, much like photographs. While they can convey information about how the model looks from that particular viewpoint, they typically lack the depth and interactivity inherent in the original 3D model. For example, in general, 2D images are static and cannot be manipulated to show different viewing positions without generating new images from the 3D model.

[0025] It will be understood that the specific shape and appearance of the digital object 102 are not limiting. Conversely, for the purpose of describing the calculation of 3D coordinates, Figure 1The digital object 102 shown, as well as the other figures discussed herein, are intended only as a general representation of a digital object that can be rendered as a 2D image. It will be understood that the techniques described herein can be applied to any of a wide variety of suitable digital models that can have any arbitrary size, shape, and purpose.

[0026] As will be discussed in more detail below, a digital model can in some cases include two or more sub-parts within the overall component. In some cases, when a user selects a specific pixel in a 2D display image depicting a given sub-part, information about that sub-part can be displayed. Figure 1 In the example, the general-purpose digital object 102 includes various sub-components, including two rectangular sub-components (one of which is labeled sub-component 106A) and three circular sub-components (one of which is labeled sub-component 106B). It will be understood that, like the digital object itself, these sub-components are non-limiting instances and are used only as a general representation of the structural types to which the techniques discussed herein can be applied.

[0027] exist Figure 1 The image shows two distinct 2D display images 104A and 104B depicting a digital object 102. Each of these 2D images depicts the digital model from a different virtual camera position—similar to photographs of a physical object taken from different angles. These 2D images can take the form of digital images, each comprising multiple pixels arranged in a two-dimensional grid. Therefore, any particular location within a 2D image can be defined by a set of image space pixel coordinates, such as the X and Y coordinates within the 2D pixel grid.

[0028] Similar to the digital object itself, it will be understood that the 2D display image included in the enhanced 2D image depicting the object can take any suitable form and have any suitable appearance. For example, during image rendering, any suitable lighting and / or texture can be applied, thus affecting the appearance of the digital object within the 2D display image. It is worth noting that the 2D display image can be rendered to simulate the lighting of the digital object under any suitable lighting conditions (e.g., any suitable intensity, direction, and / or spectrum of the illumination light). The appearance of the object in the 2D display image can be similar to or may not be similar to how the physical counterpart of the digital object would appear in the real world. For example, in some examples, the 2D display image can be rendered to have a realistic appearance. This can include rendering the 2D display image using ray-traced lighting effects. In other examples, the 2D display image can be rendered using settings that do not produce realistic results (e.g., using lighting, color, and / or texture that differ from the actual or hypothetical real-world appearance of the corresponding physical object). In other words, non-realistic rendering methods can also be used to create 2D display images, such as false-color or hidden-line removal rendering techniques. This can include additional overlay elements, such as text or lines that are not selectable but will be displayed as reference information in the 2D display image.

[0029] Augmented 2D images can be relatively smaller than 3D datasets in terms of memory usage and file size, as they comprise fewer total bytes of data. This advantageously makes augmented 2D images easier to transfer over computer networks and store in computer memory compared to 3D models. Furthermore, augmented 2D images can be relatively less resource-intensive to render for display and require fewer specialized software applications for viewing. However, as a drawback, traditional 2D images typically lack information related to the 3D model of the object, as discussed above. The techniques discussed in this paper advantageously enable the computation and display of some of this 3D information as the user views and interacts with the augmented 2D image.

[0030] therefore, Figure 2 An exemplary method 200 for creating enhanced 2D images is illustrated. The steps of method 200 can be started, terminated, and / or repeated at any suitable time and in response to any suitable conditions. The steps of method 200 can be implemented by any suitable computing system using one or more computing devices. Any computing device implementing the steps of method 200 can have any suitable capabilities, hardware configuration, and form factor. In some examples, method 200 may be implemented using the following references... Figure 11 The described computing system 1100 is used to implement this.

[0031] At 202, method 200 includes receiving a digital object representation comprising spatial data of a 3D model representing an object. (See above regarding...) Figure 1The 3D spatial data datasets discussed can be formatted in any suitable manner and can be expressed at any suitable level of detail. In some examples, a 3D dataset represents a digital object modeled using a suitable 3D modeling or sculpting application. Additionally or alternatively, a 3D dataset may correspond to a digital representation of a physical object. For example, in some examples, a 3D dataset may include or can be derived from data generated by a 3D scanning operation applied to a physical object in the real world.

[0032] Typically, a digital object representation can be "received" in any suitable manner and from any suitable source. In some examples, receiving a digital object representation may include loading a digital object representation from a computer storage device (e.g., integrated storage hardware in a computing device, and / or a removable storage device communicatively coupled to the computing device). In some examples, receiving a digital object representation may include receiving a digital object representation from another computing device via a computer network. Such a computer network may include a local area network (LAN) and / or a wide area network (WAN) such as the Internet.

[0033] At 204, method 200 includes rendering a 2D display image representing a digital object from the virtual camera position. (As mentioned above regarding...) Figure 1 The process discussed involves generating any suitable number of different 2D display images describing a given digital object from any of a variety of different virtual camera positions. This process is relative to... Figure 3 To illustrate, Figure 3 A digital object representation 300 is shown, comprising a set of spatial data 302 representing a 3D model of an object. This is fed into an image rendering process 304, which generates a 2D display image depicting the digital object from a given virtual camera position 306, resulting in a 2D display image 308. The 2D display image comprises a plurality of image pixels 310, which can be arranged as a 2D pixel grid, as described above.

[0034] Generally, generating a 2D display image may include first determining the desired virtual camera position from which the digital object should be depicted (e.g., by interacting with a suitable 3D model viewer application), and determining the desired lighting conditions for the simulated lighting of the digital object. Rendering may then include transforming the model's 3D coordinates into 2D coordinates using a projection matrix. One form of this process uses perspective shortening to transform the model's three-dimensional points into a two-dimensional plane, simulating how the human eye perceives depth. Alternatively, orthographic projection may be used in different forms that do not involve the use of a shortened projection matrix. It will be understood that 2D images can be generated from 3D digital models in any of a variety of suitable ways (e.g., using any of a variety of different graphics rendering pipelines), and the techniques discussed herein are agnostic to the specific process used to render 2D images from 3D digital objects.

[0035] exist Figure 3 In the example, 2D display images are included in image dataset 312. As used herein, an "image dataset" includes 2D display images depicting digital objects from virtual camera positions and projection settings, and also includes additional data that can be used to calculate 3D coordinates corresponding to selected pixels in the 2D display images. This dataset may also be referred to as augmented 2D images. Additional examples of data that may be included in an image dataset will be described in more detail below. In some cases, the different components of the image dataset discussed herein may be saved together as a single data file. Alternatively, aspects of the image dataset may be distributed among two or more separate data files. As with other datasets described herein, it will be understood that the data included in an image dataset may take any suitable form and may be expressed and formatted in any suitable manner.

[0036] Briefly return Figure 2 At 206, method 200 includes generating a depth map comprising multiple depth values ​​corresponding to multiple pixels in a 2D display image. Figure 3 In this image dataset 312, a depth map 314 corresponding to the 2D display image 308 is further included. The depth map includes multiple depth values ​​316, each corresponding to a multiple image pixel 310 of the 2D display image. In other words, for each pixel in the 2D display image, the depth map includes multiple values ​​indicating the distance between that portion of the digital object and the position of the virtual camera. Pixels in the 2D display image depicting portions of the digital object that are relatively close to the virtual camera are associated with relatively smaller depth values ​​in the depth map. Conversely, pixels in the 2D display image depicting portions of the digital object that are relatively far from the virtual camera are associated with relatively larger depth values.

[0037] Depth maps can be generated in any suitable manner. Generally, during the rendering of a 3D model, for each pixel in the 2D display image, the rendering engine can calculate the distance from the camera to the surface of the 3D model along the virtual camera's viewpoint. This may involve transforming the model's 3D coordinates into the camera's coordinate system and determining the depth value for each visible point. The calculated depth values ​​can be stored in a depth buffer, where each entry corresponds to a pixel in the 2D display image. In some cases, depth values ​​may be normalized to a specific range (e.g., 0 to 1) to facilitate visualization and processing. This step may involve scaling the depth values ​​based on the minimum and maximum depths in the scene. The depth buffer can then be used to generate a depth map, which is included in the image dataset along with the corresponding 2D display image.

[0038] Brief Return Figure 2At 208, method 200 includes generating an image dataset for digital object representation, the image dataset including 2D display images, depth maps, model-view transformation matrices, and projection matrices. Figure 3 In the image dataset 312, additionally, a model-view transformation matrix 318 and a projection matrix 326 (which will be described in more detail below) are included. The model-view transformation matrix is ​​a combination of a model transformation matrix used to position (in location and orientation) a 3D model in a 3D environment and a camera-view transformation matrix used to set the position (location and orientation) of a virtual camera in a 3D virtual environment. The projection matrix defines the corners of a view volume (e.g., clip space) that is projected downwards into 2D space and rasterized into a 2D image. The projection matrix can be a "perspective projection matrix" (when using perspective shortening rendering) or an "orthographic projection matrix" (when using orthographic projection rendering). It will be understood that the techniques discussed herein can be applied to both perspective and orthographic projection.

[0039] As discussed above, during the rendering of a 2D display image, a projection matrix is ​​used to transform the spatial coordinates of 3D objects into 2D pixel coordinates. This projection matrix is ​​derived from various parameters, such as the virtual camera's field of view (FOV), aspect ratio (e.g., the ratio of the width to the height of the viewing area), near clipping plane, and far clipping plane. These parameters define how the 3D scene is projected onto the 2D view. The projection matrix and / or information that can be used to recreate the projection matrix are stored in the image dataset along with the 2D display image. Therefore, as will be described in more detail below, the projection matrix can be inverted to calculate the 3D coordinates corresponding to any given pixel in the 2D display image.

[0040] Depending on the implementation, image datasets may, in some cases, include additional or alternative types of data to those discussed above—for example, 2D display images, model-view transformation matrices, projection matrices, and depth maps. For instance, as discussed above, digital objects may, in some cases, comprise two or more sub-parts. Therefore, in some cases, image datasets may include part identification maps, where different corresponding pixel colors are used to depict the pixels of two or more sub-parts. It is worth noting that the part identification map does not need to be rendered for display, but can be a separate image used as a reference to determine which sub-parts are depicted by different image pixels in the 2D display image. Figure 3 In the example, the image dataset includes a component identification map 320, which itself includes multiple component identification values ​​322. These are in the form of pixel values ​​of the component identification map, which may have values ​​different from the image pixels of the 2D display image.

[0041] Part identification maps can be generated in any suitable manner. In some examples, part identification maps can be generated via a second rendering process, where lighting conditions are altered relative to the initial image rendering process used to generate the 2D display image. In a non-limiting approach, after generating the 2D display image, the computing system may then assign distinct unique colors to each distinct sub-part (or each different kind of sub-part) in the digital object. Notably, these distinct unique colors may differ from the colors used to represent the sub-parts in the previously rendered 2D display image and may differ from the actual appearance of any corresponding real-world physical object. Next, the computing system may render a second 2D image depicting the digital object with the respective sub-parts assigned distinct unique colors, and subsequently render only with an enabled ambient lighting rendering mode (e.g., with diffuse, specular, and emissive lighting modes disabled, as well as textures, shadows, etc.). This can result in pixels depicting a given sub-part each having the same color value, and different color values ​​used for each distinct sub-part in the image. For example, each pixel depicting the first sub-part may have the same red color value, each pixel depicting the second sub-part may have the same blue color value, and so on. The second 2D image does not need to be displayed for viewing, but can be used to create a part identification map, where each unique color can be associated with an index value stored in the part identification map.

[0042] In some cases, a component identification map can also be associated with a component identification table listing identifiers for each distinct sub-component and a color value (or index value) uniquely associated with that sub-component in the component identification map. Thus, upon receiving user input for a selected pixel, the computing system can refer to the component identification map to determine the color value for the corresponding pixel in the component identification map, and then refer to the component identification table to determine the identifier for the sub-component associated with that color value. In some cases, such a component identification table may include any suitable information in addition to color values ​​and sub-component identifiers. In other words, a component identification map can be used to resolve each pixel of a 2D display image to a specific sub-component of an object.

[0043] As described above, an image dataset includes model-view transformation matrices for rendering 2D display images from 3D digital objects. However, in some cases, an image dataset may include more than one model-view transformation matrix for the same 2D display image. For example, in Figure 3In this example, image dataset 312 further includes a second model-view transformation matrix 326, also corresponding to the 2D display image 308. This may correspond, for example, to a second coordinate system. In a non-limiting example, the digital object represents a component of an aircraft. The first model-view transformation matrix can calculate the 3D coordinates of selected pixels in the 2D display image based on, for example, a first coordinate system defined relative to the entire aircraft. However, some subsystems of the aircraft may have their own corresponding coordinate systems, which are sometimes used in, for example, assembly instructions, repair instructions, design documents, and / or other documents. Therefore, the second model-view transformation matrix can be used to calculate an alternative set of 3D coordinates for the same pixels in the 2D display image, which is defined relative to an alternative coordinate system (such as a coordinate system specific to the aircraft wing). It will be understood that this example is non-limiting, and multiple coordinate systems can be used for digital objects in fields other than aerospace. Furthermore, the image dataset may include any suitable number of one or more different model-view transformation matrices corresponding to any suitable number of one or more different coordinate systems. Alternatively, different units of measurement can also be supported by including one or more optional scaling factors in the image dataset for multiplying the translation elements of the model-view matrix to enable different units of measurement (e.g., inches, feet, meters, etc.).

[0044] In addition, Figure 3 In the example, the image dataset includes text metadata 328. This can take any suitable form and can include any arbitrary text corresponding to digital objects. In this way, as will be discussed in more detail below, when a specific pixel in a 2D image is selected, the computing system can retrieve the corresponding metadata associated with the digital object. This can include, for example, a text description of the selected part, repair instructions, assembly instructions, personal notes, contact information, hyperlinks, and / or any other suitable data. In some cases, hyperlinks included in the text metadata can enable information to be passed to an application programming interface (API)—for example, configured to receive 3D coordinates associated with a selected pixel in a 2D display image, and then perform downstream operations on those 3D coordinates. Nevertheless, the selection of different objects can be associated with different types of text metadata, and different types of text metadata can be displayed when pixels depicting those objects are selected in an augmented 2D image.

[0045] This disclosure has so far focused on generating image datasets from digital object representations that include 3D data. Now it turns to... Figure 4Another exemplary method 400 for image display will be described, focusing on user-side interaction with the enhanced 2D image generated as described above. Similar to method 200, the steps of method 400 can be initiated, terminated, and / or repeated at any suitable time and in response to any suitable conditions. The steps of method 400 can be implemented by any suitable computing system of one or more computing devices. Any computing device implementing the steps of method 400 can have any suitable capabilities, hardware configuration, and form factor. In some examples, method 400 can be implemented by the following references... Figure 11 The described computing system 1100 is implemented.

[0046] At 402, method 400 includes receiving an image dataset derived from a digital object representation. As discussed above, the digital object representation includes spatial data representing a 3D model of a digital object. The image dataset includes a 2D display image depicting the digital object from a virtual camera position, a depth map including multiple depth values, and a model-view transformation matrix and projection matrix used when rendering the 2D display image. Additionally, as discussed above, the image dataset may include various other types of data, such as part identification maps, where different corresponding pixel colors are used to depict pixels of two or more sub-parts in the digital object; part identification tables; one or more additional model-view transformation matrices for an alternative coordinate system; textual metadata; and so on.

[0047] Similarly, as discussed above, image datasets can be received from any suitable source in any suitable manner. For example, receiving an image dataset may include loading the image dataset from computer memory, such as integrated storage devices or removable storage devices. Alternatively, in some instances, the image dataset may be received via a computer network. For example, the computing device implementing method 400 may be a client computing device, and the image dataset may be received from a server computing device via a computer network. In some cases, the client computing device may use a web browser and / or other suitable software applications to access and view a 2D display image of the image dataset.

[0048] Figure 5 This scenario is illustrated schematically, showing an example server computing device 500. The server computing device stores an image dataset 502, which is generated based on a 3D digital model as discussed above. The server computing device is communicatively coupled to a computer network 504, through which a client computing device 506 accesses the image dataset from the server computing device. In other words, in Figure 5In this context, the client computing device receives an image dataset from the server computing device via computer network 504. The client computing device executes a web browser application 508, which displays 2D images 510 of digital objects for the user to view. It will be understood that the server computing device and the client computing device can each take any suitable form. In some examples, the server computing device and / or the client computing device can be implemented as follows regarding... Figure 11 The described computing system 1100. Server computing devices may include workstation computers, laptop computers, rack-mounted servers, or any computing device capable of rendering 3D models. As a non-limiting example, client computing devices may include workstation computers, laptop computers, tablet computers, smartphones, smartwatches, smart TVs, embedded microprocessor-based computers, or other computers capable of displaying data and accepting user input.

[0049] Briefly return Figure 4 At 404, method 400 includes displaying a 2D display image included in an image dataset, the 2D display image depicting a digital object from a virtual camera position. At 406, method 400 includes receiving user input for selected pixels in the 2D display image having image space coordinates within the 2D display image. The 2D display image can be displayed in any suitable manner using any suitable computer display technology. Similarly, user input can be detected using any suitable modality for providing user input to a computing device. For example, user input can be provided using a computer mouse, keyboard, touchscreen, gesture interface, eye-tracking system, etc.

[0050] Figure 6A The description is shown schematically. Figure 1 An example 2D display image 600 of the digital object 102. The 2D display image is generated based on a digital object representation that includes 3D spatial data of the digital object, as discussed above. The 2D display image 600 is displayed using a suitable computer monitor, and the user has provided user input 602 to a specific location within the 2D pixel grid of the 2D display image. In this way, the user has provided user input to select a specific pixel or a group of two or more pixels within the 2D display image 600. Each pixel in the image is associated with a different set of image space pixel coordinates within the 2D pixel grid. Therefore, in Figure 6A In this context, the selected pixel is associated with image space coordinates 604.

[0051] Briefly return Figure 4At 408, method 400 includes calculating the 3D object space coordinates of a selected point on the object corresponding to the selected pixel, based at least in part on the image space coordinates of the selected pixel, the depth value corresponding to the selected pixel in the depth map, the model-view transformation matrix, and the projection matrix included in the image dataset. This is relative to... Figure 6B To illustrate, Figure 6B The image space coordinates 604 input to the 3D coordinate calculation process 606 are shown again. This uses the depth value 608 corresponding to the selected pixel, and the inverse projection matrix 610 derived from the projection matrix and the model-view transformation matrix, to give the 3D object space coordinates 612 of the selected point on the surface of the digital object corresponding to the selected pixel.

[0052] As discussed above, in some examples, the image dataset may include two or more different model-view transformation matrices and projection matrices corresponding to different coordinate systems. This is Figure 6B In the case described above, image space coordinates 604 are also input into the second 3D coordinate calculation process 614, which uses depth value 608 and a second inverse projection matrix 616 (e.g., inversely transformed from the second projection matrix and second model-view transformation matrix used in 2D display image rendering) to give a second set of 3D object space coordinates 618. In other words, the calculation system calculates the second 3D coordinates corresponding to the same second point on the object relative to a second coordinate system and using the second inverse projection matrix. For example, as a non-limiting example, the first set of coordinates 612 can be defined relative to the entire aircraft, while the second set of coordinates 618 can be defined relative to the wings of the aircraft.

[0053] Briefly return Figure 4 At 410, method 400 includes displaying the 3D object space coordinates of the selected point. This is relative to... Figure 6C Schematic illustration shows, again, a 2D display image 600 of an enhanced 2D image. Based on user input 602, the computing system now displays different types of information about the selected pixel. In this example, this includes the pixel's 2D image space coordinates 620, and the selected point's 3D object space coordinates 622 on the object's surface. It should be understood that in Figure 6C The manner in which information is displayed and the specific content of the information are non-limiting and are for illustrative purposes only. For example, the specific numbers used for 2D image space coordinates and 3D object space coordinates are arbitrarily chosen and are used only as non-limiting examples.

[0054] As discussed above, in some examples, a digital object comprises two or more distinct sub-components. After selecting the pixels to depict a particular sub-component, the computing system can display information specific to that sub-component. This is Figure 6CIn the case where user input 602 involves pixels depicting a rectangular sub-part 106A, the sub-part's identifier can be determined at least partially based on the part identifier map, as described below relative to... Figure 7 As described. After determining the selected sub-part represented by the selected pixels involved by the user input, the computing system displays the corresponding identifier 624 of the selected sub-part (e.g., the front panel in this non-limiting example).

[0055] Furthermore, in some cases, the appearance of a selected sub-part in a 2D display image can be changed after selection. In other words, after the selected sub-part is determined, the computing system can change the pixel value of each pixel depicting the selected sub-part in the 2D display image to thereby highlight the selected sub-part within the 2D display image. This is Figure 6C In the case of sub-component 106A, the selection of sub-component 106A has caused the computing system to change the pixel values ​​used to represent the sub-component (e.g., compared to...). Figure 1 Compared to the appearance of digital object 102 in the image. As one way, a highlighting effect can be produced by using a selected area of ​​the component identification map to create a mask that blends with the 2D display image, thereby changing the color of the pixels depicting the selected sub-components.

[0056] In this example, the pixel values ​​of other rectangular sub-components are also changed. In other words, selecting sub-component 106A has already caused the highlighting of every sub-component of the same type. In other examples, selecting a specific sub-component may only cause the highlighting of that specific sub-component without changing the appearance of other sub-components of the same type. In some examples, this can be a user-adjustable setting—for example, whether sub-component selection highlights only the selected instance or all instances of the same type.

[0057] In addition, Figure 6C In this context, the selection of pixels also causes the computing system to display text metadata 626 associated with the selected object. As discussed above, such text metadata can include any arbitrary text data used to express any suitable information. Figure 6C In this non-restricted example, the object represents an aircraft part, and after receiving user input, the system displays part-specific information related to the aircraft part. This includes part identifier 627. Furthermore, in Figure 6C In the text metadata, repair instructions and hyperlinks associated with the user manual are included (628).

[0058] Figure 7 A schematic diagram illustrating a component identification map corresponding to an enhanced 2D image is provided. It will be understood that the "component identification map" described herein is not required (and generally will not) to be displayed for viewing. Instead, it takes the form of a data structure that can be referenced to identify sub-components corresponding to specific selected pixels in the 2D display image. Specifically,Figure 7 This includes another example 2D display image 700 depicting a digital object 102 comprising sub-components 106A and 106B. Furthermore, Figure 7 A visual representation of a component identification diagram 702 corresponding to the 2D display image 700 (which is not required to be displayed to the user) is shown. As described above, in the component identification diagram, different corresponding pixel colors are used to depict the pixels of two or more sub-components. For example, a rectangular sub-component instance (e.g., including sub-component 106A) uses... Figure 7 The first fill pattern in the text, while the circular sub-part instance (e.g., including sub-part 106B) uses... Figure 7 Different second fill modes are available. It is worth noting that the part identification map does not need to be rendered for display; instead, it can be a separate representation of data used as a reference to determine which sub-parts are depicted by highlighted image pixels in the 2D display image. For example, as discussed above, selecting a specific pixel in the 2D display image allows the computing system to identify the color value (or index value) of the corresponding pixel in the part identification map. Based on this color value (or index value) along with the part identification table, the computing system can determine the identification of the sub-part depicted by the selected pixel.

[0059] So far, this disclosure has primarily focused on selecting a single pixel corresponding to a single selected point on the surface of an object in a 2D display image. However, in some examples, two or more different pixels corresponding to two or more points on an object or points on two separate objects in a 3D environment can be selected. The techniques described herein advantageously enable the calculation and display of information relating to the 3D relationship between the two selected points. In other words, in some examples, the computing system can receive a second user input relating to a second selected pixel in a 2D display image, calculate the second 3D object space coordinates of the second selected point on the object corresponding to the second selected pixel, and display the second 3D object space coordinates of the second selected point. The computing system can then, in some cases, calculate the 3D object space distance between the first and second selected points and display the calculated 3D object space distance. Any suitable technique can be used to calculate the 3D object space distance. As a non-limiting example, the 3D object space distance can be calculated using the Euclidean norm (e.g., the square root of the sum of squares of the differences between the individual 3D coordinate components).

[0060] This is relative to Figure 8A further example 2D display image 800 of a digital object 102 is illustrated schematically. In this example, user input has previously been provided to select a first selected pixel 802A. Now, user input is provided to select a second selected pixel 802B. The calculation system is displaying the 2D pixel coordinates 804 of the second selected pixel, and the calculated 3D object space coordinates 806 of the second selected point on the object depicted by the second selected pixel. Additionally, the calculation system is displaying the calculated 3D distance 808 between the two selected points on the object surface. It should be understood that this can be repeated any suitable number of times for any suitable number of selected pixels.

[0061] So far, this disclosure has primarily focused on converting 2D image space coordinates to 3D object space coordinates. However, it will be understood that in some scenarios, it may be desirable to reverse this process. For example, a user may provide a set of 3D object space coordinates corresponding to specific points on an object surface. The computational system can then use the techniques discussed herein (e.g., calculations based on projection matrices) to give a set of 2D image space coordinates in a 2D display image from the input 3D coordinates defined in object space. In some examples, pixels in the 2D display image depicting the user-provided 3D coordinates may be highlighted or otherwise indicated. In some examples, this can facilitate collaborative viewing sessions for multiple users simultaneously. For example, a first user may select a pixel in their own augmented 2D image, for which 3D object space coordinates are calculated. Based on these calculated 3D coordinates, the system can calculate the corresponding 2D image space coordinates in a second user's augmented 2D image, which may have a different camera position than the first user's augmented 2D image. This allows either user to select a given pixel in their own augmented 2D image such that the corresponding pixel is highlighted in another user's augmented 2D image.

[0062] Furthermore, in some instances, the techniques discussed herein can be used to generate equirectangular images for panoramic displays. In one example method, the computational system can generate multiple vertical slices, each 1 pixel wide, including scene color, depth, and part identification information. These slices can be assembled into a number of equirectangular images (e.g., 3), which can be displayed in a panoramic format. At runtime, the system can identify the 2D image coordinates of selected points (e.g., determined based on an eye-tracking system), provide an offset based on the panoramic camera view, and then process the 2D image coordinates according to the process discussed above to give the corresponding 3D coordinates in the surrounding scene. This advantageously allows users to obtain 3D coordinates, measurements, and / or partial information from a 360-degree panoramic view without the complexity of locally rendering 3D models and without compromising data security by exporting 3D models to client computing devices.

[0063] Furthermore, this disclosure has so far focused on scenarios where any given pixel in a 2D display image can be converted into corresponding 3D coordinates in object space. However, it will be understood that an augmented 2D image may, in some cases, include one or more pixels that are not selectable. Such an image may, for example, include overlay information or annotations applied to the augmented 2D image that do not represent corresponding points on the surface of the digital object model.

[0064] Figure 9 An example augmented 2D image 900 is schematically represented. The augmented 2D image includes an image dataset 902A, which includes a 2D display image 904 that depicts an object from a specific camera position, as discussed above. The 2D display image includes a plurality of image pixels 906, forming a 2D pixel grid, each with different image space coordinates. The image dataset also includes a depth map 908, which includes a plurality of depth values ​​910. The plurality of depth values ​​correspond to a plurality of image pixels 906 of the 2D display image, such that the image dataset includes depth values ​​of at least some pixels of the 2D display image.

[0065] exist Figure 9 In the image dataset 902A, a component identification map 912 is also included. As discussed above, this can be created as a second 2D image of the object, where each sub-component is assigned a different unique color and then rendered using only the enabled ambient lighting effects. Pixels depicting the same sub-component in the component identification map can each use the same color value (or index value), and different color values ​​(or index values) can be associated with different sub-components. The image dataset also includes a component identification table 914 that associates different color values ​​(or index values) with different sub-component identifiers. In this way, upon receiving user input for a selected pixel in the 2D display image, the color value (or index value) of the corresponding pixel in the component identification map can be determined, and the component identification table can be used to determine the identifier of the sub-component depicted by the selected pixel.

[0066] exist Figure 9 In this example, the image dataset 902A further includes a model-view transformation matrix 916 and a projection matrix 918. The model-view transformation matrix is ​​a combination of the model transformation matrix and the camera-view transformation matrix used for locating 3D models in a 3D environment. The projection matrix is ​​used to transform the 3D coordinates of the model into 2D coordinates during the creation of the 2D display image. Based on the image space coordinates of the selected pixel, the depth value associated with the selected pixel, the model-view transformation matrix, and the projection matrix, the computing system can calculate the 3D object space coordinates of the selected point on the object corresponding to the selected pixel in the 2D display image. Additionally, in this example, the image dataset includes text metadata 920, which may include any suitable text information related to the object.

[0067] Additionally, in this example, the enhanced 2D image 900 includes a second image dataset 902B. This can include any or all of the components discussed above with respect to the first image dataset 902A. For example, the second image dataset can include another 2D display image of an object—e.g., an object depicted in a 3D environment from a different camera position or a different object depicted from the same camera position. The second image dataset may similarly include a depth map, model-view transformation matrix, and projection matrix associated with the second 2D display image. Furthermore, the enhanced 2D image can include any suitable number of one or more image datasets, each of which can include different 2D display images and other suitable data for calculating the 3D coordinates corresponding to selected pixels in the 2D display images.

[0068] Figure 10 An exemplary method 1000 for interacting with enhanced 2D images is illustrated. As with other methods discussed above, the steps of method 1000 can be started, terminated, and / or repeated at any suitable time and in response to any suitable conditions. The steps of method 1000 can be implemented by any suitable computing system of one or more computing devices. Any computing device implementing the steps of method 1000 can have any suitable capabilities, hardware configuration, and form factor. In some examples, method 1000 can be implemented by the following... Figure 11 The described computing system 1100 is implemented.

[0069] Specifically, method 1000 focuses on optional features that can be implemented in addition to the calculation of 3D object space coordinates as discussed above. To this end, at 1002, method 1000 includes calculating the 3D object space coordinates of a first selected pixel in a 2D display image corresponding to the enhanced 2D image. This can be accomplished substantially as discussed above.

[0070] Method 1000 may proceed from 1002 to any or all of step 1004 (e.g., displaying metadata), 1006 (e.g., changing color values), and 1008 (e.g., receiving second user input). At 1004, method 1000 may optionally include displaying metadata associated with the selected object. For example, as a non-limiting example, this may include textual metadata, including the part number of the aircraft part represented by the object.

[0071] At 1006, method 1000 additionally or alternatively includes changing the color values ​​of a set of pixels in a 2D display image that depict a selected sub-partition depicted by a first selected pixel. (As stated above regarding...) Figure 6CAs discussed, this can be accomplished by referring to a component identification map and a component identification table to determine which sub-component is depicted by the first selected pixels, and then using the component identification map to generate a mask that can be combined with the 2D display image to change the appearance of the selected sub-component (e.g., thereby highlighting the sub-component within the 2D display image). In this way, metadata can be displayed for the entire object when any point on the object is selected. Furthermore, pixels depicting a specific selected sub-component of the object can be highlighted to give particular emphasis to that sub-component.

[0072] At 1008, method 1000 additionally or alternatively includes receiving a second user input for a second selected pixel in a 2D display image. Based on the second selected pixel, the computing system can calculate the 3D object space coordinates of a second selected point on the object corresponding to the second selected pixel. Thus, at 1010, method 1000 optionally includes calculating the second 3D object space coordinates corresponding to the second selected pixel. This can be done substantially as discussed above with respect to the first selected pixel—for example, based on the image space coordinates, depth map, model-view transformation matrix, and projection matrix of the second selected pixel in the 2D display image. At 1012, method 1000 optionally includes calculating the 3D object space distance between the first selected point and the second selected point on the object. This can be done substantially as discussed above with respect to the first selected pixel. Figure 8 The discussion will be completed.

[0073] The methods and processes described herein can be attached to a computing system of one or more computing devices. Specifically, such methods and processes can be implemented as executable computer applications, network-accessible computing services, application programming interfaces (APIs), libraries, or combinations of the above and / or other computing resources.

[0074] Figure 11 A simplified representation of a computing system 1100 is schematically shown, which is configured to provide any to all computing functions. The computing system 1100 may take the form of one or more network-accessible devices, personal computers, server computers, mobile computing devices, and / or other computing devices.

[0075] The computing system 1100 includes a logic subsystem 1102 and a storage subsystem 1104. The computing system 1100 may optionally include a display subsystem 1106, an input subsystem 1108, a communication subsystem 1110, and / or... Figure 11 Other subsystems not shown.

[0076] The logical subsystem 1102 includes one or more physical devices configured to execute instructions. For example, the logical subsystem may be configured to execute instructions as part of one or more applications, services, or other logical constructs. The logical subsystem may include one or more hardware processors configured to execute software instructions. Alternatively, the logical subsystem may include one or more hardware or firmware devices configured to execute hardware or firmware instructions. The processor of the logical subsystem may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. The various components of the logical subsystem may optionally be distributed across two or more separate devices, which may be remotely located and / or configured for collaborative processing. Aspects of the logical subsystem may be virtualized and executed by remotely accessible networked computing devices configured for cloud computing.

[0077] Storage subsystem 1104 includes one or more physical devices configured to temporarily and / or permanently store computer information, such as data and instructions executable by a logical subsystem. When the storage subsystem includes two or more devices, the devices may be co-located and / or remotely positioned. Storage subsystem 1104 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location-addressable, file-addressable, and / or content-addressable devices. Storage subsystem 1104 may include removable and / or built-in devices. When the logical subsystem executes instructions, the state of storage subsystem 1104 may be changed—for example, to maintain different data.

[0078] Various aspects of the logic subsystem 1102 and the storage subsystem 1104 can be integrated together into one or more hardware logic components. Such hardware logic components may include, for example, programmable and application-specific integrated circuits (PASIC / ASIC), programmable and application-specific standard products (PSSP / ASSP), system-on-a-chip (SOC), and complex programmable logic devices (CPLD).

[0079] Logical subsystems and storage subsystems can collaborate to instantiate one or more logical machines. As used herein, the term "machine" is used collectively to refer to a combination of hardware, firmware, software, instructions, and / or any other components that collaborate to provide computer functionality. In other words, a "machine" is never an abstract concept and always has a tangible form. A machine can be instantiated by a single computing device, or a machine can include two or more sub-components instantiated by two or more different computing devices. In some implementations, a machine includes local components (e.g., software applications executed by a computer processor) that collaborate with remote components (e.g., cloud computing services provided by a network of server computers). The software and / or other instructions that give a particular machine its functionality can optionally be stored as one or more unexecuted modules on one or more suitable storage devices.

[0080] When a display subsystem 1106 is included, the display subsystem 1106 can be used to present a visual representation of the data held by the storage subsystem 1104. This visual representation may take the form of a graphical user interface (GUI). The display subsystem 1106 may include one or more display devices using virtually any type of technology. In some implementations, the display subsystem may include one or more virtual, augmented, or mixed reality displays.

[0081] When the input subsystem 1108 is included, it may include or interface with one or more input devices. Input devices may include sensor devices or user input devices. Examples of user input devices include a keyboard, mouse, touchscreen, or game controller. In some embodiments, the input subsystem may include or interface with a selected Natural User Input (NUI) component. Such components may be integrated or peripheral devices, and the conversion and / or processing of input actions may be performed on-board or off-board. Examples of NUI components may include a microphone for voice and / or voice recognition; an infrared, color, stereo, and / or depth camera for machine vision and / or gesture recognition; and a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition.

[0082] When the communication subsystem 1110 is included, the communication subsystem 1110 can be configured to communicatively couple the computing system 1100 to one or more other computing devices. The communication subsystem 1110 may include wired and / or wireless communication devices compatible with one or more different communication protocols. The communication subsystem can be configured to communicate via personal, local area, and / or wide area networks.

[0083] This disclosure is presented by way of example and with reference to the associated accompanying drawings. Components, processing steps, and other elements that may be substantially the same in one or more drawings are identified in a coordinated manner and described with minimal repetition. However, it will be noted that the coordinatedly identified elements may also differ to some extent. It should be further noted that some drawings may be schematic and not drawn to scale. The various drawing scales, aspect ratios, and numbers of parts shown in the drawings may be intentionally distorted to make certain features or relationships easier to see.

[0084] In addition, this disclosure includes configurations based on the following examples.

[0085] Example 1. A method for displaying a digital image, comprising: at a computing device, receiving an enhanced two-dimensional (2D) image including an image dataset derived from a digital object representation, the digital object representation including spatial data representing a three-dimensional (3D) model of an object; displaying a two-dimensional (2D) display image included in the image dataset, the 2D display image depicting the object from a virtual camera position; receiving user input for selected pixels in the 2D display image having image spatial coordinates within the 2D display image; calculating 3D object spatial coordinates of a selected point on the object corresponding to the selected pixel, at least in part based on the image spatial coordinates of the selected pixel, a depth value of the selected pixel, a model-view transformation matrix, and a projection matrix included in the image dataset; and displaying the 3D object spatial coordinates of the selected point.

[0086] Example 2. The method according to Example 1, wherein the image dataset includes a depth map having multiple depth values ​​corresponding to multiple pixels in a 2D display image.

[0087] Example 3. The method according to Example 1 further includes: receiving a second user input for a second selected pixel in a 2D display image; calculating second 3D object space coordinates of a second selected point on an object corresponding to the second selected pixel; and displaying the second 3D object space coordinates of the second selected point.

[0088] Example 4. Based on the method of Example 3, it further includes: calculating the 3D object space distance between the selected point and the second selected point, and displaying the 3D object space distance.

[0089] Example 5. The method according to Example 1, wherein the object comprises two or more sub-parts, and wherein the image dataset further comprises a part identification map, which can be used to resolve each pixel of the 2D display image to a specific sub-part of the object.

[0090] Example 6. The method according to Example 5 further includes: determining a selected sub-part among two or more sub-parts represented by selected pixels based on a part identification map, and displaying a corresponding identifier of the selected sub-part.

[0091] Example 7. The method according to Example 6 further includes: after determining the selected sub-part, changing the color value of each pixel depicting the selected sub-part in the 2D display image to highlight the selected sub-part within the 2D display image.

[0092] Example 8. The method according to Example 1, wherein the image dataset further includes a second model-view transformation matrix corresponding to a second coordinate system, and wherein the method further includes calculating a second 3D coordinate of a selected point on the object relative to the second coordinate system based on the second model-view transformation matrix.

[0093] Example 9. Based on the method of Example 1, it further includes: displaying textual metadata associated with the object.

[0094] Example 10. The method according to Example 1, wherein the computing device is a client computing device, and wherein an enhanced 2D image comprising an image dataset is received from a server computing device via a computer network.

[0095] Example 11. The method according to Example 10, wherein the 2D display image is displayed by a web browser application of a computing device.

[0096] Example 12. The method according to Example 1, wherein the object represents an aircraft part, and wherein the method further includes, upon receiving user input for a selected pixel, displaying part-specific information related to the aircraft part, the part-specific information including a part identifier.

[0097] Example 13. A method for creating an enhanced two-dimensional (2D) image, comprising: receiving a digital object representation, the digital object representation including spatial data representing a three-dimensional (3D) model of the object; rendering a 2D display image of the digital object representation from a virtual camera position; generating a depth map including multiple depth values ​​corresponding to multiple pixels in the 2D display image; and generating an enhanced 2D image, the enhanced 2D image including an image dataset of the digital object representation, the image dataset including the 2D display image, the depth map, a model-view transformation matrix, and a projection matrix associated with the virtual camera position.

[0098] Example 14. The method according to Example 13, wherein the object comprises two or more sub-parts, and wherein the method further comprises generating a part identification map capable of resolving each pixel of a 2D display image to a specific sub-part of the object.

[0099] Example 15. The method according to Example 14, wherein the component identification map is generated by generating a second 2D image in which each sub-component is assigned a different unique color, and then the component identification map is rendered with ambient lighting enabled only.

[0100] Example 16. The method according to Example 13, wherein the image dataset further includes a second model view transformation matrix corresponding to a second coordinate system.

[0101] Example 17. The method described in Example 13, wherein the image dataset further includes textual metadata corresponding to the objects.

[0102] Example 18. The method described in Example 17, wherein the text metadata includes hyperlinks.

[0103] Example 19. A computing system comprising: a logic subsystem; and a storage subsystem holding instructions executable by the logic subsystem to: receive an enhanced 2D image comprising an image dataset derived from a digital object representation, the digital object representation comprising spatial data representing a three-dimensional (3D) model of an object; display via a computer display a 2D display image comprising the image dataset, the 2D display image depicting the object from a virtual camera position; receive user input for a selected pixel in the 2D display image having image spatial coordinates within the 2D display image; calculate 3D object spatial coordinates of a selected point on the object corresponding to the selected pixel, at least in part based on the image spatial coordinates of the selected pixel, a depth value of the selected pixel in a depth map comprising the image dataset, a model-view transformation matrix, and a projection matrix comprising the image dataset; and display the 3D object spatial coordinates of the selected point via a computer display.

[0104] Example 20. The computing system according to Example 19, wherein the object comprises two or more sub-components, and wherein the image dataset further comprises a component identification map, which can be used to resolve each pixel of the 2D display image to a specific sub-component of the object.

[0105] It will be understood that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered limiting, as many variations are possible. The particular routines or methods described herein may represent one or more of any number of processing strategies. Accordingly, the different actions illustrated and / or described may be performed in the illustrated and / or described order, in another order, in parallel, or omitted. Similarly, the order of the above processing may be changed.

[0106] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of different processes, systems and configurations, as well as other features, functions, actions, and / or properties disclosed herein, and any and all their equivalents.

Claims

1. A method for digital image display, the method comprising: at a computing device, receiving an augmented two-dimensional image comprising an image data set derived from a digital object representation, the digital object representation comprising spatial data representing a three-dimensional model of an object; displaying a two-dimensional display image included in the image data set, the two-dimensional display image depicting the object from a virtual camera position; receiving user input directed to a selected pixel in the two-dimensional display image having image space coordinates within the two-dimensional display image; based at least in part on the image space coordinates of the selected pixel, a depth value of the selected pixel, a model view transformation matrix, and a projection matrix included in the image data set, computing three-dimensional object space coordinates of a selected point on the object corresponding to the selected pixel; and displaying the three-dimensional object space coordinates of the selected point.

2. The method of claim 1, wherein, the image data set comprises a depth map having a plurality of depth values corresponding to a plurality of pixels in the two-dimensional display image.

3. The method of claim 1, further comprising: receiving second user input directed to a second selected pixel in the two-dimensional display image, computing second three-dimensional object space coordinates of a second selected point on the object corresponding to the second selected pixel, and displaying the second three-dimensional object space coordinates of the second selected point.

4. The method of claim 3, further comprising: computing a three-dimensional object space distance between the selected point and the second selected point, and displaying the three-dimensional object space distance.

5. The method of claim 1, wherein, the object comprises two or more sub-components, and wherein the image data set further comprises a component identification map usable to resolve each pixel of the two-dimensional display image to a particular sub-component of the object.

6. The method of claim 5, further comprising: determining a selected sub-component of the two or more sub-components represented by the selected pixel based on the component identification map, and displaying a corresponding identifier of the selected sub-component.

7. The method of claim 6, further comprising, after determining the selected sub-component, changing color values of each pixel of the two-dimensional display image depicting the selected sub-component to highlight the selected sub-component within the two-dimensional display image.

8. The method of claim 1, wherein, the image data set further comprises a second model view transformation matrix corresponding to a second coordinate system; and wherein the method further comprises computing second three-dimensional coordinates of the selected point on the object relative to the second coordinate system based on the second model view transformation matrix.

9. The method of claim 1, further comprising: displaying textual metadata associated with the object.

10. The method of claim 1, wherein, the computing device is a client computing device, and wherein the augmented two-dimensional image comprising the image data set is received from a server computing device via the computer network.

11. The method of claim 10, wherein, the two-dimensional display image is displayed by a web browser application of the computing device.

12. The method of claim 1, wherein, the object representation is of an aircraft component, and wherein the method further comprises, after receiving user input directed to a selected pixel, displaying component-specific information related to the aircraft component, the component-specific information comprising a component identifier.

13. A method for augmented two-dimensional image creation, the method comprising: receiving a digital object representation, the digital object representation comprising spatial data representing a three-dimensional model of an object; rendering a two-dimensional display image of the digital object representation from a virtual camera position; generating a depth map comprising a plurality of depth values corresponding to a plurality of pixels in the two-dimensional display image; and generating an augmented two-dimensional image comprising an image data set of the digital object representation, the image data set comprising the two-dimensional display image, the depth map, a model view transformation matrix and a projection matrix associated with the virtual camera position.

14. The method of claim 13, wherein, the object comprises two or more sub-components, and wherein the method further comprises generating a component identification map usable for resolving each pixel of the two-dimensional display image to a particular sub-component of the object.

15. The method of claim 14, wherein, the component identification map is generated by generating a second two-dimensional image in which each sub-component is assigned a different unique color, and then rendering the component identification map with only ambient lighting enabled.

16. The method of claim 13, wherein, the image data set further comprises a second model view transformation matrix corresponding to a second coordinate system.

17. The method of claim 13, wherein, the image data set further comprises textual metadata corresponding to the object.

18. The method of claim 17, wherein, the textual metadata comprises hyperlinks.

19. A computing system comprising: a logic subsystem; and a storage subsystem holding instructions executable by the logic subsystem to: receive an augmented two-dimensional image comprising an image data set derived from a digital object representation, the digital object representation comprising spatial data representing a three-dimensional model of an object; display, via a computer display, a two-dimensional display image included in the image data set, the two-dimensional display image depicting the object from a virtual camera position; receive user input for a selected pixel in the two-dimensional display image having image space coordinates within the two-dimensional display image; compute three-dimensional object space coordinates of a selected point on the object corresponding to the selected pixel based at least in part on the image space coordinates of the selected pixel, a depth value of the selected pixel included in a depth map of the image data set, a model view transformation matrix and a projection matrix included in the image data set; and display, via the computer display, the three-dimensional object space coordinates of the selected point.

20. The computing system of claim 19, wherein, the object comprises two or more sub-components, and wherein the image data set further comprises a component identification map usable for resolving each pixel of the two-dimensional display image to a particular sub-component of the object.

Images