Semantic texture mapping system

Abstract

A semantic texture map system to generate a semantic texture map based on a 3D model comprising a plurality of vertices including coordinates indicating positions of the plurality of vertices, a UV map, and a semantic segmented image comprising a set of semantic tags.

Description

[0001] This application is a divisional application of Chinese patent application No. 202080040448.X, entitled "Semantic Texture Mapping System" (filed on March 26, 2020).

[0002] Priority requirements

[0003] This application claims the benefit of priority to U.S. Patent Application Serial No. 16 / 372,215, filed April 1, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0004] Embodiments of this disclosure generally relate to texture mapping, and more particularly to systems for generating semantic texture maps. Background Technology

[0005] A texture map is a two-dimensional (2D) image file that can be applied to the surface of a three-dimensional (3D) model to add color, texture, or other surface details. Texture maps are typically developed to directly correspond to the “UV” coordinates of a 3D model and may be based on photographs or other images. The letters “U” and “V” represent the axes of a 2D texture (since “X”, “Y”, and “Z” are already used to represent the axes of 3D objects).

[0006] For example, when creating a 3D model (such as a polygonal mesh), a UV map including a set of UV coordinates can be generated based on the positions of the mesh vertices. Therefore, the UV map includes the projection of the 2D image onto the surface of the 3D model.

[0007] While traditional computational texture analysis provides sufficient means to generate texture maps for a variety of applications, differences remain between the spatial "visual" and "semantic" features. This means they are not suitable for applications designed for high-level user interaction. Attached Figure Description

[0008] To facilitate the identification of any discussion of a particular element or action, the highest one or more significant digits in the figure references refer to the figure number in which the element was first introduced.

[0009] Figure 1 This is a block diagram illustrating an example messaging system for exchanging data (e.g., messages and associated content) over a network according to some embodiments, wherein the messaging system includes a semantic texture graph system.

[0010] Figure 2 This is a block diagram illustrating further details of a messaging system according to an example embodiment.

[0011] Figure 3 This is a block diagram illustrating various modules of a semantic texture graph system according to certain example embodiments.

[0012] Figure 4 This is a flowchart illustrating a method for generating a texture map including semantic labels, according to some example embodiments.

[0013] Figure 5 This is a flowchart illustrating a method for rendering content based on semantic features derived from a texture map including semantic tags, according to certain example embodiments.

[0014] Figure 6 This is a flowchart illustrating a method for applying semantic texture maps to the rendering of an environment at a client device, according to certain example embodiments.

[0015] Figure 7 This is a flowchart illustrating a method for generating semantic texture maps according to certain example embodiments.

[0016] Figure 8 It is an interface diagram depicting augmented reality (AR) content rendered using semantic tags based on semantic texture maps according to certain example embodiments.

[0017] Figure 9 This is a block diagram illustrating a representative software architecture that can be used in conjunction with the various hardware architectures described herein to implement various embodiments.

[0018] Figure 10 This is a block diagram illustrating components of a machine, according to some example embodiments, capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any or more of the methods discussed herein. Detailed Implementation

[0019] As discussed above, texture mapping is a method for defining the details, surface texture, or color information of a 3D model. While existing texture mapping systems are sufficient to add visual detail to 3D models, they do not provide a method for assigning properties and labels to the various surfaces of a 3D model, making them less than ideal for applications that expect a high level of user interaction. Therefore, the example embodiments described herein relate to systems and methods for generating “semantic” texture maps of 3D objects, wherein the semantic texture map defines a set of semantic segmentation labels for a particular 3D object.

[0020] The example embodiments described herein relate to a semantic texture map system to perform operations including: accessing an image comprising a set of image features; obtaining a semantic segmentation image based on the set of image features of the image, the semantic segmentation image comprising a set of semantic labels; generating a 3D model (such as a 3D mesh model) based on the set of image features of the image, the 3D model comprising multiple vertices including coordinates indicating the positions of the multiple vertices; generating a UV map based on the set of image features and the coordinates of the multiple vertices of the 3D model, the UV map comprising a set of two-dimensional (2D) texture coordinates based on the coordinates of the multiple vertices of the 3D mesh model; projecting the semantic segmentation image onto the 3D mesh model based on the 2D texture coordinates of the UV map; mapping a set of semantic labels of the semantic segmentation image to the 2D texture coordinates of the UV map; and generating a semantic texture map comprising a set of texels, wherein each texel comprises one or more semantic labels and a set of 2D texture coordinates.

[0021] "Semantic segmentation" describes a computer vision technique used to predict class labels for each pixel of an image. Therefore, a semantically segmented image comprises an image containing a set of class labels that identify elements or objects depicted by the pixels of the image. According to some embodiments, semantically segmented images can be generated by a semantic texture map system using a semantic segmentation neural network.

[0022] For example, a semantic texture graph system can receive image data, including metadata identifying locations or objects, as input from multiple distributed client devices. The image data can be classified and sorted by the semantic texture graph system based on the corresponding image metadata. The semantic texture graph system can maintain a database comprising a collection of image data sorted according to the locations or objects depicted by the image data. A semantic segmentation neural network can then be trained based on the image collection from the database to generate semantically segmented images that include semantic class labels that identify semantic features of the pixels in the images.

[0023] Semantic features may include, for example: contextual features corresponding to a physical object, location, or surface; analogical features referencing other known categories or classes; visual features defining the visual or graphic properties of an object or surface; and material parameters defining the properties of an object or surface and may include “roughness value,” “metallic value,” “mirror value,” and “primary color value.”

[0024] Therefore, in response to receiving or accessing an image including image metadata and a set of image features, the semantic texture map system generates or obtains a semantic segmentation image based on the image (image features or image metadata of the image), wherein the semantic segmentation image includes a set of semantic feature labels assigned to the pixels of the image.

[0025] Semantic texture mapping (SMT) systems generate 3D models based on images accessed by the system. For example, a SMT system can identify a location or object depicted in an image based on image metadata or image features, and retrieve a set of images depicting that location or object from a database. The SMT system processes the set of images through a 3D reconstruction pipeline to obtain a dense 3D mesh (i.e., a 3D model) based on the set of images corresponding to the location or object depicted in the accessed images. This dense 3D mesh comprises multiple vertices, each of which includes corresponding coordinates. Similarly, a SMT system generates a UV map based on an image and a set of images. The UV map includes a set of 2D texture coordinates to project the 2D image onto the 3D mesh. For example, the 3D mesh can be "wrapped" with a 2D image based on the 2D texture coordinates of the UV map.

[0026] The semantic texture map system projects a semantically segmented image corresponding to an image onto a 3D model based on vertex coordinates, and maps the semantic feature labels of the semantically segmented image to 2D texture coordinates of the UV map. Therefore, the resulting semantic texture map includes a set of texels (where each texel includes one or more semantic feature labels) and 2D texture coordinates based on the UV map.

[0027] According to some embodiments, a semantic texture map generated by a semantic texture map system can be applied to one or more objects detected within the rendering of an environment displayed at a client device. For example, the semantic texture map can be associated with a set of coordinates corresponding to an object (i.e., a building, structure, sculpture). In response to detecting a client device at or within a threshold distance of an object (i.e., based on geofencing), the semantic texture map system accesses the semantic texture map and applies it to the depiction of the object within the rendering of the environment including the object's display at the client device. For example, based on the coordinates of vertices of a 3D model, the texture map can be applied as an overlay to the object such that the texels of the texture map occupy corresponding positions on the object's surface.

[0028] By applying texture maps that include semantic feature labels to objects in the rendering of an environment at a client device, AR content can be accessed and rendered within the environment based on these semantic feature labels. As discussed above, the semantic feature labels of the texels in the texture map can provide categorization information about the features of objects rendered in the environment. For example, semantic feature labels may include identification of semantic information about the texels, such as which texels are “doors,” “windows,” “walls,” “roofs,” or “chimneys,” and information about the material properties of the features represented by the texels, including but not limited to “hardness,” “softness,” “flexibility,” and “gloss.”

[0029] This allows AR content to interact more realistically with the user through its spatial representation at the client device. As an illustrative example, by labeling which features of the space are “chimneys,” AR content that realistically interacts with those features can be accessed and presented. For instance, in response to detecting semantic feature labels such as “chimney” and “roof,” an AR “Santa Claus” can be accessed and presented in the spatial representation. In some embodiments, other attributes can be considered for selecting AR content, such as user profile information, time information, seasonal information, and location information.

[0030] Figure 1 This is a block diagram illustrating an example messaging system 100 for exchanging data (e.g., messages and associated content) over a network. Messaging system 100 includes multiple client devices 102, each hosting multiple applications including messaging client applications 104. Each messaging client application 104 is communicatively coupled to other instances of messaging client applications 104 and a messaging server system 108 via a network 106 (e.g., the Internet).

[0031] Therefore, each messaging client application 104 is able to communicate and exchange data with another messaging client application 104 and messaging server system 108 via network 106. The data exchanged between messaging client devices 104 and between messaging client application 104 and messaging server system 108 includes functions (e.g., commands to call functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0032] Messaging server system 108 provides server-side functionality to a specific messaging client application 104 via network 106. Although some functions of messaging system 100 are described herein as being performed by messaging client application 104 or by messaging server system 108, it should be understood that the location of certain functions within messaging client application 104 or messaging server system 108 is a design choice. For example, it is technically preferable to first deploy certain technologies and functions within messaging server system 108 and then migrate those technologies and functions to messaging client application 104, in which client device 102 has sufficient processing power.

[0033] The messaging server system 108 supports various services and operations provided to the messaging client application 104. These operations include sending data to and receiving data from the messaging client application 104, and processing data generated by the messaging client application 104. In some embodiments, this data includes message content, client device information, geolocation information, media annotations and overlays, message content persistence conditions, social network information, and on-site event information, as examples. In other embodiments, other data is used. Data exchange in the messaging system 100 is invoked and controlled via functions available through the GUI of the messaging client application 104.

[0034] Now, specifically to the messaging server system 108, the application programming interface (API) server 110 is coupled to the application server 112 and provides a programming interface to the application server 112. The application server 112 is communicatively coupled to the database server 118, which facilitates access to the database 120, which stores data associated with the messages processed by the application server 112.

[0035] A special application programming interface (API) server 110 is provided to receive and send message data (e.g., commands and message payloads) between client device 102 and application server 112. Specifically, API server 110 provides a set of interfaces (e.g., routines and protocols) that can be invoked or queried by messaging client application 104 to call functions of application server 112. Application Programming Interface (API) server 110 exposes various functions supported by application server 112, including: account registration, login functionality, sending messages from one messaging client application 104 to another messaging client application 104 via application server 112, sending media files (e.g., images or videos) from messaging client application 104 to messaging server application 114 for possible access by another messaging client application 104, setting up media data (e.g., story) collections, obtaining the friend list of the user of client device 102, obtaining such collections, obtaining messages and content, adding and deleting friends in the social graph, locating friends within the social graph, and opening and applying events (e.g., related to messaging client application 104).

[0036] Application server 112 hosts multiple applications and subsystems, including messaging server application 114, image processing system 116, social networking system 122, and semantic texture graph system 124. Messaging server application 114 implements multiple messaging techniques and functions, particularly involving the aggregation and other processing of content (e.g., text and multimedia content) received from multiple instances of messaging client application 104. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., referred to as stories, galleries, or collections). Messaging server application 114 then makes these collections available to messaging client application 104. Considering the hardware requirements of such processing, additional processor- and memory-intensive processing of the data can also be performed on the server side by messaging server application 114.

[0037] Application server 114 also includes an image processing system 118, which is dedicated to performing various image processing operations, typically relating to images or videos received within the payload of messages at message server application 116.

[0038] Social networking system 122 supports various social networking features and services, and makes these features and services available to messaging server application 114. To this end, social networking system 122 maintains and accesses entity graph 304 within database 120. Examples of features and services supported by social networking system 122 include identifying other users of messaging system 100 who have relationships with a particular user or who are “following” that particular user, as well as other entities and interests of the particular user.

[0039] Application server 112 is communicatively coupled to database server 118, which facilitates access to database 120, where data associated with messages processed by message server application 114 is stored.

[0040] Figure 2 This is a block diagram illustrating further details of a messaging system 100 according to an example embodiment. Specifically, the messaging system 100 is shown as including a messaging client application 104 and an application server 112, which in turn embody several subsystems, namely a short-timer system 202, a collection management system 204, and an annotation system 206.

[0041] The short-time timer system 202 is responsible for performing temporary access to content permitted by the messaging client application 104 and the messaging server application 114. To this end, the short-time timer system 202 incorporates multiple timers that selectively display and enable access to messages and associated content via the messaging client application 104 based on duration and display parameters associated with messages, message sets, or graphical elements. Further details regarding the operation of the short-time timer system 202 are provided below.

[0042] The collection management system 204 is responsible for managing collections of media (e.g., media collections that include collections of text, images, video, and audio data). In some examples, collections of content (e.g., messages that include images, videos, text, and audio) can be organized into “event galleries” or “event stories.” Such collections can be available for a specified time period (e.g., the duration of an event related to the content). For example, content related to a concert can be available as a “story” for the duration of that concert. The collection management system 204 can also be responsible for publishing icons that notify the user interface of the messaging client application 104 of the existence of a specific collection.

[0043] The collection management system 204 also includes a curation interface 208, which allows collection managers to manage and curate specific collections of content. For example, the curation interface 208 enables event organizers to curate collections of content related to a specific event (e.g., removing inappropriate content or redundant messages). Additionally, the collection management system 204 employs machine vision (or image recognition technology) and content rules to automatically curate content collections. In some embodiments, users may be compensated for including user-generated content in the collection. In this case, the curation interface 208 operates to automatically pay such users for using its content.

[0044] Annotation system 206 provides various functions that enable users to annotate or otherwise modify or edit media content, such as user support content received by the user for forwarding or reassignment to one or more recipients. For example, annotation system 206 provides functions related to media overlays for message generation and publishing processed by messaging system 100. Annotation system 206 operatively provides media overlays to messaging client application 104 based on the geographic location of client device 102. In another example, annotation system 206 operatively provides media overlays to messaging client application 104 based on other information, such as the social network information of the user of client device 102. Media overlays may include audio and visual content and visual effects, as well as augmented reality overlays. Examples of audio and visual content include images, text, logos, animations and sound effects, as well as animated facial models, image filters, and augmented reality media content. Examples of visual effects include color overlays. Audio and visual content or visual effects may be applied at client device 102 to media content items (such as photos, videos, or live streams). For example, media overlays may include text that can be overlaid on a photograph generated by client device 102. In another example, media overlays may include identifiers for location overlays (e.g., Venice Beach), names of on-site events, or names of merchant overlays (e.g., beach cafes). In yet another example, annotation system 206 uses the geographic location of client device 102 to identify media overlays that include the name of a merchant at that geographic location. Media overlays may include additional tags associated with the merchant. Media overlays may be stored in database 120 and accessible via database server 118.

[0045] In one example embodiment, annotation system 206 provides a user-based publishing platform that allows users to select geographic locations on a map and upload content associated with those locations. Users can also specify under what circumstances specific media overlays should be made available to other users. Annotation system 206 generates media overlays that include the uploaded content and associates the uploaded content with the selected geographic location.

[0046] In another example embodiment, annotation system 206 provides a merchant-based publishing platform that enables merchants to select specific media overlays associated with geographic locations. For example, annotation system 206 associates the media overlay of the highest bidder with the corresponding geographic location within a predetermined time period.

[0047] Figure 3This is a block diagram illustrating components of a semantic texture map system 124 according to certain example embodiments, which configures the semantic texture map system 124 to generate a semantic texture map by performing operations including: accessing an image comprising a set of image features; obtaining a semantic segmentation image in response to accessing the image comprising a set of image features, wherein the semantic segmentation image comprises a set of semantic labels; generating a 3D model in response to accessing the image, at least based on a set of image features of the image, wherein the 3D model comprises a plurality of vertices containing coordinates; generating a UV map based on a set of image features and the coordinates of the plurality of vertices of the 3D mesh model, the UV map comprising a set of 2D texture coordinates based on the coordinates of the plurality of vertices; projecting the semantic segmentation image onto the 3D model based on the 2D texture coordinates of the UV map; mapping a set of semantic labels of the semantic segmentation image to the 2D texture coordinates; and generating a texture map comprising a set of texels based on the semantic labels, the UV map, and the 3D mesh model.

[0048] The semantic texture map system 124 is shown as including an image module 302, a 3D mesh module 304, a UV map module 306, and a semantic tagging module 308, all of which are configured to communicate with each other (e.g., via a bus, shared memory, or a switch). Any or more of these modules may be implemented using one or more processors 310 (e.g., by configuring the one or more processors to perform the functions described for the module) and thus may include one or more of the processors 310.

[0049] Any one or more of the described modules can be implemented individually using hardware (e.g., one or more of the processors 310 of a machine) or a combination of hardware and software. For example, any module of the described semantic texture graph system 124 may physically include an arrangement of one or more processors 310 (e.g., a subset of one or more processors of a machine or a subset thereof) configured to perform the operations described herein for that module. As another example, any module of the semantic texture graph system 124 may include software, hardware, or both that configure an arrangement of one or more processors 310 (e.g., in one or more processors of a machine) to perform the operations described herein for that module. Thus, different modules of the semantic texture graph system 124 may include different arrangements of the processors 310 or a single arrangement of the processors 310 at different points in time. Furthermore, any two or more modules of the semantic texture graph system 124 may be combined into a single module, and the functionality described herein for a single module may be subdivided into multiple modules. Moreover, according to various example embodiments, modules described herein as being implemented within a single machine, database, or device may be distributed across multiple machines, databases, or devices.

[0050] Figure 4This is a flowchart illustrating a method 400 for generating a texture map including semantic tags according to certain example embodiments. The operation of method 400 can be described above relative to... Figure 3 The module described is used for execution. For example... Figure 4 As shown, method 400 includes one or more operations 402, 404, 406, 408, 410, 412 and 414.

[0051] At operation 402, image module 302 accesses an image comprising a set of image features. For example, image module 302 may receive an image from client device 102, wherein the image includes an image presented on the display of client device 102 and depicts a real-world environment. For example, client device 102 may include one or more cameras configured to generate and stream image data to be presented on the display of client device 102 or on an auxiliary device (e.g., streaming video data from client device 102 to a separate display).

[0052] In some embodiments, the image accessed by the image module 302 may include image metadata or other identifying features. For example, image metadata may identify a location based on geographic coordinates, or identifying features may include barcodes or other scannable coded images.

[0053] At operation 404, the semantic tagging module 308 obtains a semantically segmented image including a set of semantic labels based on the image features of the image. For example, the semantic tagging module 308 may access a repository such as database 120 to obtain a semantically segmented image corresponding to the image based on the image's metadata or image features.

[0054] In some embodiments, the semantic tagging module 308 generates semantic segmentation images for locations or objects based on a semantic segmentation neural network. For example, the semantic tagging module 308 may access an image repository that includes a set of images associated with locations or objects, and use the set of images to train the semantic segmentation neural network to generate semantic segmentation images that include a set of semantic labels.

[0055] At operation 406, 3D mesh module 304 generates a 3D mesh model based on a set of image features of the image, wherein the 3D mesh model includes multiple vertices containing coordinates. For example, 3D mesh module 304 can identify objects depicted in an image based on a set of image features of image metadata and obtain a set of images associated with the objects or locations, wherein the set of images includes a set of images depicting the objects or locations depicted by the images accessed by image module 302. 3D mesh module 304 processes the image through a 3D reconstruction pipeline to obtain a dense 3D mesh representation of the objects or locations depicted in the image or set of images, and their 3D pose relative to the objects.

[0056] At operation 408, UV map module 306 generates a UV map based on a set of image features of the image and the coordinates of multiple vertices of the 3D mesh model. Therefore, the UV map may include a set of 2D texture coordinates corresponding to the vertex coordinates of the 3D mesh model. By applying a 2D image to the UV map, the 3D mesh model can be "wrapped" by the 2D image.

[0057] 2D images can be generated based on images accessed by image module 302 or based on a collection of images from database 120. For example, a 2D image may include a photograph of an object, or an image generated based on a collection of images depicting the object from database 120.

[0058] At operation 410, semantic tagging module 308 projects the semantic segmentation image onto a 3D mesh model based on 2D texture coordinates from the UV map, and at operation 412, maps a set of semantic labels of the semantic segmentation image to 2D texture coordinates.

[0059] In operation 414, the semantic tagging module 308 generates a semantic texture map based on the mapping of semantic tags to 2D texture coordinates of the UV map. The texture map includes a set of texels, where each texel includes 2D texture coordinates and one or more semantic tags. Therefore, the semantic texture map can include a mapping of semantic tags to the surface of a 3D mesh based on 2D texture coordinates.

[0060] In some embodiments, the semantic tagging module 308 can associate a semantic texture map with an object category or location within a database, such as database 120. For example, the semantic texture map can be associated with a set of geographic coordinates that identify a location of interest or an object at that location. As an illustrative example, the semantic texture map can be generated based on a building or structure at a specific location identified by a set of geographic coordinates, wherein the semantic tags of the semantic texture map can assign semantic feature descriptors to one or more elements of the building or structure. For example, texels in the semantic feature map can indicate that a particular surface of the building or structure is a doorway, or an exit of the building or structure.

[0061] Figure 5 This is a flowchart illustrating a method 500 for rendering content based on semantic features derived from a texture map including semantic tags, according to certain example embodiments. The operation of method 500 can be described above relative to... Figure 3 The module described is used for execution. For example... Figure 5 As shown, method 500 includes one or more operations 502, 504, and 506, which can serve as... Figure 4 A portion of the method 400 described herein (e.g., a subroutine) is executed. For example, method 500 may be executed after the operation of method 400.

[0062] For example, after generating a semantic texture map, the semantic texture map system 124 can associate the semantic texture map with a location or object of interest. In response to detecting a location of interest at the client device 102 or receiving image data that identifies the location or object of interest, the semantic texture map system 124 acquires the semantic texture map and applies it to the image data received from the client device 102. The semantic texture map can be overlaid on an image presented at the display of the client device 102, wherein the position of the semantic texture map in the image is based on features of the image itself. For example, the semantic texture map can be applied to a building depicted in an image presented at the client device 102, wherein the semantic labels of the semantic texture map identify one or more features of the building and assign characteristics to one or more features of the building (e.g., doorway, exit, window, roof, chimney, resilient floor).

[0063] At operation 502, the semantic tagging module 308 assigns material parameters to feature classes corresponding to semantic tags in a set of semantic labels in the semantic texture map. For example, an administrator of the semantic texture map system 124 can assign material parameters to classes identified by one or more semantic feature tags within the database 120. Material parameters may include graphic properties such as reflectivity or gloss, and physical properties such as softness or flexibility. Material parameters can be defined based on a set of interaction variables, where texels assigned parameters can be interacted with by the user or by AR content based on the associated material parameters.

[0064] For example, material parameters may include reflectivity values ​​such that high reflectivity values ​​correspond to mirrors, highly reflective surfaces (i.e., mirrors), while low reflectivity values ​​correspond to non-reflective surfaces (i.e., brick walls).

[0065] At operation 504, after assigning material parameters to a class, image module 302 detects texels in a semantic texture map that includes semantic feature labels corresponding to the class. For example, a user of client device 102 can display a rendering that includes the location of an object. In response to the detection of client device 102 at that location or an object within the rendering at that location, semantic texture map system 124 accesses the semantic texture map corresponding to that location or object and applies the semantic texture map to one or more relevant surfaces. For example, the semantic texture map can be created by locating a 3D mesh model associated with an object at a location based on the object's position within the rendering of the environment, and then wrapping the 3D mesh model with texels in a UV map that includes a set of semantic feature labels, including feature labels associated with the class.

[0066] At operation 506, AR content can be rendered in space based on the material parameters of the texels in the semantic texture map. As an illustrative example, the material parameters may include a high reflectivity value, and the AR content may include the display of an animated graphical avatar at a location within the spatial rendering. After detecting the material parameters of the texels, the AR content can be rendered accordingly so that the reflection of the animated graphical avatar can be displayed in the texels.

[0067] Figure 6 This is a flowchart illustrating a method 600 for presenting a semantic texture map to an environment at a client device 102, according to certain example embodiments. The operation of method 600 can be described above relative to... Figure 3 The module described is used for execution. For example... Figure 6 As shown, method 600 includes one or more operations 602, 604, and 606, which can serve as... Figure 4 This is performed as a part (e.g., a subroutine) of the method 400 described herein. For example, method 600 may be performed after a semantic texture map has been generated based on the image data.

[0068] At operation 602, the semantic texture map system 124 receives input that assigns a semantic texture map to a location or target object within the database 120. For example, the semantic texture map may be associated with a set of geographic coordinates identifying the location, or in a further embodiment, it may be assigned to a set of image features corresponding to the target object.

[0069] At operation 604, the semantic texture map system 124 detects the client device 102 at a location identified by geographic coordinates associated with a semantic texture map in the database 120. For example, a geofence can be configured based on geographic coordinates associated with a semantic texture map, wherein the geofence contains locations identified by the geographic coordinates.

[0070] At operation 606, in response to the detection of client device 102 at a location identified by geographic coordinates, semantic texture mapping system 124 applies a semantic texture map associated with that location to the rendering of an image at client device 102.

[0071] Figure 7 This is a flowchart 700 illustrating a method for generating semantic texture maps according to some example embodiments.

[0072] Semantic texture map system 124 generates a semantic texture map based on one or more images including image features. At operation 705 of flowchart 700, semantic texture map system 124 accesses a repository, such as database 120, wherein the repository contains a collection of images depicting the target object or location of interest.

[0073] As can be seen in flowchart 700, the set of images may include, for example, images depicting a target object, such as house 725. In response to accessing the set of images, at operation 710, various modules of the semantic texture map system 124 generate a 3D mesh model 730 of the target object depicted in the images by applying the set of images to the 3D reconstruction pipeline. The 3D mesh model 730 includes a set of vertices, which includes coordinates indicating the position of the vertices relative to each other.

[0074] At operation 715, as discussed in operation 408 of method 400, the semantic texture graph system 124 generates a UV graph 735 based on the coordinates of multiple vertices. The UV graph 735 includes a set of 2D texture coordinates based on the coordinates of multiple vertices of a 3D mesh model. As an illustrative example, the UV graph 735 can be described as an “unfolded” layer of the 3D mesh model 730, which defines a set of 2D coordinates that can be used to apply graphics to the 3D mesh model 730.

[0075] At operation 720, the semantic texture map system 124 generates a semantic segmentation image 740 based on the set of images, including image 725, by applying the set of images to a semantic segmentation neural network. The semantic segmentation image 740 generated by the semantic texture map system 124 provides semantic feature labels to one or more objects depicted in the set of images. According to some embodiments, the semantic segmentation image 740 may include semantic feature labels describing one or more properties of texels, including texture properties, physical properties, and graphical properties.

[0076] As discussed in operation 410 of method 400, the semantic segmentation image 740 is projected onto the 3D mesh model 725 based on coordinates from the UV map 735, and the semantic feature labels of the semantic segmentation image 740 are mapped to the coordinates of the UV map 735 to generate a semantic texture map.

[0077] Figure 8 It is an interface diagram 800 depicting AR content 805, 810, and 815 presented based on semantic tags according to certain example embodiments, such as from Figure 7 Semantic labels for image 740 of semantic segmentation.

[0078] As can be seen in interface diagram 800, AR content 805, 810, and 815 can be presented based on semantic feature labels of semantic texture maps generated by semantic texture map system 124. For example, Figure 8As shown, AR content 805, 810, and 815 may include graphical elements rendered at specific locations within texels of an image displayed based on semantic feature labels of a semantic texture map. For example, texel 820 (emphasized by thick lines) may include a set of semantic feature labels 825, wherein the semantic feature labels identify characteristics of the texel. As can be seen in interface diagram 800, the semantic feature label 825 of texel 820 may indicate that the texel includes a window. Therefore, the semantic feature map system 124 may generate and render AR content 815 based on one or more of the semantic feature labels 825, providing an added layer of realism to the AR content.

[0079] Software Architecture

[0080] Figure 9 This is a block diagram illustrating an example software architecture 906 that can be used in conjunction with the various hardware architectures described herein. Figure 9 This is a non-limiting example of software architecture, and it will be understood that many other architectures can be implemented to facilitate the functionality described herein. Software Architecture 906 can be applied in various ways, such as... Figure 10 The execution is performed on the hardware of machine 1000, which includes processor 1004, memory 1014, and I / O components 1018, etc. A representative hardware layer 1504 is shown and can represent, for example... Figure 9 The machine 900. A representative hardware layer 952 includes a processing unit 954 having associated executable instructions 904. The executable instructions 904 represent executable instructions of the software architecture 906, including implementations of the methods, components, etc., described herein. Hardware layer 952 also includes a memory and / or storage module memory / storage device 956, which also has executable instructions 904. Hardware layer 952 may also include other hardware 958.

[0081] exist Figure 9 In the example architecture, software architecture 906 can be conceptualized as a stack of layers, where each layer provides specific functionality. For example, software architecture 906 may include layers such as an operating system 902, libraries 920, applications 916, and a presentation layer 914. Operationally, applications 916 and / or other components within a layer can call application programming interface (API) calls 908 via the software stack and receive responses in response to API calls 908. The layers shown are representative in nature, and not all software architectures have all layers. For example, some mobile or dedicated operating systems may not provide a framework / middleware 918, while others may provide such a layer. Other software architectures may include additional or different layers.

[0082] Operating system 902 manages hardware resources and provides public services. Operating system 902 may include, for example, a kernel 922, services 924, and drivers 926. Kernel 922 can act as an abstraction layer between hardware and other software layers. For example, kernel 922 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Services 924 can provide other public services to other software layers. Drivers 926 can be responsible for controlling or interfacing with the underlying hardware. For example, depending on the hardware configuration, drivers 926 may include display drivers, camera drivers, Bluetooth® drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, audio drivers, power management drivers, etc.

[0083] Library 920 provides a common infrastructure used by application 916 and / or other components and / or layers. Library 920 provides functionality that allows other software components to perform tasks in a way that is easier than directly interfaced with the functions of the underlying operating system 902 (e.g., kernel 922, service 924, and / or driver 926). Library 920 may include system libraries 944 (e.g., the C standard library), which provides functions such as memory allocation, string manipulation, and mathematical functions. Furthermore, Library 920 may include API libraries 946 such as media libraries (e.g., libraries for rendering and manipulating various media formats such as MPREG4, H.264, MP3, AAC, AMR, JPG, and PNG), graphics libraries (e.g., the OpenGL framework for rendering 2D and 3D graphics content on a display), database libraries (e.g., SQLite providing various relational database functions), network libraries (e.g., WebKit providing web browsing capabilities), and so on. Library 920 may also include a wide variety of other libraries 948 to provide multiple additional APIs to application 916 and other software components / modules.

[0084] The framework / middleware 918 (sometimes also called middleware) provides advanced, general-purpose infrastructure that can be used by applications 916 and / or other software components / modules. For example, the framework / middleware 918 provides various graphical user interface (GUI) functionalities, advanced resource management, advanced location services, etc. The framework / middleware 918 also provides a wide range of other APIs that can be used by applications 916 and / or other software components / modules, some of which may be specific to a particular operating system 902 or platform.

[0085] Application 916 includes built-in applications 938 and / or third-party applications 940. Examples of representative built-in applications 938 may include, but are not limited to: contact applications, browser applications, book reader applications, location applications, media applications, messaging applications, and / or game applications. Third-party applications 940 may include those used by entities other than the seller of a specific platform using Android. TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on mobile operating systems such as iOS. TM ANDROID TM Mobile software running on a mobile operating system (such as Windows® Phone or other mobile operating systems). Third-party applications 940 may invoke API calls 908 provided by the mobile operating system (such as operating system 902) to facilitate the functions described herein.

[0086] Application 916 can use built-in operating system functions (e.g., kernel 922, services 924, and / or drivers 926), libraries 920, and frameworks / middleware 918 to create user interfaces to interact with the system's users. Alternatively or additionally, in some systems, user interaction may occur through a presentation layer such as presentation layer 914. In these systems, the application / component "logic" can be separated from the aspects of the application / component that interact with the user.

[0087] Figure 10 This is a block diagram illustrating components of a machine 1000, according to some example embodiments, capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and executing any one or more of the methods discussed herein. Specifically, Figure 10A schematic representation of a machine 1000 in an example form of a computer system is shown, in which instructions 1010 (e.g., software, programs, applications, applets, or other executable code) can be executed to cause the machine 1000 to perform any or more methods discussed herein. Thus, instructions 1010 can be used to implement the modules or components described herein. Instructions 1010 transform a generic, unprogrammed machine 1000 into a specific machine 1000 programmed to perform the described and illustrated functions in the described manner. In alternative embodiments, machine 1000 operates as a standalone device or can be coupled (e.g., networked) to other machines. In a network deployment, machine 1000 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1000 may include, but is not limited to: server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, networked appliances, network routers, network switches, bridges, or any machine capable of executing instructions 1010 sequentially or otherwise, specifying the actions that machine 1000 will take. Furthermore, although only a single machine 1000 is shown, the term "machine" should also be considered to include a collection of machines that individually or collectively execute instructions 1010 to perform any or more of the methods discussed herein.

[0088] Machine 1000 may include processor 1004, memory / storage device 1006, and I / O components 1018, which may be configured to communicate with each other, such as via bus 1002. Memory / storage device 1006 may include memory 1014, such as main memory or other memory storage devices, and storage cells 1016, which may be accessed by processor 1004, such as via bus 1002. Storage cells 1016 and memory 1014 store instructions 1010 embodying any one or more of the methods or functions described herein. Instructions 1010 may also reside wholly or partially in memory 1014, in storage cell 1016, in at least one of processor 1004 (e.g., in the processor's cache memory), or any suitable combination thereof during execution by machine 1000. Thus, memory 1014, storage cell 1016, and the memory of processor 1004 are examples of machine-readable media.

[0089] I / O component 1018 may include a wide variety of components for receiving input, providing output, generating output, sending information, exchanging information, acquiring measurements, etc. The specific I / O component 1018 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It should be understood that I / O component 1018 may be included in... Figure 10 Several other components are not shown. For the purpose of simplifying the discussion below, the I / O components 1018 are grouped according to function, and this grouping is by no means limiting. In various example embodiments, the I / O components 1018 may include output components 1026 and input components 1028. Output components 1026 may include visual components (e.g., displays, such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), auditory components (e.g., speakers), haptic components (e.g., vibration motors, resistive mechanisms), other signal generators, etc. Input components 1028 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), haptic input components (e.g., physical buttons, touchscreens providing the location and / or force or touch gesture of a touch, or other haptic input components), audio input components (e.g., microphones), etc.

[0090] In a further example embodiment, I / O component 1018 may include a variety of other components such as biometric component 1030, motion component 1034, environmental component 1036, or position component 1038. For example, biometric component 1030 may include components for detecting expressions (e.g., hand gestures, facial expressions, voice expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), and identifying a person (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 1034 may include accelerometer components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), etc. Environmental component 1036 may include, for example, a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting the concentration of harmful gases or measuring pollutants in the atmosphere for safety purposes), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. Position component 1038 may include a position sensor component (e.g., a Global Positioning System (GPS) receiver component), an altitude sensor component (e.g., an altimeter or barometer for detecting from which altitude the air pressure can be obtained), an orientation sensor component (e.g., a magnetometer), etc.

[0091] Various technologies can be used to implement communication. I / O component 1018 may include communication component 1040, which is operable to couple machine 1000 to network 1032 or device 1020 via coupling 1022 and coupling 1024, respectively. For example, communication component 1040 may include a network interface component or other suitable device interfaced with network 1032. In a further example, communication component 1040 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components that provide communication via other forms. Device 1020 may be another machine or any of a wide variety of peripheral devices, such as peripheral devices coupled via Universal Serial Bus (USB).

[0092] Furthermore, the communication component 1040 may detect identifiers or include components operable to detect identifiers. For example, the communication component 1040 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes (e.g., Quick Response (QR) codes, Aztec codes, data matrices, digital graphics, maximum codes, PDF417, super codes, UCC RSS-2D barcodes), and other optical codes), or an acoustic detection component (e.g., a microphone for identifying the tagged audio signal). Additionally, various information can be obtained via the communication component 1040, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting NFC beacon signals indicating a specific location, etc.

[0093] Vocabulary

[0094] In this context, "carrier signal" refers to any intangible medium capable of storing, encoding, or carrying instructions for execution by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of such instructions. Instructions can be sent or received over a network using transmission media via network interface devices and using any of a number of well-known transmission protocols.

[0095] In this context, "client device" means any machine connected to a communication network interface to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to: mobile phones, desktop computers, laptops, portable digital assistants (PDAs), smartphones, tablets, ultrabooks, netbooks, laptops, multiprocessor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0096] In this context, "communication network" refers to one or more parts of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless local area network (WLAN), wide area network (WAN), wireless wide area network (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a POTS network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, a network or a part of a network may include a wireless or cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or another type of cellular or wireless coupling. In this example, coupling can enable any of a variety of data transmission technologies, such as single-carrier radio transmission technology (1xRTT), evolved data optimization (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate GSM evolution (EDGE) technology, the 3rd Generation Partnership Project (3GPP) including 3G, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standard, other standards defined by various standards-setting organizations, other remote protocols, or other data transmission technologies.

[0097] In this context, a "short-lived message" refers to a message that is accessible for a limited time. Short-lived messages can be text, images, videos, etc. The access time for a short-lived message can be set by the message sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is short-lived.

[0098] In this context, "machine-readable medium" means a component, device, or other tangible medium capable of temporarily or permanently storing instructions and data, and may include, but is not limited to: random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage devices (e.g., erasable programmable read-only memory (EPROM)), and / or any suitable combination thereof. The term "machine-readable medium" should be understood to include a single medium or multiple media capable of storing instructions (e.g., a centralized or distributed database, or associated caches and servers). The term "machine-readable medium" should also be understood to include any medium or combination of media capable of storing machine-executable instructions (e.g., code) that, when executed by one or more processors of the machine, cause the machine to perform any one or more methods described herein. Therefore, "machine-readable medium" refers to a single storage device or apparatus, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or apparatuses. The term "machine-readable medium" does not include the signal itself.

[0099] In this context, a “component” refers to a device, physical entity, or logic having boundaries defined by functional or subroutine calls, branch points, application programming interfaces (APIs), or other partitions or modular technologies for specific processing or control functions. Components can be combined through their interfaces with other components to perform machine processes. A component can be an encapsulated functional hardware unit designed to be used as part of a program that performs a specific function, typically related to another component. Components can constitute software components (e.g., code embodied on a machine-readable medium) or hardware components. A “hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various example embodiments, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or a group of processors) of a computer system may be configured by software (e.g., an application or application portion) to operate to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component may include dedicated circuitry or logic permanently configured to perform certain operations. A hardware component may be a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). Hardware components may also include programmable logic or circuitry temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. After being configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) specifically tailored to perform the configured function and is no longer a general-purpose processor. It is understood that the decision to implement a hardware component mechanically in dedicated and permanently configured circuitry or in temporarily configured circuitry (e.g., software-configured) may be driven by cost and time considerations. Therefore, the phrase “hardware component” (or “hardware-implemented component”) should be understood to include tangible entities that are physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmable) entities that operate or perform certain operations described herein in a certain way. Considering embodiments in which hardware components are temporarily configured (e.g., programmed), each hardware component does not need to be configured or instantiated at any given time. For example, in cases where hardware components include a general-purpose processor configured by software to become a dedicated processor, the general-purpose processor may be configured as different dedicated processors (e.g., including different hardware components) at different times. Therefore, the software is configured accordingly to a specific processor or hardware component, for example, to constitute a specific hardware component at one time and a different hardware component at another time. Hardware components can provide information to and receive information from other hardware components. Therefore, the described hardware components can be considered communicationally coupled.In the presence of multiple hardware components, communication can be achieved through signal transmission between two or more hardware components (e.g., via appropriate circuitry and buses). In embodiments where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, by storing and retrieving information in a memory structure accessible to the multiple hardware components. For example, a hardware component may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. Another hardware component may then access the memory device at a later time to retrieve and process the stored output. Hardware components may also initiate communication with input or output devices and may operate on resources such as collections of information. The various operations of the example methods described herein may be performed at least in part by one or more processors configured, either temporarily (e.g., via software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented components that operate to perform one or more of the operations or functions described herein. As used herein, a “processor-implemented component” refers to a hardware component implemented using one or more processors. Similarly, the methods described herein may be implemented at least in part by a processor, wherein a particular processor or processor is an example of hardware. For example, at least some operations of a method may be performed by one or more processors or processor-implemented components. Furthermore, the one or more processors may be operable to support the execution of the associated operations in a “cloud computing” environment or as “Software as a Service” (SaaS). For example, at least some operations may be performed by a group of computers (as an example of a machine including processors), wherein these operations may be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces (APIs)). The execution of certain operations may be distributed among processors, residing not only within a single machine but also deployed across multiple machines. In some example embodiments, the processor or processor-implemented component may reside in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other example embodiments, the processor or processor-implemented component may be distributed across multiple geographic locations.

[0100] In this context, a "processor" refers to any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (such as "commands," "opcodes," "machine codes," etc.) and generates corresponding output signals suitable for operating a machine. A processor can be, for example, a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Radio Frequency Integrated Circuit (RFIC), or any combination thereof. A processor can further be a multi-core processor having two or more independent processors (sometimes called "cores") capable of executing instructions simultaneously.

[0101] In this context, a "timestamp" refers to a series of characters or encoded information that identifies when a specific event occurred, such as giving a date and time, sometimes accurate to a fraction of a second.

Claims

1. A method comprising: Detect the client device at the location identified by geographic coordinates; Access a texture map corresponding to the location identified by the geographic coordinates, the texture map comprising a set of texels, the set of texels including at least a first texel, the first texel including a semantic label; Based on the semantic tags, the media content is obtained; as well as At the client device, the media content is presented within the rendering of the texels.

2. The method according to claim 1, wherein, Detecting the client device at the location identified by the geographic coordinates includes: The client device is detected within a geofence containing the location identified by the geographic coordinates.

3. The method according to claim 1, wherein, The semantic tags correspond to material parameters, and the media content is presented within the rendering of the texels based on the material parameters.

4. The method according to claim 3, wherein, The material parameters include one or more of the following: Roughness value; Metallic value; Mirror value; and Primary color value.

5. The method according to claim 1, wherein, The client device is associated with a user profile, which includes user profile data, and obtaining the media content includes: The media content is obtained based on the user profile data and the semantic tags.

6. The method according to claim 1, wherein, Obtaining the aforementioned media content includes: The media content is obtained based on the time data and the semantic tags.

7. The method according to claim 1, wherein, Detecting the client device at the location identified by the geographic coordinates includes: Image data, including a depiction of an object, is received from the client device. Identify objects depicted by the image data, the objects corresponding to the geographic coordinates; and Access the texture map based on the geographic location coordinates.

8. A system comprising: Memory; as well as At least one hardware processor coupled to the memory and including instructions to cause the system to perform operations, said operations including: Detect the client device at the location identified by geographic coordinates; Access a texture map corresponding to the location identified by the geographic coordinates, the texture map comprising a set of texels, the set of texels including at least a first texel, the first texel including a semantic label; Based on the semantic tags, obtain the media content; and At the client device, the media content is presented within the rendering of the texels.

9. The system according to claim 8, wherein, Detecting the client device at the location identified by the geographic coordinates includes: The client device is detected within a geofence containing the location identified by the geographic coordinates.

10. The system according to claim 8, wherein, The semantic tags correspond to material parameters, and the media content is presented within the rendering of the texels based on the material parameters.

11. The system according to claim 10, wherein, The material parameters include one or more of the following: Roughness value; Metallic value; Mirror value; and Primary color value.

12. The system according to claim 8, wherein, The client device is associated with a user profile, which includes user profile data, and obtaining the media content includes: The media content is obtained based on the user profile data and the semantic tags.

13. The system according to claim 8, wherein, Obtaining the aforementioned media content includes: The media content is obtained based on the time data and the semantic tags.

14. The system according to claim 8, wherein, Detecting the client device at the location identified by the geographic coordinates includes: Image data, including a depiction of an object, is received from the client device. Identify objects depicted by the image data, the objects corresponding to the geographic coordinates; and Access the texture map based on the geographic location coordinates.

15. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations including: Detect the client device at the location identified by geographic coordinates; Access a texture map corresponding to the location identified by the geographic coordinates, the texture map comprising a set of texels, the set of texels including at least a first texel, the first texel including a semantic label; Based on the semantic tags, the media content is obtained; as well as At the client device, the media content is presented within the rendering of the texels.

16. The non-transitory machine-readable storage medium according to claim 15, wherein, Detecting the client device at the location identified by the geographic coordinates includes: The client device is detected within a geofence containing the location identified by the geographic coordinates.

17. The non-transitory machine-readable storage medium according to claim 15, wherein, The semantic tags correspond to material parameters, and the media content is presented within the rendering of the texels based on the material parameters.

18. The non-transitory machine-readable storage medium according to claim 17, wherein, The material parameters include one or more of the following: Roughness value; Metallic value; Mirror value; and Primary color value.

19. The non-transitory machine-readable storage medium according to claim 15, wherein, The client device is associated with a user profile, which includes user profile data, and obtaining the media content includes: The media content is obtained based on the user profile data and the semantic tags.

20. The non-transitory machine-readable storage medium according to claim 15, wherein, Obtaining the aforementioned media content includes: The media content is obtained based on the time data and the semantic tags.

Images