Face-facing geometry flow

Abstract

The present disclosure provides computer-implemented methods, systems, and apparatus for optimizing data used in depicting a virtual world. A computing system accesses data associated with a plurality of three-dimensional tiles representing a portion of a virtual space. The computing system generates a plurality of sub-tiles, wherein a respective sub-tile of the plurality of sub-tiles represents a portion of the one or more objects associated with the viewing orientation. The computing system receives a request for data associated with one or more of the three-dimensional tiles, the request including a location and a viewing orientation within the virtual space. The computing system determines the one or more of the three-dimensional tiles based on the location. The computing system selects one or more sub-tiles from the one or more of the three-dimensional tiles based on the viewing orientation. The computing system transmits the selected one or more sub-tiles for display.

Description

Technical Field

[0001] The present disclosure generally relates to providing data for rendering images of a virtual world. More specifically, the present disclosure relates to optimizing data for rendering a three-dimensional view based on a viewing angle. Background Art

[0002] As computer technology advances, the number and types of services that can be provided to users have also increased significantly. Services provided via computer technology include providing data associated with displaying a three-dimensional virtual world. Such services can provide the data required to display a virtual world to a client system upon request. Summary of the invention

[0003] Various aspects and advantages of the embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0004] An example aspect relates to a computer-implemented method. The method includes accessing data associated with a plurality of three-dimensional tiles by a computing system including one or more computing devices, each of which represents a portion of a virtual space. The method further includes, for a corresponding three-dimensional tile in the plurality of three-dimensional tiles: generating a plurality of sub-tiles by the computing system, wherein a corresponding sub-tile in the plurality of sub-tiles represents a portion of one or more objects included in the corresponding three-dimensional tile associated with a specific viewing orientation. The method further includes receiving a request for data associated with one or more of the three-dimensional tiles by the computing system and from a client device, the request including a position and a viewing orientation within the virtual space. The method further includes determining, by the computing system, one or more of the three-dimensional tiles associated with the request based on the position. The method further includes selecting, by the computing system, one or more sub-tiles from one or more of the three-dimensional tiles based at least in part on the viewing orientation. The method further includes transmitting, by the computing system, the selected one or more sub-tiles to the client device for display.

[0005] Another example aspect of the present disclosure relates to a computing device. The computing device includes one or more processors; and a computer-readable memory. The computer-readable memory stores instructions that, when executed by the one or more processors, cause the device to access data associated with a plurality of three-dimensional tiles, each of which represents a portion of a virtual space. The instructions further cause the device to generate a plurality of sub-tiles for a corresponding three-dimensional tile in the plurality of three-dimensional tiles, wherein a corresponding sub-tile in the plurality of sub-tiles represents a portion of one or more objects included in the corresponding three-dimensional tile associated with a specific viewing orientation. The instructions further cause the device to receive a request for data associated with one or more of the three-dimensional tiles from a client device, the request including a position and a viewing orientation within the virtual space. The instructions further cause the device to determine one or more of the three-dimensional tiles associated with the request based on the position. The instructions further cause the device to select one or more sub-tiles from one or more of the three-dimensional tiles based at least in part on the viewing orientation. The instructions further cause the device to transmit the selected one or more sub-tiles to the client device for display.

[0006] Another example aspect of the present disclosure relates to a computer-readable medium storing instructions. These instructions, when executed by one or more processors, cause the one or more processors to access data associated with a plurality of three-dimensional tiles, each of which represents a portion of a virtual space. These instructions further cause the one or more processors to generate a plurality of sub-tiles for a corresponding three-dimensional tile in the plurality of three-dimensional tiles, wherein a corresponding sub-tile in the plurality of sub-tiles represents a portion of one or more objects included in the corresponding three-dimensional tile associated with a specific viewing orientation. These instructions further cause the one or more processors to receive a request for data associated with one or more of these three-dimensional tiles from a client device, the request including a position and a viewing orientation in the virtual space. These instructions further cause the device to determine one or more of the three-dimensional tiles associated with the request based on the position. These instructions further cause the one or more processors to select one or more sub-tiles from one or more of the three-dimensional tiles based at least in part on the viewing orientation. These instructions further cause the one or more processors to transmit the selected one or more sub-tiles to the client device for display.

[0007] Other aspects of the disclosure relate to various systems, apparatuses, non-transitory computer-readable media, user interfaces, and electronic devices.

[0008] These and other features, aspects and advantages of various embodiments of the present disclosure will be better understood with reference to the following description and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and, together with the description, serve to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] A detailed discussion of embodiments for those of ordinary skill in the art is set forth in this specification with reference to the accompanying drawings, in which:

[0010] Figure 1 An example computing environment including a user computing device according to an example embodiment of the present disclosure is shown;

[0011] Figure 2 An example client-server computing environment is shown according to an example embodiment of the present disclosure;

[0012] Figure 3 An example secondary tile generation system according to an example embodiment of the present disclosure is shown;

[0013] Figure 4 An example request processing system according to an example embodiment of the present disclosure is depicted;

[0014] Figure 5 An example diagram showing a plurality of sub-tiles according to an example embodiment of the present disclosure; and

[0015] Figure 6 An example flow chart according to an example embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0016] Reference will now be made in detail to embodiments of the present disclosure, one or more examples of which are shown in the accompanying drawings. Each example is provided by way of explanation of the present disclosure, rather than by way of limitation. In fact, it will be apparent to those skilled in the art that various modifications and changes may be made to the present disclosure without departing from the scope or spirit of the present disclosure. For example, a feature shown or described as part of one embodiment may be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to encompass modifications and changes as within the scope of the appended claims and their equivalents.

[0017] In general, the present disclosure relates to a system for increasing the efficiency of a data server system that provides data associated with a virtual depiction of a virtual world or a physical world. The amount of data associated with rendering and displaying a virtual world can be very large. Therefore, a large virtual world can be divided into multiple volumetric portions. These volumetric portions can be referred to as three-dimensional tiles, and the three-dimensional tiles can define the content and properties of a specific area or volume of the virtual world.

[0018] The server system may determine which three-dimensional tiles are needed to render one or more images from a particular viewpoint. For example, the three-dimensional tiles may be determined to be rendered based on visibility measurements. The visibility measurements may determine what should be rendered as visible based on the location of a viewing entity within the virtual world and the viewing orientation of the entity viewing the world.

[0019] Client systems may request information based on their location, and the server system may respond by determining which three-dimensional tiles are visible to the viewing entity. Each three-dimensional tile may include data describing the content of the associated volumetric portion of the virtual world. For example, a three-dimensional tile may include data describing the size, location, appearance, and orientation of one or more objects. In some examples, each object may include multiple faces arranged in a triangular mesh. A face is a flat surface or planar area that forms the boundary of a physical object.

[0020] In some cases, using data associated with an entire tile to render an image may be wasteful because some objects (or faces) cannot be viewed for every possible viewing orientation. If data describing objects and faces that cannot be viewed is used when rendering an image (or series of images) for display to a user, these objects and / or faces will be clipped. Power and processor time used to render and then clip these objects and faces is wasted.

[0021] In order to reduce the amount of wasted resources, the tile data may be preprocessed so that unviewable objects and faces are not used for rendering or are not transmitted to the client device for rendering. To this end, the data processing system may generate multiple sub-tiles for each of the multiple three-dimensional tiles representing the entire virtual world. In some examples, each sub-tile may include only the information necessary to display the content of the three-dimensional tile from a specific viewing orientation. For example, objects and faces visible when viewing the volume portion from above are different from objects and faces visible when viewing the volume portion from the west. Once a sub-tile has been generated for each of the multiple three-dimensional tiles, the sub-tile can be used instead of the entire three-dimensional tile to respond to a user request.

[0022] In some examples, the client system may send a request to a server system associated with the virtual world. In some examples, the request may include the viewer's position within the virtual world and the viewer's viewing orientation. It should be noted that viewing orientation is used herein to refer to the orientation (relative to a set of coordinates defined with respect to the virtual world) and the width, height, and depth of the viewing window.

[0023] Based on the received position, the server system may identify one or more relevant three-dimensional tiles that will be (at least partially) in the viewing area. The server system may use the viewing orientation to select one or more secondary tiles for each identified three-dimensional tile. The secondary tiles may include data that can be used to generate an image of the content of the one or more three-dimensional tiles as seen from the viewing orientation submitted with the request. The image (or images) may be rendered at the server system, at the client system, or one or more steps of the rendering process may be completed on some combination of both the client system and the server system.

[0024] Once sub-tiles have been generated for all three-dimensional tiles, the client system may transmit a request for tile information from a server system that has access to the stored sub-tiles. The server system may receive a request including a location within a virtual space and a viewing orientation. Using the location within the virtual space, the server system may identify one or more three-dimensional tiles associated with the space. The server system may use the viewing orientation to identify the specific perspective from which each of the identified tiles is being viewed. Using this information, the server system may select only the sub-tiles associated with the viewing orientation included in the request. In some implementations, only the selected sub-tiles are used to render a virtual image for display on the client system. By removing or otherwise not including elements that are not visible from certain perspectives, the system can significantly reduce the time and processing power required to render a scene for a user.

[0025] For example, a client system may transmit a request to a server system indicating a request for image data at a specific location in a virtual world when a viewer is looking west. The server system may receive the request and identify one or more three-dimensional tiles visible to a viewer looking west from the specific location. The server system may select one or more sub-tiles from each three-dimensional tile. The selected sub-tiles may be only those associated with a viewer viewing from the west. In this example, each three-dimensional tile is decomposed into six sub-tiles based on a top view, a bottom view, an east view, a west view, a north view, and a south view. Therefore, the server system may access the sub-tiles associated with viewing the three-dimensional tile from the west. The sub-tile data may be transmitted to the client system without including sub-tile data for data that will not be visible because they cannot be viewed by the user from that angle. When the image is later rendered for display, only those portions of the sub-tiles that are visible will be used during rendering, so rendering will require less memory and computing power, and the bandwidth required to transmit data to the client system will be reduced.

[0026] More specifically, a server computing system may be associated with a service that provides visualization data associated with a virtual world or a virtual augmentation for overlaying a physical world. In some examples, the virtual world may be associated with a video game engine, an augmented reality application, a virtual world application, or with a 3D Geographic Information System (GIS) implementation. In some examples, the entire data set describing such a virtual world may be so large that it is not feasible to store all of this data in the memory of a single computing device (e.g., a user's client computing device). Therefore, instead of storing all of the visualization data (e.g., data necessary to render an appropriate image of the virtual world) on the user computing device, a server system (e.g., one or more computing devices with which a client computing system can communicate over a network) may store this data and deliver the relevant visualization data to the client system upon request.

[0027] In order to effectively provide visualization data, the server system may divide the entire area of ​​the virtual world into a plurality of three-dimensional tiles, each of which is associated with a corresponding volumetric portion of the virtual world. For example, each three-dimensional tile may be of a standard size (e.g., a 10-meter cube). In other examples, the three-dimensional tiles may be of other sizes. In addition, in some examples, the three-dimensional tiles may have different sizes, depending on the amount of content included in each three-dimensional tile. Therefore, areas with a high object density in the virtual world may have smaller tiles, while areas with a lower density may have larger tiles. In some examples, the three-dimensional tiles may be arranged in a hierarchical structure so that the child tiles in the hierarchical structure include more details or textures at a higher resolution than the parent tiles.

[0028] In some examples, a volume portion (e.g., a three-dimensional tile) may include data describing one or more objects within a virtual world region associated with the three-dimensional tile. In some examples, the data may be described as pre-image data including a triangulated mesh of faces, information about object locations, textures to be overlaid on the objects, information about lighting, and the like. In some examples, the data may describe multiple faces associated with each object. These faces may be explicitly referenced by the data describing the object. In other examples, the faces may be determined based on the data. For example, if an object is represented by a wireframe sketch of points, the underlying set of points may be analyzed to determine the multiple faces that make up the object.

[0029] In some examples, each face may have an associated appearance, position, and orientation. The object and face data associated with each tile may be used to render the scene so that when an area of ​​the tile is to be displayed to a user, the system may generate an image appropriate for the view described by the request based on the data. Because objects in a three-dimensional tile are three-dimensional, some of the data describing the object will describe a portion of the object that is not visible from a particular viewing orientation.

[0030] Therefore, it would be advantageous to be able to discard the portion of the object that is not visible to a particular viewer at a particular location. Existing solutions may discard this additional data during the rendering process. In order to improve the efficiency of the system, the server system (or a related processing system) may generate multiple sub-tiles for each volume portion. In some examples, the multiple sub-tiles may include six sub-tiles, each of which represents a perspective from one side of the cube. Therefore, the sub-tiles may be associated with viewers above (+y dimension), viewers below (-y dimension), viewers in front (+z dimension), viewers behind (-z dimension), viewers on the left (-x dimension), and viewers on the right (+x dimension). Each sub-tile may be associated with a view that will be visible to the user from a specific viewing orientation. In this way, the three-dimensional tiles can be divided into different sub-tiles of priority, each of which is associated with a specific viewing orientation.

[0031] In some examples, the secondary tiles may be paired so that the secondary tile is paired with a secondary tile associated with an opposite viewing orientation. When a request is received, only one of the two tiles need be used to satisfy the request because no viewing angle will be able to see both the first secondary tile associated with a particular orientation and its opposite secondary tile, which represents a view from the opposite orientation.

[0032] Once the number of secondary tiles and their associated orientations are determined, the server system can access data associated with a particular three-dimensional tile. As described above, the data can include multiple faces associated with multiple objects in the three-dimensional tile. Each face can be analyzed to determine its orientation. The face can then be assigned to the secondary tile with the most similar orientation. For example, the cosine similarity between the angle of the corresponding face and the orientation of each secondary tile can be calculated. The corresponding face can be assigned to the secondary tile based on the results of the cosine similarity. Each face can be assigned to a single secondary tile.

[0033] Once all faces in a three-dimensional tile have been assigned to sub-tiles, all sub-tiles and associated faces can be stored at the server system for later retrieval. In some examples, a user may request visual information for a specific location within a virtual space. Such a request may be sent to a server system and may include the location of the requested visual data and the perspective from which the view is obtained. The server system may use this information to identify a specific three-dimensional tile associated with the request. For each associated three-dimensional tile, the server system may identify a specific sub-tile that the user is viewing. In some examples, a viewing orientation is associated with more than one sub-tile. For example, a viewer in the northeast can see some portions of both the north and east sides of the three-dimensional tile. Therefore, the server system may select all sub-tiles representing viewing orientations that are at least partially in view. The sub-tiles visible for a specific viewing orientation may be half or less than half of all sub-tiles.

[0034] The data associated with each selected secondary tile may be transmitted to the requesting user. In this case, the total amount of data transmitted is significantly reduced because data associated with faces that are not visible to the user is not transmitted. In other examples, the server system itself may render or perform one or more pre-rendering steps and then transmit the rendered or partially rendered data to the client system. In this case, the server system may reduce its workload by reducing the amount of data that needs to be used during the rendering step.

[0035] Alternatively, the server system may first identify a particular viewpoint that will be visible from the requested location to determine an appropriate secondary tile group. For example, if the secondary tiles are stored in groups such that all secondary tiles with similar viewing orientations are grouped together, the server system may determine which group is relevant to the request and then select a secondary tile from that group that is associated with the visible three-dimensional tile. Thus, if the viewpoint orientation is from the west, the server system may search within the west secondary tile group for a secondary tile that is associated with the visible three-dimensional tile.

[0036] The technical benefits of these systems and methods may include reducing the amount of data transmitted to the client system, so that less bandwidth and memory are required. In addition, the disclosed technology reduces the amount of data used to render the scene, resulting in less power usage and less processing time. Specifically, a server system that provides data for visualizing a virtual space can reduce the amount of data transmitted to the client system by half or more based on a specific viewpoint orientation. Since half (or less) of all sub-tiles are used to render any particular view, a similar reduction in power usage can be achieved. Both the server system and the client system reduce the need for bandwidth and memory to store related data. In addition, the rendering system (at the server or client) uses less power and processing time to render images for display. Reducing power usage can increase the service life of a computing system running on a battery. In general, these benefits reduce the time and computing cost of rendering images for display, and represent an improvement to the computing system as a whole.

[0037] Referring now to the accompanying drawings, example embodiments of the present disclosure will be discussed in more detail.

[0038] Figure 1 An example computing system 100 is depicted in accordance with an example embodiment of the present disclosure. In some example embodiments, computing system 100 may be any suitable device, including but not limited to a smartphone, a tablet, a laptop, a desktop computer, a server computing system, or any other computing device configured such that it may allow a person to execute an application associated with providing data for a virtual representation of a space or access a virtual world data service at a server computing system. Computing system 100 may include one or more processors 102, memory 104, one or more interface systems 110, a secondary tile generation system 130, a request response system 118, and a transmission system 120.

[0039] The one or more processors 102 may be any suitable processing device, such as a microprocessor, a microcontroller, an integrated circuit, or other suitable processing device. The memory 104 may include any suitable computing system or medium, including but not limited to non-transitory computer-readable media, RAM, ROM, a hard drive, a flash drive, or other memory devices. The memory 104 may store information accessible by the one or more processors 102, including instructions 108 executable by the one or more processors 102. The instructions may be any set of instructions that, when executed by the one or more processors 102, cause the one or more processors 102 to provide the desired functionality.

[0040] Specifically, in some devices, the memory 104 may store instructions for implementing one or more interface systems 110, sub-tile generation systems 130, request response systems 118, and transmission systems 120. The computing system 100 may implement one or more interface systems 110, sub-tile generation systems 130, request response systems 118, and transmission systems 120 to perform various aspects of the present disclosure, including accessing volumetric portion data (e.g., three-dimensional tile data) of multiple sub-portions of a virtual three-dimensional world, generating multiple sub-tiles for each three-dimensional tile (e.g., each tile) of the virtual world, storing the sub-tiles, receiving a request for data for generating a visual depiction of a three-dimensional space, selecting one or more sub-tiles for use in generating a visual depiction, and transmitting the selected one or more sub-tiles (or rendered images or pre-rendered image data) to a client computing device for display.

[0041] It should be understood that the term "system" or "engine" can refer to dedicated hardware, computer logic executed on a more general processor, or some combination thereof. Thus, the system or engine can be implemented in hardware, dedicated circuits, firmware, and / or software that controls a general-purpose processor. In one embodiment, the system can be implemented as a program code file stored on a storage device, loaded into a memory, and executed by a processor, or can be provided from a computer program product (e.g., computer executable instructions) stored in a tangible computer-readable storage medium (such as RAM, a hard disk, or an optical or magnetic medium).

[0042] The memory 104 may also include data 106, such as visualization data associated with a virtual world, which may be retrieved, manipulated, created, or stored by the one or more processors 102. In some example embodiments, such data may be accessed, displayed, or transmitted as desired.

[0043] In some example embodiments, computing system 100 includes one or more interface system 110 , secondary tile generation system 130 , request response system 118 , and transmission system 120 .

[0044] The one or more interface systems 110 may be interfaces that allow computing system 100 to interact with other computing devices over a network. In some examples, computing system 100 may be a server system that provides data to a user's client computing system for depicting a virtual world in a display associated with the user. In some examples, the data request is for data for displaying augmented reality visualizations that are displayed as layers on top of images of the physical world. In other examples, the virtual world may be associated with a virtual reality display, a virtual game world, a virtual representation of the physical world, or any other virtual depiction of a space.

[0045] In some examples, the one or more interface systems may include an application programming interface (API). The API may have an associated specification that describes a standard way to interact with the API to achieve a desired result provided by the service to which the API is associated. In this example, the API serves as an interface between one or more client computing systems and a server system associated with a virtual world service provided by computing system 100.

[0046] In some examples, the one or more interface systems 110 may request and receive requests from remote computing systems. For example, if data associated with a three-dimensional tile of a virtual world is stored at a remote location, the computing system 100 may request the tile data from the remote system as needed.

[0047] Sub-tile generation system 130 may include object analysis system 112, face assignment system 114, and storage system 116. In some examples, object analysis system 112 may access data associated with one or more volumetric portions (e.g., three-dimensional tiles) of a virtual space. In some examples, the tile data is accessed from visualization data storage 234. In other examples, the tile data may be accessed from a remote server via an interface system.

[0048] In some examples, the tile data may include data describing one or more objects located within the tile area. In other examples, the tile data may describe the size of the three-dimensional tile, one or more conditions that exist within the three-dimensional tile, and any other information required to visually represent the contents of a particular tile. In some examples, each tile may be part of a hierarchical data structure. In this type of hierarchical data structure, a root tile or parent tile may represent the entire volume portion. One or more child tiles may depend on the parent tile and may represent a sub-portion of the entire tile at a higher resolution. These child tiles may themselves have child tiles, so that tiles lower in the hierarchy have higher resolution data for a smaller total area than the parent tile. In this way, an area within a particular tile can be rendered at a higher resolution by including data from the child tiles of the particular tile being rendered.

[0049] Once the tile data has been accessed, the object analysis system 112 may analyze the tile data to identify one or more objects in the data. In some examples, the objects in the data may be defined based on data describing the location of the object within the tile, one or more points describing a wireframe of the object (e.g., an object boundary), data describing the location of the object within the tile, and data describing one or more textures to be applied to the object. For example, if one of the objects is a building, the data may describe where the building is located, including one or more points determining the location of each wall of the building (including the roof), and data describing the appearance of the building from each side (e.g., each face associated with the building).

[0050] Once an object is identified, the object analysis system 112 may determine one or more faces of the object. In some examples, an object is defined by a series of faces that include the outer boundaries of the object, and includes data on the appearance of the object. As described above, a face is a flat surface or planar area that forms the boundary of a solid object. Thus, a face may be read from the data or determined based on the location of one or more vertices. Each determined face may have an associated location, appearance, and angle. The face data may be transmitted to the face assignment system 114.

[0051] The face assignment system 114 may determine the number of sub-tiles to be generated for each three-dimensional tile. For example, the face assignment system 114 may generate six sub-tiles for each three-dimensional tile. The six sub-tiles may represent views of the tile from the top, bottom, front, back, left, and right. These viewpoints may also be referred to as x, y, and z dimensions. In some examples, a particular sub-tile may have an associated opposite sub-tile. The opposite sub-tile may have a viewing orientation opposite to the corresponding sub-tile. For example, a sub-tile associated with a view from the top may be paired with a sub-tile associated with a view from the bottom. The two sub-tiles are paired because the viewing angle from the top is opposite to the viewing angle from the bottom. When the x, y, and z terms are used to refer to the viewpoint of a particular sub-tile, the opposite sub-tile of the +x sub-tile may be a -x sub-tile.

[0052] The face assignment system 114 may determine, for each face in a particular tile, which secondary tile the face should be assigned to. To do this, the face assignment system 114 may compare the angle associated with the face and the viewing angle associated with the secondary tile. The face may then be assigned to the secondary tile with which it is most closely associated. In some examples, this comparison may be accomplished by determining a normal vector for each face and determining an angle between the two normal vectors (e.g., using cosine similarity). Other methods of determining differences may be used.

[0053] The face assignment system 114 may then assign a given face to the sub-tile that has an associated viewing angle that most closely matches the angle of the face. In this manner, all faces may be assigned to one sub-tile. In some examples, if the tile data is arranged in a hierarchical structure, the data associated with the sub-tile may also have faces assigned to a particular sub-tile. In this manner, the sub-tile may contain all necessary data for display at the requested resolution. Once all faces have been assigned, the sub-tile may be transferred to the storage system 116. The storage system 116 may store the sub-tile data (including the associated face data) in the visualization data storage device 234.

[0054] In some examples, computing system 100 includes a request response system. Request response system 118 can receive a request for visual data of a specific location in a virtual world from a client system via one or more interface systems. In some examples, the request can include a location within the virtual world and a perspective from which the location will be viewed.

[0055] In some examples, request response system 118 may analyze the request to identify one or more tiles associated with a location and / or viewing orientation. For example, the request may include a set of coordinates associated with a particular location in the virtual world. Request response system 118 may identify the tile in which the location is located. In addition to the tile in which the location included in the request is located, request response system 118 may also determine which other tiles will be in view based on the viewing orientation of the request and / or the distance at which the viewer can see the object.

[0056] For each tile determined to be included in the viewing window, the request response system 118 may select one or more secondary tiles based on the angle at which the tile is viewed. In some examples, the selected secondary tiles may be less than half of all the total secondary tiles. For example, only one of each secondary tile pair may be selected at a time because they represent opposite viewing angles. In some examples, the transmission system 120 may transmit the selected secondary tiles to the requesting client system.

[0057] Additionally or alternatively, the selected secondary tiles may be used by computing system 100 to render one or more images for transmission to a client system for display. Additionally or alternatively, the selected secondary tiles may be used to perform one or more pre-processing steps that may produce a processed version of the secondary tiles. The processed version of the secondary tiles may then be transmitted to a requesting client system via transmission system 120 for rendering.

[0058] Figure 2An example client-server environment 200 is depicted according to an example embodiment of the present disclosure. The client-server system environment 200 includes one or more client systems 210 and a server computing system 230. One or more communication networks 220 may interconnect these components. The communication network 220 may be any of a variety of network types, including a local area network (LAN), a wide area network (WAN), a wireless network, a wired network, the Internet, a personal area network (PAN), or a combination of these networks.

[0059] Client system 210 may include, but is not limited to, a personal computer, a laptop, a computing system integrated into a vehicle, a virtual reality system, an augmented reality device (e.g., smart glasses), etc. In some examples, client system 210 may be any computing system that can be used to display visual data associated with a virtual world (or visual data associated with an augmented reality system), request visual data from server computing system 230, and display the resulting rendered image.

[0060] In some examples, client system 210 may include one or more applications 212, such as a search application, a communication application, a navigation application, a productivity application, a gaming application, a word processing application, or any other application. The application may include a web browser. Client system 210 may use a web browser (or other application) to send and receive requests to and from server computing system 230. The application may include a virtual world visual data display system that displays rendered images based on a location within a virtual world.

[0061] In some examples, client system 210 may be associated with a user of an application associated with a virtual world. For example, application 212 may be a gaming application with an associated virtual world. In some examples, application 212 may be a virtual representation of a physical world. The user may navigate in the virtual world, and thus, application 212 may display an updated representation of the virtual world based on the user's position and viewing orientation within the virtual world. Instead of storing the data necessary to visually depict the entire virtual world at client system 210, server computing system 230 may provide data for visually depicting the entire virtual world. To receive the data, client system 210 may send a request to server computing system 230. The request may include a description of the location and viewing area within the virtual world, including height, width, depth, and resolution. Server computing system 230 may respond with a fully rendered image to be displayed or data that still needs to be rendered locally before it is ready for display to the user.

[0062] like Figure 2 As shown, the server computing system 230 may generally be based on a three-tier architecture consisting of a front-end layer, an application logic layer, and a data layer. As those skilled in the relevant computer and Internet related fields understand, Figure 2 Each component shown may represent an executable software instruction set and corresponding hardware (e.g., memory and processor) for executing these instructions. To avoid unnecessary detail, the following description has been omitted. Figure 2 Various components and engines that are not germane to conveying an understanding of the various examples are omitted. However, those skilled in the art will readily recognize that various additional components and engines may be used with the example Figure 2 The server computing system 230 shown is used in conjunction with the server computing system 230 to facilitate additional functionality not specifically described herein. Figure 2 The various components depicted in the FIG2000 may reside on a single server computer or may be distributed across several server computers in various arrangements. Figure 2 , but the various example embodiments are by no means limited to this architecture.

[0063] like Figure 2 As shown, the front end may consist of an interface system 222 that receives communications from one or more server-client environments 200 and transmits appropriate responses to the server-client environments 200. For example, the interface system 222 may receive requests in the form of hypertext transfer protocol (HTTP) requests or other web-based application programming interface (API) requests. The client system 210 may execute a traditional web browser application or an application that has been developed for a specific platform to include any of a variety of computing devices and operating systems.

[0064] like Figure 2 As shown, the data layer may include a visualization data storage device 234. The visualization data storage device 234 may store various data useful for generating an image of the virtual world for display to the user. The data describing the three-dimensional world may be stored as a three-dimensional model of the world. For example, the data may be stored as a three-dimensional mesh having a series of points defining the boundaries of the object and accompanying texture data for applying the appropriate image to the 3D model. Other three-dimensional modeling techniques may be used, such as parametric models, voxel-based models, etc. In some examples, other representations such as 3D point clouds or depth maps may also be used.

[0065] The three-dimensional space of the virtual world is divided into multiple volumetric portions. These volumetric portions may be referred to as three-dimensional tiles. Each three-dimensional tile may be associated with a specific portion of the three-dimensional virtual world. Each three-dimensional tile may have one or more objects within it, which are depicted when the image of the tile is generated. These objects may be defined based on a three-dimensional model based on points at the vertices of the objects. As described above, other methods of representing objects may be used. In addition to vertices, textures may be used to determine the appearance of an object when it is represented in a user's view.

[0066] The visualization data storage device 234 may also store the secondary tile data when it has been generated. Each secondary tile is associated with a particular tile and is associated with a particular viewing angle of the tile. For example, a particular secondary tile may represent a view of a three-dimensional tile when viewed from above. In some examples, six secondary tiles are generated for each three-dimensional tile. Additionally or alternatively, a different number of tiles may be used depending on the needs of the three-dimensional world and the display system.

[0067] The sub-tile data may be stored in visualization data storage 234. Once stored, the sub-tile data may be used to respond to requests from client system 210. Upon receiving the request, server computing system 230 may identify the appropriate sub-tiles and prepare them for use in generating an image for display at client system 210.

[0068] The secondary tile generation system 240 may be used to process data associated with a plurality of three-dimensional tiles to generate a plurality of secondary tiles for each tile. As described above, an example of the number of secondary tiles may be six secondary tiles. However, other numbers of secondary tiles may be used as desired.

[0069] To generate a secondary tile, the secondary tile generation system 240 may access data associated with a particular three-dimensional tile. The secondary tile generation system 240 may identify multiple faces of a particular three-dimensional tile. In some examples, the three-dimensional data stored for the tile includes points representing vertices of a wireframe sketch of one or more objects. In this case, the secondary tile generation system may process these points to identify one or more faces of the object. In other examples, the tile data includes data that describes the face. The data associated with the face may be data describing the size of the face, the angle of the face, and any other information associated with the appearance of the face (e.g., its color, shape, any texture associated with it, etc.).

[0070] Once the location, size, orientation, and appearance of each face in a particular tile have been identified, the sub-tile generation system 240 can assign each face to a particular sub-tile. In order to assign faces to sub-tiles, the sub-tile generation system 240 can compare the angle of the face to the viewing angle or viewing orientation associated with each sub-tile. Thus, faces that are primarily facing upward can be assigned to a sub-tile associated with a view when viewed from above. Similarly, faces facing west can be associated with a sub-tile viewed from the west. Each service can be assigned to a specific sub-tile until all services have been assigned. Once all faces have been assigned, the sub-tile can be stored in the visualization data storage device 234.

[0071] In some examples, request response system 242 may receive the request via interface system 222. In some examples, the request may include a location within the virtual world, a viewing orientation, and a size of a viewing window. Request response system 242 may determine one or more three-dimensional tiles that may be selected based on the received location, viewing orientation, and size of the viewing window. The selected three-dimensional tiles may be used to generate an image of the virtual world. It should be noted that the request may also include a viewing resolution.

[0072] Request response system 242 may access one or more secondary tiles for each tile to be shown in the image associated with the user request from visualization data storage device 234. The secondary tiles selected are selected based on the viewing orientation of the user. In this way, only secondary tiles that include faces that will be visible from the viewpoint are selected. In this way, faces that are obscured by the viewpoint (e.g., on the back of the object from the viewpoint of the request) may be excluded from the response to the request. This allows the rendering of the image to be more efficient.

[0073] In some examples, the request response system 242 performs one or more pre-rendering steps on the selected sub-tiles before transmitting them to the requesting client system 210. In some examples, the request response system 242 performs the entire rendering operation and transmits one or more images to the client system 210 in response to the request. In other examples, the sub-tile data itself is transmitted to the client system without any pre-processing. In all three cases, the data transmitted to the client system is reduced, so that less bandwidth is used and / or the amount of data required during the rendering process is reduced, thereby increasing the efficiency and speed of the rendering process while reducing the amount of power and processing time required.

[0074] Figure 3 An example secondary tile generation system 130 is shown according to an example embodiment of the present disclosure. The secondary tile generation system 130 may include a data access system 112, a face recognition system 322, a position calculation system 324, and a tile assignment system 326.

[0075] For example, the secondary tile generation system 130 may generate a series of secondary tiles for each of a plurality of 3D tiles associated with the virtual world. By pre-generating these secondary tiles (once or periodically as needed), the server computing system may improve the efficiency of providing visual information to the client system. For example, the server system may reduce the total amount of information transmitted to the client system and / or reduce the amount of computation required to render one or more images of the virtual world.

[0076] To this end, the data access system 112 may access data for one or more three-dimensional tiles. In some examples, the data for the tiles is stored in the visualization data storage device 234. Additionally or alternatively, the data associated with the tiles may be stored at a remote computing system, and the secondary tile generation system 130 uses the data access system 112 to access the remote computing system via a network.

[0077] The tile data may be provided to the face recognition system 322. The face recognition system 322 may identify one or more services contained within a particular tile. As described above, in some examples, the tile data will include a direct description of the multiple faces that comprise the objects within the tile. In other examples, the face recognition system 322 may perform analysis on the three-dimensional 3D data to identify multiple tiles based on vertex or point data present within the tile data. For each face, the face recognition system 322 may identify the location, size, and appearance.

[0078] The list of faces may be transmitted to the position calculation system 324. The position calculation system 324 may determine, for each face, a position or angle associated with the face. In some examples, the angle associated with the face may be determined based on the position and position of the face within the virtual world.

[0079] Once the orientation has been calculated for each face, the tile assignment system 326 may assign each face to the secondary tile that is most closely aligned with it. For example, if a secondary tile is associated with a view that a user would see from above the tile, then faces that are primarily oriented toward the top of the tile may be assigned to that secondary tile. The same process may be followed for each face until all faces are associated with a particular secondary tile. The tile assignment system 326 may store the secondary tiles in the visualization data storage device 234.

[0080] Figure 4 An example request processing system according to an example embodiment of the present disclosure is depicted. The request response system 118 may include a request receiving system 402, a tile identification system 422, a secondary tile selection system 424, and a transmission system 120. Each of these systems may access data from the visualization data storage device 234 to perform their tasks.

[0081] The request receiving system 402 can receive a request for visualization data from the client system 210. The request can include a location within the virtual world, a viewer location, a viewing window size, and a resolution value. Once the request receiving system 402 receives the request, the relevant data can be passed to the tile recognition system 422.

[0082] Tile identification system 422 can determine which specific three-dimensional tiles or portions of the virtual world will be used to generate the requested image based on the data in the request. For example, the three-dimensional tile at the spatial location can be selected, as well as one or more secondary tiles in the direction the view is pointing. If necessary, additional three-dimensional tiles can be selected based on the width or height of the viewing hole and additional data associated with the increased resolution.

[0083] In some examples, the tile recognition system 422 will select only one or two 3D tiles. However, in some examples, more than two tiles may be selected based on the viewer's position, viewing orientation, range of view, and desired resolution. In some examples, additional details for additional resolutions may be determined based on a hierarchical set of 3D tiles, where child 3D tiles provide additional details for parent 3D tiles.

[0084] Once one or more three-dimensional tiles have been selected, the selected tiles may be communicated to the secondary tile selection system 424. The secondary tile selection system 424 may select one or more secondary tiles for each identified tile. The secondary tiles are selected so that the selected secondary tiles represent only those secondary tiles that are viewable from the user's viewing position. Thus, sometimes tiles associated with the back or far side of a three-dimensional tile will not be selected. In this manner, the secondary tile selection system 424 may select half or less than half of the total number of available secondary tiles.

[0085] In some examples, the transmission system 426 may receive the sub-tiles selected by the sub-tile selection system 424 and transmit them to the requesting client system 210. In some examples, the request response system 118 or an associated server system may perform one or more pre-processing steps to render an image from the data. In some examples, the selected sub-tile data may be sent to the requesting client system 210 without any pre-processing.

[0086] Figure 5 An example diagram of multiple sub-tiles according to an example embodiment of the present disclosure is shown. The view includes a three-dimensional tile 502, in which one or more objects are depicted as being inside. The three-dimensional tile 502 can be divided into multiple sub-tiles. In this example, three sub-tiles are depicted, but a total of six sub-tiles are implied. The depicted sub-tiles include a sub-tile 504 associated with a positive x-direction, a sub-tile 510 associated with a positive y-direction, and a sub-tile 508 associated with a negative z-direction.

[0087] Not depicted here, but implied, are corresponding sub-tiles in the negative direction of each of x and y, and a sub-tile corresponding to the positive direction of z. As depicted here, the sub-tiles include only the faces associated with a user viewing from that direction. Thus, the positive x sub-tile 504 includes a front view of the object. Similarly, the positive y sub-tile 510 and the z negative sub-tile 508 include only those faces most closely associated with viewing from a particular direction.

[0088] Likewise, although not shown, the corresponding negative sub-tile will include, for example, a view from the bottom of all objects (if the objects are fully visible from the bottom). Similarly, the negative X and negative Y sub-tiles will display the objects from the opposite direction of their corresponding positive sub-tiles. In this way, any viewing orientation can be represented as a combination of up to three sub-tiles. In this way, data associated with other sub-tiles can be omitted from the transmission step and / or rendering step, thereby improving efficiency.

[0089] Figure 6 An example flow chart according to an example embodiment of the present disclosure is shown. One or more parts of the method can be implemented by one or more computing devices, such as the computing devices described herein. In addition, one or more parts of the method can be implemented as an algorithm on a hardware component of the device described herein. Figure 6 Elements are depicted as being performed in a specific order for purposes of illustration and discussion. One of ordinary skill in the art, using the disclosure provided herein, will appreciate that elements of any method discussed herein may be adjusted, rearranged, expanded, omitted, combined, and / or modified in various ways without departing from the scope of the present disclosure. The method may be performed by, for example, Figures 1 to 4 The invention may be implemented by one or more computing devices of one or more of the computing devices depicted in the specification.

[0090] At 602, a computing system (e.g., Figure 1 A computing system 100 in a computer system may access data associated with a plurality of three-dimensional tiles, each tile representing a portion of a virtual space. In some examples, computing system 100 is a server computing system. In some examples, the virtual space may be associated with a virtual world. Such a virtual world may be associated with a gaming application or a virtual reality application. In some examples, the virtual space is associated with a geographic location. For example, a virtual replica of the physical world based on satellite imagery.

[0091] In some examples, a three-dimensional tile may be associated with a volumetric portion of a virtual world. The data associated with each respective three-dimensional tile of the plurality of three-dimensional tiles may include data for one or more objects within the portion of the virtual space associated with the respective three-dimensional tile defining the respective three-dimensional tile. In some examples, the one or more objects include a plurality of faces, each face having a size, an appearance, and an orientation.

[0092] In some examples, the one or more objects may be defined by one or more points representing the boundaries of the object. However, other methods of representing objects may be used to represent data of objects within a particular three-dimensional tile. For example, objects may be represented using point clouds, range images, grids, parameter definitions, voxels, etc.

[0093] For a corresponding 3D tile among the plurality of 3D tiles, the computing system 100 may generate a plurality of sub-tiles at 604, wherein a corresponding sub-tile among the plurality of sub-tiles represents a portion of one or more objects included in the corresponding 3D tile that is viewable from a specific viewing orientation. For example, a sub-tile may represent an object (and face) that is visible from the perspective of a viewer viewing the 3D tile from a specific viewing orientation.

[0094] In some examples, computing system 100 may determine, for each face of an object associated with a corresponding three-dimensional tile, a secondary tile having a viewing orientation that most closely matches the orientation of the corresponding face. For example, computing system may calculate the similarity between the angle of a particular face and the angle of each secondary tile. This calculation may be performed using a cosine similarity method. Other methods may also be used.

[0095] The computing system 100 may assign a particular face to the secondary tile that is most similar to it. In this manner, each face in a particular three-dimensional tile may be assigned to a particular secondary tile. In some examples, the plurality of secondary tiles include six secondary tiles for each three-dimensional tile. In some examples, the plurality of secondary tiles include one or more secondary tile pairs, the secondary tile pairs including two secondary tiles associated with opposite viewing orientations.

[0096] In some examples, at 606, the computing system may receive a request from a client device for data associated with one or more three-dimensional tiles, the request including a location within the virtual space and a viewing orientation. In some examples, the location within the virtual space is determined based on a geographic location. In some examples, the request may also include a requested resolution and field of view (e.g., the width, height, and depth of the data to be displayed).

[0097] In some examples, at 608, computing system 100 may determine one or more three-dimensional tiles associated with the request based on the location. At 610, computing system 100 may select one or more secondary tiles from the one or more three-dimensional tiles based at least in part on the viewing orientation. Selecting the secondary tiles may include calculating the difference between the viewing angle and the viewing orientation associated with each potential secondary tile. Computing system 100 may select up to half of all possible secondary tiles based on the viewer's location and viewing orientation.

[0098] At 612, computing system 100 may transmit the selected one or more secondary tiles to a client device for display. In some examples, computing system 100 may generate one or more images from the one or more secondary tiles using a rendering process before transmitting the selected one or more secondary tiles to a client device for display. Additionally or alternatively, computing system 100 may generate pre-rendering data to enable a client system to generate images for display using a rendering process.

[0099] The technology discussed herein relates to sensors, servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a variety of possible configurations, combinations, and partitioning of tasks and functions between and among components. For example, the processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0100] Although the subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided by way of explanation, rather than limitation of the present disclosure. Those skilled in the art, after understanding the foregoing, may easily produce changes, modifications, and equivalents to these embodiments. Therefore, the present disclosure does not exclude such modifications, changes, and / or additions to the subject matter that would be readily understood by those of ordinary skill in the art. For example, a feature shown or described as part of one embodiment may be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover these changes, modifications, and equivalents.

Claims

1. A computer-implemented method, the method comprising: Accessing, by a computing system including one or more computing devices, data associated with a plurality of three-dimensional tiles, each three-dimensional tile representing a portion of a virtual space; For a corresponding three-dimensional image tile among the plurality of three-dimensional image tiles: generating, by the computing system, a plurality of sub-tiles, wherein a corresponding sub-tile of the plurality of sub-tiles represents a portion of one or more objects included in the corresponding three-dimensional tile that is associated with a specific viewing orientation; receiving, by the computing system and from a client device, a request for data associated with one or more of the three-dimensional tiles, the request comprising a position within the virtual space and a viewing orientation; determining, by the computing system, the one or more of the three-dimensional tiles associated with the request based on the location; selecting, by the computing system, one or more secondary tiles from the one or more of the three-dimensional tiles based at least in part on the viewing orientation; as well as The selected one or more secondary tiles are transmitted by the computing system to the client device for display.

2. A computer-implemented method as described in claim 1, wherein the data associated with the multiple three-dimensional tiles includes data for each three-dimensional tile defining one or more objects within the portion of the virtual space associated with the corresponding three-dimensional tile. 3 . The computer-implemented method of claim 2 , wherein the one or more objects include a plurality of faces, each face having a size, an appearance, and an orientation.

4. The computer-implemented method of claim 3, wherein generating the plurality of secondary tiles comprises: For each face associated with the corresponding three-dimensional image tile, determining a secondary image tile having a viewing orientation that most closely matches the orientation of the corresponding face; as well as The corresponding surface is assigned to the determined sub-tile. 5 . The computer-implemented method of claim 1 , wherein the plurality of sub-tiles comprises six sub-tiles for each three-dimensional tile.

6. The computer-implemented method of any preceding claim, wherein the plurality of secondary tiles comprises one or more secondary tile pairs, each secondary tile pair comprising two secondary tiles associated with opposite viewing orientations.

7. A computer-implemented method as claimed in any preceding claim, wherein each face is assigned to only a single secondary tile.

8. A computer-implemented method as claimed in any preceding claim, wherein the virtual space is associated with a geographical location.

9. The computer-implemented method of claim 8, wherein the position within the virtual space is determined based on the geographic location.

10. The computer-implemented method of any preceding claim, wherein the virtual space is associated with an augmented reality application.

11. A computer implemented method as claimed in any preceding claim, wherein the virtual space is associated with a gaming application.

12. The computer-implemented method of any preceding claim, further comprising, before transmitting the selected one or more secondary tiles to the client device for display: One or more images are generated from the one or more secondary tiles using a rendering process.

13. The computer-implemented method of any preceding claim, wherein transmitting the selected one or more secondary tiles to a client device for display comprises: Prerender data is generated by the one or more computing devices to enable the client system to generate an image for display using a rendering process.

14. A computing system comprising: one or more processors; and A computer readable memory storing instructions which, when executed by the one or more processors, cause the computing system to perform the method of any one of claims 1 to 13.

15. A computer readable medium storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform the method of any one of claims 1 to 13.

Images