Similarity analysis of three-dimensional (3D) objects

Abstract

The present invention relates to a method, a system, and a computer-readable medium for performing similarity analysis on three-dimensional (3D) objects. In some implementations, the method includes determining geometric asset features of a candidate 3D object based on multiple images of the candidate 3D object, determining a semantic feature vector of the candidate 3D object, determining a similarity between the candidate 3D object and a reference 3D object, and classifying the candidate 3D object based on the similarity. The method may further include calculating a first vector distance between the geometric asset features of the candidate 3D object and the geometric asset features of the reference 3D object, calculating a second vector distance between the semantic feature vector of the candidate 3D object and the semantic feature vector of the reference 3D object, and generating a fused vector distance by combining the first vector distance and the second vector distance.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Patent Application No. 18 / 427,230, filed January 30, 2024, entitled "Similarity Analysis of Three-Dimensional (3D) Objects," which is a continuation-in-part of U.S. Patent Application No. 18 / 231,474, filed August 8, 2023, entitled "Classification of Three-Dimensional (3D) Objects in a Virtual Environment," which claims priority to U.S. Provisional Application No. 63 / 454,852, filed March 27, 2023, entitled "Classification of Three-Dimensional (3D) Objects." This application also claims the benefit of priority to U.S. Provisional Application No. 63 / 620,967, filed January 15, 2024, entitled "Similarity Analysis of Three-Dimensional (3D) Objects." The contents of U.S. Patent Application Nos. 18 / 427,230, 18 / 231,474, 63 / 454,852, and 63 / 620,967 are incorporated herein by reference in their entireties.

[0002] Embodiments relate generally to computer-based virtual experiences, and more particularly to methods, systems, and computer-readable media for three-dimensional (3D) object similarity analysis and detection of counterfeit and / or similar 3D objects. [Background technology]

[0003] Some online virtual experience platforms allow users to connect with each other, interact with each other (e.g., within virtual experiences), create virtual experiences, and share information with each other over the Internet. Users of online virtual experience platforms may participate in multiplayer environments (e.g., in virtual three-dimensional environments), design custom environments, design characters, 3D objects, and avatars, decorate avatars, exchange virtual items / objects with other users, communicate with other users using audio or text messages, etc. Environments such as metaverse or multiverse environments may also allow participating users to share, sell, or trade objects they create with other users.

[0004] The background art discussion provided herein is intended to provide a context for the present disclosure. The work of the named inventors to the extent described in this background art section, and aspects of the description that may not otherwise qualify as prior art at the time of filing, are not admitted explicitly or implicitly as prior art to any prior disclosure. Summary of the Invention [Means for solving the problem]

[0005] One or more computer systems may be configured to perform specific operations or actions by having software, firmware, hardware, or a combination thereof installed on the system that causes the system to perform the actions during operation. One or more computer programs may be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform an action. One broad aspect includes a computer-implemented method including: determining geometric asset features of a candidate 3D object based on a plurality of images of the candidate 3D object; determining a semantic feature vector for the candidate 3D object; determining a similarity between the candidate 3D object and a reference 3D object based on a comparison of the geometric asset features and semantic feature vector of the candidate 3D object with the geometric asset features and semantic feature vector of a reference 3D object; and classifying the candidate 3D object based on the similarity between the candidate 3D object and the reference 3D object. Other embodiments of this aspect include corresponding computer systems, devices, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0006] The implementation may include a computer-implemented method in which determining a similarity between the candidate 3D object and the reference 3D object may include calculating a first vector distance between a geometric asset feature of the candidate 3D object and a geometric asset feature of the reference 3D object, calculating a second vector distance between a semantic feature vector of the candidate 3D object and a semantic feature vector of the reference 3D object, and generating a fused vector distance by combining the first vector distance and the second vector distance. Classifying the candidate 3D object may include determining whether the fused vector distance satisfies a non-authentic object threshold, and classifying the candidate 3D object as a non-authentic object if the fused vector distance satisfies the non-authentic object threshold, and classifying the candidate 3D object as a genuine object if the fused vector distance does not satisfy the non-authentic object threshold. Classifying the candidate 3D object may include determining whether the fused vector distances satisfy a similarity threshold, classifying the candidate 3D object as an object similar to the reference 3D object if the fused vector distances satisfy the similarity threshold, and classifying the candidate 3D object as an object dissimilar to the reference 3D object if the fused vector distances do not satisfy the similarity threshold. Combining the first vector distance and the second vector distance may further include applying respective transformation functions to the first vector distance and the second vector distance. Each updated feature vector represents the fused vector distance of the corresponding 3D object to other 3D objects among the plurality of 3D objects, and the dimensionality of each updated feature vector is lower than the dimensionality of the corresponding geometric asset feature and the corresponding semantic feature vector.Determining geometric asset features of the candidate 3D object may include determining one or more histogram of oriented gradients (HOG) vectors for each image of a plurality of images of the candidate 3D object, and calculating geometric asset features of the candidate 3D object based on the one or more HOG vectors for each image of the plurality of images of the candidate 3D object. Determining semantic feature vectors of the candidate 3D object may include obtaining one or more images of the candidate 3D object, and analyzing the one or more images with a pre-trained machine learning model to obtain semantic feature vectors of the candidate 3D object, where the machine learning model is trained by contrastive learning based on predicting matching pairs of images and associated text from a training dataset, and the semantic feature vector is a high-dimensional vector, and each dimension of the high-dimensional vector encodes respective semantic information. The step of classifying the candidate 3D object may include a step of determining the uniqueness of the candidate 3D object, and the step of determining the uniqueness of the candidate 3D object may include a step of determining corresponding fused vector distances between the candidate 3D object and each of a plurality of reference 3D objects; a step of determining a plurality of adjacent 3D objects of the candidate 3D object; a step of determining a local density of the candidate 3D object by calculating an average fused vector distance between the candidate 3D object and the plurality of adjacent 3D objects; and a step of determining a uniqueness score based on the local density of the candidate 3D object and the maximum local density of the plurality of reference 3D objects.Prior to determining the similarity between the candidate 3D object and the reference 3D object, the method may further include: obtaining a first plurality of semantically similar reference 3D objects; obtaining a second plurality of geometrically similar reference 3D objects; forming a combined pool of geometrically similar and semantically similar reference 3D objects based on the first plurality of semantically similar reference 3D objects and the second plurality of geometrically similar reference 3D objects; and selecting a reference 3D object from the combined pool of geometrically similar and semantically similar reference 3D objects. Each image of the plurality of images of the candidate 3D object is from a respective camera position of the two or more camera positions. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0007] One broad aspect includes a computer-implemented method including: generating a plurality of images of a candidate 3D object, each image of the plurality of images of the candidate 3D object from a respective one of two or more camera positions; determining geometric asset features of the candidate 3D object based on the plurality of images of the candidate 3D object; determining respective distances between the geometric asset features of the candidate 3D object and the geometric asset features of each of a plurality of reference 3D objects; determining a plurality of neighboring 3D objects of the candidate 3D object based at least in part on the respective distances; determining a local density of the candidate 3D object by calculating an average distance between the candidate 3D object and the plurality of neighboring 3D objects; and determining a geometric uniqueness score based on the local density of the candidate 3D object and the maximum local density of the plurality of reference 3D objects. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the acts of the method.

[0008] Implementations may include a computer-implemented method, wherein determining a plurality of neighboring 3D objects of the candidate 3D object may include applying an approximate nearest neighbor technique to geometric asset features of the candidate 3D object. The computer-implemented method may include determining a price for the candidate 3D object based on the geometric uniqueness score. The computer-implemented method may include displaying a price for the candidate 3D object on a user interface. The computer-implemented method may include determining an asset monitoring index for the candidate 3D object based on the geometric uniqueness score. The computer-implemented method may include determining a market valuation index for the candidate 3D object based on the geometric uniqueness score. Implementations of the described techniques may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0009] One broad aspect includes a non-transitory computer-readable medium having stored thereon instructions that, when executed, perform operations including determining geometric asset features of a candidate 3D object based on a plurality of images of the candidate 3D object, determining a semantic feature vector of the candidate 3D object, determining a similarity between the candidate 3D object and a reference 3D object based on a comparison of the geometric asset features and semantic feature vector of the candidate 3D object with the geometric asset features and semantic feature vector of the reference 3D object, and classifying the candidate 3D object based on the similarity between the candidate 3D object and the reference 3D object. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the acts of the method.

[0010] The implementation may include a non-transitory computer-readable medium, in which determining the similarity between the candidate 3D object and the reference 3D object may include calculating a first vector distance between a geometric asset feature of the candidate 3D object and a geometric asset feature of the reference 3D object, calculating a second vector distance between a semantic feature vector of the candidate 3D object and a semantic feature vector of the reference 3D object, and generating a fused vector distance by combining the first vector distance and the second vector distance. Classifying the candidate 3D object may further include determining whether the fused vector distance satisfies a similarity threshold, and classifying the candidate 3D object as an object similar to the reference 3D object if the fused vector distance satisfies the similarity threshold, and classifying the candidate 3D object as an object dissimilar to the reference 3D object if the fused vector distance does not satisfy the similarity threshold. Implementations of the described techniques may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0011] One broad aspect includes a system including a memory storing instructions and a processing device coupled to the memory, the processing device configured to access the memory and execute the instructions, where execution of the instructions causes the processing device to perform operations that may include determining geometric asset features of the candidate 3D object based on a plurality of images of the candidate 3D object, determining a semantic feature vector for the candidate 3D object, determining a similarity between the candidate 3D object and a reference 3D object based on a comparison of the geometric asset features and semantic feature vector of the candidate 3D object with the geometric asset features and semantic feature vector of the reference 3D object, and classifying the candidate 3D object based on the similarity between the candidate 3D object and the reference 3D object. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the acts of the method.

[0012] An implementation may include a system in which determining a similarity between a candidate 3D object and a reference 3D object may include calculating a first vector distance between a geometric asset feature of the candidate 3D object and a geometric asset feature of the reference 3D object, calculating a second vector distance between a semantic feature vector of the candidate 3D object and a semantic feature vector of the reference 3D object, and generating a fused vector distance by combining the first vector distance and the second vector distance. Determining the geometric asset feature of the candidate 3D object may include determining one or more Histogram of Oriented Gradient (HOG) vectors for each image of a plurality of images of the candidate 3D object, and determining the geometric asset feature of the candidate 3D object based on the one or more HOG vectors for each image of the plurality of images of the candidate 3D object. Implementations of the described techniques may include hardware, methods or processes, or computer software on a computer-accessible medium. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram of an exemplary environment in which classification of three-dimensional (3D) objects is performed, according to some implementations. [Figure 2A] FIG. 1 illustrates an example system architecture for classifying three-dimensional (3D) objects, according to some implementations. [Figure 2B] FIG. 1 illustrates an example system architecture for performing similarity analysis of three-dimensional (3D) objects, according to some implementations. [Figure 3A] 1A-1C illustrate exemplary camera positions utilized to capture images of a 3D object, according to some implementations. [Figure 3B] 1 is a diagram illustrating images of a 3D object representing views from different camera positions, according to some implementations. [Figure 4] 1 illustrates a diagram depicting a Histogram of Gradients (HOG) vector of an image, according to some implementations. [Figure 5] 1 is a flow diagram illustrating an example method for classifying candidate 3D objects, according to some implementations. [Figure 6A] FIG. 1 is a schematic diagram illustrating an asset feature generator according to some implementations. [Figure 6B] 1 is a schematic diagram depicting a comparison of a candidate 3D object with a real object according to some implementations. [Figure 7] FIG. 1 is a schematic diagram depicting the generation of semantic feature vectors according to some implementations. [Figure 8] 1 is a flow diagram illustrating an example method for classifying candidate 3D objects, according to some implementations. [Figure 9] 1 is a flow diagram illustrating an example method for performing a similarity analysis, according to some implementations. [Figure 10] 1 is a flow diagram illustrating an example method for determining similarity, according to some implementations. [Figure 11A]1 is a flow diagram illustrating an example method for determining a geometric uniqueness score of a 3D object, according to some implementations. [Figure 11B] FIG. 1 depicts an example 3D object with a relatively high geometric uniqueness score. [Figure 11C] FIG. 1 depicts an example 3D object with a relatively low geometric uniqueness score. [Figure 12] FIG. 1 is a block diagram illustrating an exemplary computing device according to some implementations. DETAILED DESCRIPTION OF THE INVENTION

[0014] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like numerals generally identify like elements unless context dictates otherwise. The illustrative embodiments set forth in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. The aspects of the present disclosure, as generally described herein and illustrated in the figures, can be arranged, substituted, combined, divided, and designed in a wide variety of different configurations, all of which are contemplated herein.

[0015] References herein to "some embodiments," "embodiments," "exemplary embodiments," and the like indicate that the described embodiments may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, such feature, structure, or characteristic may also be implemented in connection with other embodiments, whether or not explicitly described.

[0016] Online virtual experience platforms (also referred to as "user-generated content platforms" or "user-generated content systems") provide various ways for users to interact with one another. For example, users of an online virtual experience platform may work together toward a common goal, share various virtual experience items, send electronic messages to one another, etc. Users of an online virtual experience platform may participate in a virtual experience, e.g., a game or other experience, as virtual characters, playing particular roles. For example, virtual characters may be part of a team or multiplayer environment where each character is assigned a particular role and has associated parameters corresponding to the role, e.g., clothing, armor, weapons, skills, etc. In another example, a virtual character may be joined by computer-generated characters, for example, when a single player is part of a game.

[0017] A virtual experience platform may allow users (developers) of the platform to create objects, new games, and / or characters. For example, users of an online gaming platform may be enabled to create, design, and / or customize new characters (avatars), new animation packages, new three-dimensional objects, etc., and make them available to other users.

[0018] Objects, e.g., virtual objects, may be traded, exchanged, or bought and sold in online marketplaces for virtual and / or real currency. Virtual objects may be offered in any quantity within a virtual experience or environment, such that there may be a single instance of a particular object (a "unique object"), very few instances (a "rare object"), a limited number of instances (a "limited quantity"), or an unlimited number of instances (a "common object") within the virtual experience or environment. Allowing object creators to set limits on the number of instances of an object may allow creators to charge different prices (e.g., in virtual or real currency) for their creations, enabling a virtual economy to emerge in which different objects are priced differently.

[0019] However, because a virtual object may be a representation generated from information, such as an object mesh that defines the object's structure (e.g., shape) and motion (e.g., rotation, translation, etc.), a texture that defines the object's surface (e.g., the object's color, the object's surface properties, such as how light reflects off the object, etc.), and / or other attributes, it may be possible for the object to be copied by other creators. Such copied objects may be referred to as inauthentic or counterfeit objects. Counterfeit objects may be copied and redistributed without resellers or purchasers being aware of the virtual object's counterfeit nature. The presence of such counterfeit objects may be harmful to the virtual experience or environment and may affect the economy within the virtual experience, for example, because users may mistake the counterfeit objects for authentic objects, the counterfeit objects may have different properties than authentic objects, etc. Therefore, users and platform providers of virtual experience platforms may benefit from techniques that automatically detect counterfeit or inauthentic objects.

[0020] The proliferation or presence of counterfeit objects in the marketplace can be difficult to detect; for example, the quantity and nature of virtual objects can make human intervention (manual detection) in detecting counterfeit virtual objects difficult, as human detection may not be scalable to large numbers, e.g., millions, of objects.

[0021] The objective of the virtual experience platform owner or administrator is to mitigate (e.g., remove, block, etc.) counterfeit objects and provide incentives to original content creators. The technical problem for the virtual experience platform operator and / or administrator is the automatic, accurate, scalable, cost-effective, and reliable classification of 3D objects and the detection of inauthentic (counterfeit) objects across the platform.

[0022] Detecting inauthentic (counterfeit) objects may make it difficult and / or expensive for creators of inauthentic objects to create and propagate inauthentic virtual objects. A virtual experience platform that prevents the uploading and / or viewing of inauthentic objects can effectively deter creators of inauthentic objects and incentivize creators of authentic objects.

[0023] To evade detection, creators sometimes manipulate original objects to create manipulated objects that may be classified as originals by some counterfeit detection techniques, even though the difference between the manipulated object and the true (authentic) object may not be perceptible.

[0024] Various implementations described herein address the above-mentioned shortcomings by providing a technique for automatically detecting counterfeit objects that resemble real objects known to a gaming platform. The geometric and / or semantic similarity of counterfeit virtual objects with real (original) virtual objects is utilized for 3D object classification and counterfeit 3D object detection.

[0025] A technical problem for operators of virtual experience platforms is the timely classification of digital assets and the accurate detection of inauthentic digital assets. Another technical problem for operators of virtual experience platforms is the timely identification of 3D objects available on the virtual experience platform that are similar to a candidate 3D object. Described herein are implementations for automatically classifying 3D objects, identifying similar 3D objects, and / or detecting inauthentic 3D objects on a virtual experience platform, for example, a gaming platform.

[0026] 3D object classification and false 3D object detection pose several technical problems and challenges, including accurately determining the size, orientation, features, texture, etc. of a candidate 3D object in comparison with a true 3D object, which may also be associated with a respective size, orientation, features, texture, etc.

[0027] Various implementations described herein address the aforementioned problems and provide robust techniques for classifying candidate 3D objects and detecting inauthentic 3D objects that may be imitations of authentic 3D objects. Techniques are described that take into account the entire geometry of a 3D asset and do not rely solely on thumbnail views. Vector representations of the 3D object's geometry (asset feature vectors) are defined and generated in a way that 3D objects with similar geometric structures result in asset feature vectors that are close to each other in the vector space, and tertiary objects with dissimilar geometric structures result in asset feature vectors that are far from each other in the vector space.

[0028] In some implementations, in addition to performing analysis based on geometry, the analysis may also take into account semantic attributes of 3D objects. A combined analysis can leverage the strengths of each individual technique; for example, using geometric models may perform better by capturing the physical structure and shape of 3D assets, while using semantic models may perform better by characterizing the visual content of 3D assets (objects) based on their other attributes.

[0029] The combined approach can offer technical advantages over the use of each of the models individually when applied to perform tasks related to the visual similarity of 3D assets. For example, the use of geometric models may pose challenges in capturing abstract visual features, such as color, texture, and overall visual appearance, that are important for accurately comparing assets. Semantic models may not fully capture the complex details of shape and structure, leading to potential errors in similarity assessments. Semantic models may also contain a domain gap between the semantic understanding obtained from a training set and its application to real-world evaluation of 3D objects. This may be particularly pronounced for virtual experience platforms, where some assets may be user-generated, innovative, and outside the usual semantic domain and assets commonly used to train machine learning models. Thus, the combination of semantic and geometric analysis may offer better performance, e.g., speed, accuracy, etc., when compared to similarity analysis approaches that focus on either geometric features or semantic information.

[0030] The techniques of this disclosure may be utilized to enhance the accuracy and robustness of visual similarity analysis of 3D assets by addressing the limitations of individual approaches (e.g., either geometric or semantic) and combining geometric and semantic models using a distance fusion approach, which leverages the complementary strengths of both models to provide a comprehensive representation of similarity.

[0031] Distance fusion techniques involve extracting geometric features from a 3D asset by rendering the 3D asset from multiple viewpoints and using the multiple renderings, e.g., by concatenating their respective feature representations. Simultaneously, semantic features are obtained using pre-trained machine learning (ML) or neural network models, e.g., contrastive language-image pretraining, by extracting features from images, e.g., thumbnails of the 3D asset. These semantic features capture higher-level concepts and enable reasoning about the visual content of the asset in a more abstract sense.

[0032] The fusion process may involve separately computing pairwise distances between all assets based on geometric and semantic features. To combine these distances, a distance scalar function is applied to normalize the distances between 0 and 1, such as min-max scaling. The normalized distances are combined to create a new distance matrix that serves as a hybrid representation of asset similarity.

[0033] By combining geometric and semantic models using distance fusion, both the physical structure and semantic content of a 3D asset can be utilized in the analysis, resulting in a comprehensive and accurate representation of visual similarity. Various implementations described herein can overcome the limitations of individual models and provide nuanced and precise comparisons, facilitating tasks such as shape retrieval, 3D model retrieval, and content-based recommendation systems.

[0034] A three-dimensional (3D) model of the 3D object, e.g., a 3D mesh (including two or more vertices or connection points and rigid and / or flexible connections between the vertices), is obtained, and images of the candidate 3D object are taken at one or more camera positions. The multiple different camera positions provide images associated with different respective orientations of the candidate 3D object, allowing for better detection of spurious 3D objects.

[0035] Based on the captured images, a feature vector for each image is generated based on a histogram of oriented gradients (HOG) vector, which is an embedding that represents each image. In some implementations, pyramidal HOG (P-HOG) vectors are used, which are based on HOG vectors calculated from images at different resolutions.

[0036] In some implementations, an asset feature vector may be generated by concatenating pyramidal HOG vectors obtained from each image. This provides asset features that provide scale invariance and progressive geometric space for the feature vector. The progressive geometric space of the asset feature vector provides similarity detection between candidate 3D objects and genuine 3D objects. For example, apparently similar and apparently different 3D objects can be automatically classified, and candidate 3D objects with feature vectors within a distance threshold range from the feature vector of the genuine 3D object can be flagged for classification with human input. The threshold can be selected to ensure that false positives (objects flagged as potential forgeries but that are not forgeries) and false negatives (forgeries that remain undetected by comparison) meet performance requirements (such as scalability and reliability).

[0037] Based on the asset feature vector, a computationally efficient reduced-dimensional asset feature vector may be generated by performing a principal component analysis (PCA) or principal coordinate analysis (PCoA) operation on the asset feature vector.

[0038] Furthermore, the comparison between the candidate 3D object and the real 3D object may be performed in a rotation-invariant manner by generating a rolled version of the asset feature vector or by transforming the asset feature vector using spherical harmonics. A rolled asset feature vector is a transformed version of the original asset feature vector that corresponds to a particular orientation of the candidate 3D object.

[0039] In some implementations, textures associated with candidate 3D objects may be replaced with white cloth (or other neutral texture) to enable standardization of asset feature vectors across candidate objects.

[0040] The semantic feature vectors of the 3D object may be determined by a trained machine learning model previously trained to generate feature vectors and / or labels from images. For example, a contrastive learning technique may be utilized in which the model learns an embedding space in which similar images stay close to each other in their vector representations (e.g., have a short relative vector distance), while dissimilar images are far apart (e.g., have a long relative vector distance). The method may further include training the model with images in a contrastive manner to enable the model to learn attributes (e.g., labels) that are common between data classes and attributes that separate the data classes.

[0041] A contrastive machine learning model is utilized to combine attributes of an image and its associated textual description, thereby enabling the contrastive machine learning model to learn the meaning associated with the visual representation. The pre-training process involves training the model to predict matching image-text pairs while contrasting matching image-text pairs with mismatched pairs. The objective function of this contrastive training enables the model to capture semantic similarities and differences between different images and their corresponding text.

[0042] Images of 3D objects, e.g., thumbnails of assets, can be provided to a trained model to obtain semantic features that capture the asset's high-level concepts and visual semantics. These features are represented as high-dimensional vectors, with each dimension encoding specific semantic information learned by the model during training. The semantic features extracted by the contrasting machine learning model provide a holistic representation of the asset's visual content.

[0043] Various implementations described herein can perform similarity analysis and classify 3D objects. For example, when a candidate 3D object is newly uploaded to the platform by a user, it may be determined whether the candidate 3D object is substantially identical to an authentic object already available on the platform. In some scenarios, duplicate 3D objects may not be allowed to be uploaded and / or stored on the platform.

[0044] As another example, a similarity analysis may be performed to determine a set of 3D objects to be suggested to a user based on 3D objects purchased by the user. The set of 3D objects may be suggested based on a determination of which 3D objects on the platform are visually similar to the 3D objects already purchased.

[0045] In some implementations, the embedding space generated by the geometric features exhibits a progressive difference in geometry. The magnitude of the difference between the geometric asset feature vectors is a measure of the difference between two 3D objects with respect to their geometry. This difference measure can be further utilized to quantify the geometric uniqueness of a 3D object based on the geometry of the 3D object's mesh.

[0046] Some implementations leverage the MultiViewHoG algorithm to generate feature vectors for 3D objects and calculate a Local Density Score to measure the sparsity of the space around the object relative to other objects in the dataset.

[0047] In some implementations, similarity analysis may be utilized to improve search performance for 3D objects, provide recommendations to users, detect object geometric spaces with high potential demand, and suggest appropriate suggestions to creators.

[0048] FIG. 1 is a diagram of an exemplary system architecture for classification of three-dimensional (3D) objects, according to some implementations. FIG. 1 and other figures use similar reference numbers to identify similar elements. A letter after a reference number, such as "110," indicates that the text refers specifically to the element with the particular reference number. A reference number in text without a following letter, such as "110," refers to any or all of the elements in the figure with that reference number (e.g., "110" in the text refers to reference numbers "110a," "110b," and / or "110n" in the figures).

[0049] System architecture 100 (also referred to herein as the “system”) includes online virtual experience server 102, data store 120, user devices 110a, 110b, and 110n (collectively referred to herein as “user devices 110”), and developer devices 130a and 130n (collectively referred to herein as “developer devices 130”), where virtual experience server 102, content management server 140, data store 120, user devices 110, and developer devices 130 are coupled via network 122. In some implementations, user devices 110 and developer devices 130 may refer to the same device or the same type of device.

[0050] The online virtual experience server 102 may include a virtual experience engine 104, one or more virtual experiences 106, and a graphics engine 108. The user device 110 may include a virtual experience application 112 and an input / output (I / O) interface 114 (e.g., input / output devices). The input / output devices may include one or more of a microphone, speakers, headphones, a display device, a mouse, a keyboard, a game controller, a touchscreen, a virtual reality console, etc. The input / output devices may also include accessory devices connected to the user device by cable (wired) or wirelessly.

[0051] The content management server 140 may include a graphics engine 144 and a classification controller 146. In some implementations, the content management server may include multiple servers, which may be arranged in a hierarchy based, for example, on respective prioritization values ​​assigned to the content sources.

[0052] A graphics engine 144 may be utilized for rendering one or more objects, e.g., 3D objects associated with the virtual environment. A classification controller 146 may be utilized for classifying assets, such as 3D objects, detecting non-real digital assets, and the like. A data store 148 may be utilized for storing search indexes, model information, and the like.

[0053] The developer device 130 may include a virtual experience application 132 and an input / output (I / O) interface 134 (e.g., input / output devices). The input / output devices may include one or more of a microphone, speakers, headphones, a display device, a mouse, a keyboard, a game controller, a touchscreen, a virtual reality console, etc.

[0054] System architecture 100 is provided for illustrative purposes. In different implementations, system architecture 100 may include the same, fewer, more, or different elements arranged in the same or different ways as shown in FIG.

[0055] In some implementations, the network 122 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.11 network, a Wi-Fi network, or a wireless LAN (WLAN)), a cellular network (e.g., a 5G network, a Long Term Evolution (LTE) network, etc.), a router, a hub, a switch, a server computer, or a combination thereof.

[0056] In some implementations, data store 120 may be non-transitory computer-readable memory (e.g., random access memory), a cache, a drive (e.g., a hard drive), a flash drive, a database system, a cloud storage system, or another type of component or device capable of storing data. Data store 120 may also include multiple storage components (e.g., multiple drives or multiple databases), possibly spread across multiple computing devices (e.g., multiple server computers).

[0057] In some implementations, the online virtual experience server 102 may include a server having one or more computing devices (e.g., a cloud computing system, a rack-mounted server, a server computer, a cluster of physical servers, etc.) In some implementations, the online virtual experience server 102 may be a standalone system, may include multiple servers, or may be part of another system or server.

[0058] In some implementations, the online virtual experience server 102 may include one or more computing devices (such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a distributed computing system, etc.), data stores (e.g., hard disks, memory, databases), networks, software components, and / or hardware components that may be used to execute operations on the online virtual experience server 102 and provide users with access to the online virtual experience server 102. The online virtual experience server 102 may also include websites (e.g., web pages) or application backend software that may be used to provide users with access to content provided by the online virtual experience server 102. For example, a user may access the online virtual experience server 102 using a virtual experience application 112 on a user device 110.

[0059] In some implementations, the online virtual experience server 102 may be some type of social network that provides connections between users, or some type of user-generated content system that allows users (e.g., end users or consumers) to communicate with other users on the online virtual experience server 102, where the communication may include voice chat (e.g., synchronous and / or asynchronous voice communication), video chat (e.g., synchronous and / or asynchronous video communication), or text chat (e.g., synchronous and / or asynchronous text-based communication). In some implementations of the present disclosure, a "user" may be represented as a single individual. However, other implementations of the present disclosure encompass a "user" (e.g., a creating user) being a collection of users or an entity controlled by an automated source. For example, a collection of individual users federated as a community or group within a user-generated content system may be considered a "user."

[0060] In some implementations, the online virtual experience server 102 may be an online game server. For example, the virtual experience server may provide single-player or multiplayer games to a community of users who may access or interact with the games using user devices 110 over the network 122. In some implementations, the games (also referred to herein as “video games,” “online games,” or “virtual games”) may be, for example, two-dimensional (2D) games, three-dimensional (3D) games (e.g., 3D user-generated games), virtual reality (VR) games, or augmented reality (AR) games. In some implementations, users may participate in gameplay with other users. In some implementations, games may be played in real time with other users of the game.

[0061] In some implementations, gameplay may refer to the interaction of one or more players using a user device (e.g., 110) in a game (e.g., a game that is part of the virtual experience 106) or the presentation of the interaction on a display or other output device (e.g., 114) of the user device 110.

[0062] In some implementations, the virtual experience 106 may include electronic files that can be executed or loaded using software, firmware, or hardware configured to present game content (e.g., digital media items) to an entity. In some implementations, a virtual experience application 112 may be executed, and the virtual experience 106 may run in association with the virtual experience engine 104. In some implementations, the virtual experiences (e.g., games) 106 may have a common set of rules or a common goal, and the environments of the virtual experiences 106 share a common set of rules or a common goal. In some implementations, different games may have different rules or goals from one another.

[0063] In some implementations, a virtual experience may have one or more environments (also referred to herein as "game environments" or "virtual environments"), and multiple environments may be linked. An example of an environment may be a three-dimensional (3D) environment. One or more environments of a virtual experience application 106 may be collectively referred to herein as a "world" or "game world" or "virtual world" or "universe." An example of a world may be the 3D world of a game 106. For example, a user may build a virtual environment that is linked to another virtual environment created by another user. A character in a virtual game may cross a virtual boundary to enter an adjacent virtual environment.

[0064] It may be noted that a 3D environment or world uses graphics that use three-dimensional representations of geometric data representing the game content (or at least present the game content to appear as 3D content, regardless of whether 3D representations of geometric data are used), whereas a 2D environment or world uses graphics that use two-dimensional representations of geometric data representing the game content.

[0065] In some implementations, the online virtual experience server 102 may host one or more virtual experiences 106 and may allow users to interact with the virtual experiences 106 using a virtual experience application 112 on a user device 110. Users of the online virtual experience server 102 may play, create, interact with, or build a virtual experience 106, communicate with other users, and / or create and construct objects (e.g., also referred to herein as “items” or “game objects” or “virtual game items”) for the virtual experience 106. For example, in generating a user-generated virtual item, a user may create a character, decorations for a character, one or more virtual environments for an interactive game, or build a structure to be used in a game. In some implementations, users may buy, sell, or trade virtual game objects, such as in-platform currency (e.g., virtual currency), with other users of the online virtual experience platform 102. In some implementations, the online virtual experience server 102 may send game content to the virtual experience application (e.g., 112). In some implementations, game content (also referred to herein as "content") may refer to any data or software instructions (e.g., game objects, games, user information, videos, images, commands, media items, etc.) related to the online virtual experience server 102 or the virtual experience application. In some implementations, game objects (e.g., also referred to herein as "items" or "objects" or "virtual objects" or "virtual game items") may refer to objects used, created, shared, or otherwise depicted in the virtual experience application 106 of the online virtual experience server 102 or the virtual experience application 112 of the user device 110.For example, game objects may include parts, models, characters, accessories, tools, weapons, clothing, buildings, vehicles, currency, flora, fauna, components of the above (e.g., windows of a building), and the like.

[0066] It may be noted that the online virtual experience server 102 hosting the virtual experience 106 is provided for purposes of example and not limitation. In some implementations, the online virtual experience server 102 may host one or more media items, which may include communication messages from one user to one or more other users. Media items may include, but are not limited to, digital videos, digital movies, digital photos, digital music, audio content, melodies, website content, social media updates, e-books, e-magazines, digital newspapers, digital audiobooks, e-journals, weblogs, real simple syndication (RSS) feeds, e-comics, software applications, and the like. In some implementations, media items may be electronic files that can be executed or loaded using software, firmware, or hardware configured to present the digital media items to entities.

[0067] In some implementations, a virtual application 106 may be associated with a particular user or a particular group of users (e.g., a private game) or may be made generally available to users who have access to the online virtual experience server 102 (e.g., a public game). In some implementations in which the online virtual experience server 102 associates one or more virtual experiences 106 with a particular user or group of users, the online virtual experience server 102 may associate a particular user with a virtual experience 106 using user account information (e.g., a user account identifier such as a username and password).

[0068] In some implementations, the online virtual experience server 102 or the user device 110 may include a virtual experience engine 104 or a virtual experience application 112. In some implementations, the virtual experience engine 104 may be used for developing or executing the virtual experience 106. For example, the virtual experience engine 104 may include, among other features, a rendering engine (“renderer”) for 2D, 3D, VR, or AR graphics, a physics engine, a collision detection engine (and collision response), a sound engine, scripting capabilities, an animation engine, an artificial intelligence engine, networking capabilities, streaming capabilities, memory management capabilities, threading capabilities, scene graph capabilities, or video support for cinematics. Components of the virtual experience engine 104 may generate commands (e.g., rendering commands, collision commands, physics commands, etc.) that help compute and render the game. In some implementations, the virtual experience application 112 of the user device 110 may work independently, in cooperation with the virtual experience engine 104 of the online virtual experience server 102, or a combination of both.

[0069] In some implementations, both the online virtual experience server 102 and the user device 110 may run a virtual experience engine (104 and 112, respectively). The online virtual experience server 102, using its virtual experience engine 104, may perform some or all of the virtual experience engine's functions (e.g., generating physics commands, rendering commands, etc.) or may offload some or all of the virtual experience engine's functions to the virtual experience engine 104 of the user device 110. In some implementations, each virtual application 106 may have a different ratio between the virtual experience engine's functions executed on the online virtual experience server 102 and the virtual experience engine's functions executed on the user device 110. For example, the virtual experience engine 104 of the online virtual experience server 102 may be used to generate physics commands when there is a collision between at least two virtual application objects, while further virtual experience engine functions (e.g., generating rendering commands) may be offloaded to the user device 110. In some implementations, the ratio of virtual experience engine functions executed on the online virtual experience server 102 to virtual experience engine functions executed on the user device 110 may be changed (e.g., dynamically) based on gameplay conditions. For example, if the number of users participating in gameplay of a particular virtual application 106 exceeds a threshold number, the online virtual experience server 102 may execute one or more virtual experience engine functions previously executed by the user device 110.

[0070] For example, a user may be playing a virtual application 106 on a user device 110 and may send control instructions (e.g., user input such as right, left, up, down, user selection, or character position and velocity information) to the online virtual experience server 102. After receiving the control instructions from the user device 110, the online virtual experience server 102 may send gameplay instructions (e.g., position and velocity information for a character participating in group gameplay, or commands such as rendering commands, collision commands, etc.) to the user device 110 based on the control instructions. For example, the online virtual experience server 102 may perform one or more logical operations (e.g., using the virtual experience engine 104) according to the control instructions to generate gameplay instructions for the user device 110. In other cases, the online virtual experience server 102 may pass one or more of the control instructions from one user device 110 to other user devices participating in the virtual application 106 (e.g., from user device 110a to user device 110b). The user device 110 may use the gameplay instructions to render the gameplay for presentation on the display of the user device 110.

[0071] In some implementations, control instructions may refer to instructions indicating in-game actions of a user's character. For example, control instructions may include user input, user selection, gyroscope position and orientation data, force sensor data, etc. for controlling in-game actions such as right, left, up, down, etc. The control instructions may include character position and velocity information. In some implementations, control instructions are transmitted directly to the online virtual experience server 102. In other implementations, control instructions may be transmitted from the user device 110 to another user device (e.g., from user device 110b to user device 110n), which generates gameplay instructions using a local virtual experience engine 104. The control instructions may include instructions to play voice communication messages or other sounds from another user on an audio device (e.g., speakers, headphones, etc.), for example, voice communication or other sounds generated using audio spatialization techniques as described herein.

[0072] In some implementations, gameplay instructions may refer to instructions that enable the user device 110 to render gameplay of a game, such as a multiplayer game. The gameplay instructions may include one or more of user input (e.g., control instructions), character position and velocity information, or commands (e.g., physics commands, rendering commands, collision commands, etc.).

[0073] In some implementations, the online virtual experience server 102 may store characters created by the user in the data store 120. In some implementations, the online virtual experience server 102 maintains a character catalog and a game catalog that may be presented to the user. In some implementations, the game catalog includes images of virtual experiences stored on the online virtual experience server 102. In addition, the user may select a character (e.g., a character created by the user or another user) from the character catalog to participate in a selected game. The character catalog includes images of characters stored on the online virtual experience server 102. In some implementations, one or more of the characters in the character catalog may have been created or customized by the user. In some implementations, the selected character may have a character setting that defines one or more of the character's components.

[0074] In some implementations, a user's character may include a configuration of components, and the configuration and appearance of the components, as well as the character's appearance more generally, may be defined by a character setting. In some implementations, a user's character's character setting may be selected, at least in part, by the user. In other implementations, a user may select a character with a default character setting or another user-selected character setting. For example, a user may select a default character from a character catalog that has a predefined character setting, and the user may further customize the default character by changing parts of the character setting (e.g., adding a shirt with a customized logo). A character setting may be associated with a particular character by the online virtual experience server 102.

[0075] In some implementations, the virtual experience platform may support three-dimensional (3D) objects represented by a 3D model, including a surface representation (also known as a skin or mesh) and a hierarchical collection of interconnected bones (also known as a skeleton or rig) used to render the character or object. The rig may be utilized to animate the object and simulate the object's motion. The 3D model may be represented as a data structure, and one or more parameters of the data structure may be modified to change various characteristics of the character, such as dimensions (height, width, girth, etc.), shape, movement style, number / type of parts, proportions, etc.

[0076] In some implementations, the 3D model may include a 3D mesh. The 3D mesh may define a three-dimensional structure of an unauthenticated virtual 3D object. In some implementations, the 3D mesh may also define one or more surfaces of the 3D object. In some implementations, the 3D object may be a virtual avatar, e.g., a virtual character such as a humanoid character, an animal character, a robot character, or the like.

[0077] In some implementations, a mesh may be received (imported) in an FBX file format. The mesh file includes data that provides dimensional data for the polygons that comprise a virtual 3D object and UV map data that describes how portions of a texture should be applied to the various polygons that comprise the 3D object. In some implementations, the 3D object may correspond to an accessory, such as a hat, weapon, or clothing, worn by or otherwise depicted in association with a virtual avatar.

[0078] In some implementations, the platform may allow users to submit (upload) candidate 3D objects for use on the platform. According to some implementations, a virtual experience development environment (developer tools) may be provided by the platform. The virtual experience development environment may provide a user interface that allows developer users to design and / or create virtual experiences, e.g., games. One example of a virtual experience development environment is Roblox™ Studio from Roblox™ Corporation. Other development tools and environments provided by other companies may be used in various embodiments. The virtual experience development environment may be a client-based tool (e.g., downloaded to, installed on, and operated from a client device), a server-based tool (e.g., installed on, and executed on a server remote from the client device, and accessed and operated by the client device), or a combination of both client-based and service-based elements.

[0079] The virtual experience development environment may be operated by a virtual experience developer, e.g., a game developer, or any other person seeking to create a virtual experience that may be published by an online virtual experience platform and utilized by others. A user interface of the virtual experience development environment may be rendered on a display screen of a client device, such as, for example, developer device 130 described with reference to FIG. 1 , to enable the author / developer to interact with the development environment using actions such as typing, highlighting, selecting, dragging and dropping, clicking, etc., via a mouse, keyboard, or other input device configured to communicate with the user interface. The user interface may include a menu bar, a toolbar, a workspace pane, and multiple secondary panes. Depending on the particular implementation, the user interface may include alternative or additional elements, arrangements, operational features, etc. of the virtual experience development environment compared to those shown and described herein.

[0080] A developer user (creator) may utilize the virtual experience development environment to create a virtual experience. As part of the development process, the developer / creator may upload various types of digital content to enhance the virtual experience, such as object files (meshes), image files, audio files, short videos, etc.

[0081] In implementations where the candidate (unauthenticated) 3D object is an accessory, data indicating the object's use in the virtual experience may also be received. For example, a "shoe" object may include an annotation indicating that the object may be depicted as being worn on the feet of a virtual humanoid character, while a "shirt" object may include an annotation indicating that it may be depicted as being worn on the torso of a virtual humanoid character.

[0082] In some implementations, the 3D model may further include texture information associated with the 3D object. For example, the texture information may indicate the color and / or pattern of the exterior surface of the 3D object. The texture information may enable varying the transparency, reflectivity, diffuseness, material properties, and refractory behavior of textures and meshes associated with the 3D object. Examples of textures include plastic, cloth, grass, a sheet of light blue glass, ice, water, concrete, brick, carpet, wood, etc.

[0083] In some implementations, the user devices 110 may each include a computing device such as a personal computer (PC), a mobile device (e.g., a laptop, a mobile phone, a smartphone, a tablet computer, or a netbook computer), a network-connected television, a game console, etc. In some implementations, the user devices 110 may also be referred to as "client devices." In some implementations, one or more user devices 110 may connect to the online virtual experience server 102 at any time. It may be noted that the number of user devices 110 is given by way of example. In some implementations, any number of user devices 110 may be used.

[0084] In some implementations, each user device 110 may include a respective instance of a virtual experience application 112. In one implementation, the virtual experience application 112 may enable a user to use and interact with the online virtual experience server 102, such as controlling a virtual character in a virtual game hosted by the online virtual experience server 102, or viewing or uploading content such as virtual experiences 106, images, video items, web pages, documents, etc. In one example, the virtual experience application may be a web application (e.g., an application that operates in conjunction with a web browser) that can access, retrieve, present, or navigate content provided by a web server (e.g., virtual characters in a virtual environment, etc.). In another example, the virtual experience application may be a native application (e.g., a mobile application, app, or game program) that is installed and executed locally on the user device 110 and enables a user to interact with the online virtual experience server 102. The virtual experience application may render, display, or present content (e.g., a web page, a media viewer) to the user. In implementations, the virtual experience application may also include an embedded media player (e.g., a Flash player) that is embedded in a web page.

[0085] In some implementations, the virtual experience application may include an audio engine 116 that is installed on the user device and enables sound playback on the user device. In some implementations, the audio engine 116 may work in conjunction with an audio engine 144 that is installed on a sound server.

[0086] According to aspects of the present disclosure, the virtual experience application may be an online virtual experience server application through which a user builds, creates, edits, uploads content to, and interacts with the online virtual experience server 102 (e.g., participates in a virtual experience 106 hosted by the online virtual experience server 102). Thus, the virtual experience application may be provided to the user device 110 by the online virtual experience server 102. In another example, the virtual experience application may be an application that is downloaded from a server.

[0087] In some implementations, each developer device 130 may include a respective instance of a virtual experience application 132. In one implementation, the virtual experience application 132 may enable a developer user to use and interact with the online virtual experience server 102, such as controlling a virtual character in a virtual game hosted by the online virtual experience server 102, or viewing or uploading content for the game 106, images, video items, web pages, documents, etc. In one example, the virtual experience application may be a web application (e.g., an application that operates in conjunction with a web browser) that can access, retrieve, present, or navigate content provided by a web server (e.g., virtual characters in a virtual environment, etc.). In another example, the virtual experience application may be a native application (e.g., a mobile application, app, or virtual experience program) that is installed and executed locally on the user device 130 and enables the user to interact with the online virtual experience server 102. The virtual experience application may render, display, or present content (e.g., a web page, a media viewer) to the user. In implementations, the virtual experience application may also include an embedded media player (e.g., a Flash player) that is embedded in a web page.

[0088] According to aspects of the present disclosure, the virtual experience application 132 may be an online virtual experience server application through which a user builds, creates, edits, uploads content to the online virtual experience server 102, and interacts with the online virtual experience server 102 (e.g., to provide and / or play games 106 hosted by the online virtual experience server 102). Thus, the virtual experience application may be provided to the user device 130 by the online virtual experience server 102. In another example, the virtual experience application 132 may be an application downloaded from the server. The virtual experience application 132 may be configured to interact with the online virtual experience server 102 and to be able to access user credentials, user currency, etc. for one or more virtual applications 106 developed, hosted, or provided by a virtual experience application developer.

[0089] In some implementations, a user may log in to the online virtual experience server 102 through a virtual experience application. A user may access a user account by providing user account information (e.g., username and password), which is associated with one or more characters that can be used to participate in one or more games 106 of the online virtual experience server 102. In some implementations, appropriate credentials may enable a virtual experience application developer to access virtual experience application objects, such as in-platform currency (e.g., virtual currency), avatars, special abilities, accessories, etc., owned by or associated with other users.

[0090] In general, functionality described in one implementation as being performed by the online virtual experience server 102 may be performed by the user device 110 or server, as appropriate, in other implementations. In addition, functionality attributed to a particular component may be performed by different or multiple components working together. The online virtual experience server 102 may also be accessed as a service offered to other systems or devices through an appropriate application programming interface (API), and thus is not limited to use in a website.

[0091] In some implementations, the online virtual experience server 102 may include a graphics engine 108. In some implementations, the graphics engine 108 may be a system, application, or module that enables the online virtual experience server 102 to provide graphics and animation capabilities. In some implementations, the graphics engine 108 and / or the content management server 140 may perform one or more of the operations described below in connection with the flow diagram shown in FIG.

[0092] FIG. 2A illustrates an exemplary system architecture for classification of three-dimensional (3D) objects, according to some implementations.

[0093] 2A, object classification system 200 includes various modules for classifying candidate 3D objects and detecting non-real 3D objects. Object classification system 200 includes a module for feature extraction 202, a feature encoder 204, and a match detector 210. The object classification system further includes data stores (e.g., similar to data store 120 depicted in FIG. 1)—data store 230 storing real 3D objects and data store 220 storing candidate 3D objects. A search index 225 may also be part of object classification system 200.

[0094] The object repository 230 may store real 3D objects used across the platform. The storage may support scenarios in which a particular 3D object is used in a single virtual experience, as well as scenarios in which a particular asset (e.g., a 3D object) is used in multiple virtual experiences across the virtual experience platform.

[0095] The candidate object data store 220 may store candidate 3D objects received at the platform (e.g., from developers creating objects using on-platform or off-platform tools) for storage prior to their classification (and / or use on the platform). The candidate 3D objects may be received from users, e.g., developer users, for use on the platform. The 3D objects may be made available for free or for purchase on the platform by users, e.g., developer users and content creators of the platform.

[0096] The search index 225 is an index of asset feature vectors of real 3D objects. The search index 225 enables efficient searching of asset feature vectors of real 3D objects.

[0097] The feature extractor 202 may be utilized to generate images of 3D objects, for example, real 3D objects, candidate 3D objects, etc.

[0098] The feature encoder 204 may implement suitable techniques for generating asset features and / or embeddings of the 3D object based on images of the 3D object. In some implementations, the feature extractor 202 and feature encoder 204 may generate a vector representation of the 3D object that can be utilized for 3D object classification and detection of inauthentic candidate 3D objects. In some implementations, the feature extractor 202 and feature encoder 204 may include support for different types of embeddings, which may be based on the type of 3D object for which asset features are to be generated. In some implementations, the embeddings may be based on pyramidal histograms of oriented gradients (HOG) of the candidate 3D object.

[0099] In some implementations, the match detector 210 may compare the candidate 3D object with the real 3D object by utilizing an appropriate algorithm, for example, the algorithm described in connection with block 525 of Figure 5. The match detector 210 may include a collision detector 212 and a vector distance calculator 214 that may be utilized to perform the comparison of the candidate 3D object with the real 3D object.

[0100] In some implementations, the match detector 210 may implement a hashing technique to determine candidate 3D objects that are not genuine. For example, a hashing technique may be used to match a hash of a candidate 3D object with a reference hash of a genuine 3D object.

[0101] FIG. 2B illustrates an example system architecture for performing similarity analysis of three-dimensional (3D) objects, according to some implementations.

[0102] Similarity analysis system 250 includes a geometric feature extractor 252, a semantic feature extractor 254, an analysis module 260, a candidate object storage 280, a search index storage 282, and a reference object repository 284. Analysis module 260 includes a vector distance calculator 262, a normalization module 264, a match detector 266, a uniqueness evaluator 268, an object ranking module 270, and a pricing module 272.

[0103] The geometric feature extractor 252 generates a vector representation of the 3D object, e.g., a feature vector based on geometric properties of the 3D object based on an image of the 3D object. In some implementations, the geometric feature extractor may be configured to determine the geometric feature vector, which may be based on a pyramidal histogram of oriented gradients (HOG) of the candidate 3D object.

[0104] The semantic feature extractor 254 generates a vector representation of the 3D object, for example, a feature vector based on semantic attributes of the 3D object based on an image of the 3D object.

[0105] In some implementations, the semantic feature extractor may be configured to generate a semantic feature vector that may be based on a zero-shot classifier applied to a trained machine learning model that is trained using the image and text labels.

[0106] The analysis module 260 performs various operations on the generated feature vectors. A vector distance calculator 262 is utilized to calculate the respective distance between two feature vectors, e.g., Euclidean distance, cosine similarity, etc. A normalization module 264 may apply an appropriate normalization and / or transformation function to the determined distance of a first type to enable it to be combined with another distance of a second type, e.g., to enable a combination of geometric distance and semantic distance.

[0107] The match detector 266 applies an appropriate predetermined threshold to compare 3D objects based on the distance between their feature vectors. The comparison may be used to determine relationships between the 3D objects, such as similarity, overlap, etc.

[0108] The uniqueness evaluator 268 determines a measure of the uniqueness of the 3D object based on the distance between the feature vector of the 3D object and the feature vectors of the neighboring 3D objects.

[0109] The object ranking module 270 may be utilized to rank 3D objects based on their relative distance from each other and to search a list of 3D objects based on criteria, for example, to search for a specific number of similar objects, to search for identical 3D objects, to search for a specific number of unique 3D objects, etc.

[0110] The price predictor 272 determines a suggested price for a 3D object based on certain attributes of the 3D object, such as similarity to other 3D objects, uniqueness, and the like.

[0111] Reference object storage (repository) 284 may store real and / or previously classified 3D objects utilized across the platform. The storage may support scenarios in which a particular 3D object is utilized in a single virtual experience, as well as scenarios in which a particular asset (e.g., a 3D object) is utilized in multiple virtual experiences across the virtual experience platform.

[0112] Candidate object storage 280 includes candidate 3D objects that may be received on the platform (e.g., from developer users who may create objects using on-platform or off-platform tools) for storage prior to their classification (and / or use on the platform). Candidate 3D objects may be received from users, e.g., developer users, for use on the platform. 3D objects may be made available on the platform for free or for purchase by users, e.g., developer users and content creators of the platform.

[0113] The search index 282 is an index of asset feature vectors of 3D objects to enable efficient and timely retrieval of the geometric and semantic feature vectors of the 3D objects.

[0114] FIG. 3A is a diagram illustrating exemplary camera positions utilized to capture images of a 3D object, according to some implementations.

[0115] The geometric details of a 3D object, e.g., a candidate 3D object, are captured by generating custom views of the 3D object from different viewpoints. The custom views may be generated by rendering the 3D object based on a 3D model of the candidate 3D object within a standardized rendering environment. Standardization may include applying a standard texture (e.g., white plastic) to the object, uniform lighting (e.g., to ensure all objects are imaged in similar lighting conditions and allow comparisons), and using standard camera settings for the virtual camera.

[0116] Prior to image capture, the rendering environment (scene) may be configured with uniform lighting. The lighting is standardized across all image captures used to generate images for 3D object classification to ensure accurate results. For example, the same lighting settings used to generate images of genuine 3D objects may be used to generate images of candidate 3D objects.

[0117] In some implementations, the camera distance from the candidate 3D object is determined by determining the field of view of the camera and ensuring that the 3D object occupies at least a certain percentage of the captured image, which can ensure that the generated captured image is invariant to changes in the size of the candidate 3D object when compared to the real 3D object.

[0118] In some implementations, a modified Ritter's algorithm may be used for the scale calculation and for determining the camera distance from the object. According to Ritter's algorithm, the following calculations may be performed:

[0119] Once the candidate 3D object is rendered, the minimum and maximum coordinates (x, y) of the input points along the surface of the candidate 3D object are determined. This calculation can be performed efficiently (in linear time). The diameter of the point set is determined by calculating the Euclidean distance between the two points that are the greatest distance from each other. This can also be performed in linear time using efficient techniques, such as by utilizing a rotating calipers algorithm. The midpoint of the diameter is determined, which represents the center of an initial bounding circle. The radius of the bounding circle is determined, where the radius is half the previously determined diameter.

[0120] This process is repeated for all points in the rendered set, and for each point not within the bounding circle, its distance is calculated from the center of the bounding circle. If this calculated distance is greater than the radius of the currently used bounding circle, the bounding circle is expanded such that its radius is set equal to the calculated distance, and the center of the updated bounding circle is set to be the midpoint between that point and the center of the previously used bounding circle. This is repeated until all points have been processed.

[0121] In some implementations, additional steps may be performed to further enhance fraud detection of fraudulent duplicate 3D model submissions that may, for example, manipulate the scale of an object. In some implementations, the increase in radius due to the addition of a new vertex may be analyzed, and the newly added vertex may be excluded from the calculation of the radius of the bounding circle based on a determination that the increase in radius due to the addition of the new vertex (point) meets a predetermined threshold.

[0122] In some implementations, for each point (P) or vertex, the square of the distance between P and the center of the bounding circle is determined. If including P would cause a rapid increase in radius and the distance exceeds the square of the radius, the vertex is excluded from further consideration. The radius and center of the bounding circle are updated based on the remaining valid vertices and used to calculate the final radius of the bounding sphere.

[0123] Two or more camera viewpoints (camera locations) are selected within the rendering environment to provide multiple views of the candidate 3D object. The number of camera viewpoints and corresponding images of the candidate 3D object may be determined based on accuracy requirements, computational (time) budget, availability of computational resources, etc. For example, in some implementations, approximately 200 camera viewpoints may be utilized to capture images of the candidate 3D object such that the space around the candidate 3D object is covered by approximately 200 points distributed throughout the rendering environment.

[0124] The location of each camera viewpoint may be specified by its azimuth angle α and its elevation angle e. Depending on the number of images to be captured, camera points are selected that cover a range of azimuth and elevation values.

[0125] For example, in some implementations, the camera azimuth parameter may be selected to be between 0 and 360 degrees in 18 degree increments, and the camera elevation parameter may be selected to be between 0 and 180 degrees in 18 degree increments, which provides 200 different camera locations covering the candidate 3D object from various orientations. Camera azimuth parameter (α) = [0:360:18] Camera elevation parameter (e) = [0:180:18] Camera position = α X e

[0126] A virtual camera is placed at each of these positions defined by the orientation parameters, and the candidate 3D object is placed at the center of the rendering scene. The generated images (renders) allow capturing the object's geometry from all around it.

[0127] In some implementations, the texture specified for the candidate 3D object may be replaced with a uniform white plastic material (or other neutral / suitable texture for capturing images of the object from multiple camera positions). In some implementations, a graphics processing unit (GPU) processor may be utilized to achieve hardware-accelerated rendering.

[0128] FIG. 3B depicts an example image of a 3D object representing views from different camera positions, according to some implementations.

[0129] As depicted in Figure 3B, images of the candidate 3D object may be captured from different camera positions. This allows for better performance during classification and detection of non-real objects compared to using a single view, e.g., a thumbnail image of the candidate 3D object. In this illustrative example, the images are generated at a resolution of 128 x 128 pixels.

[0130] FIG. 4 depicts histograms of Histogram of Oriented Gradient (HOG) vectors determined at different resolutions of an image according to some implementations.

[0131] The comparison between the candidate 3D object and the real 3D object may be performed by comparing their respective feature vectors. In some implementations, the feature vectors may be constructed (extracted) based on Histogram of Oriented Gradients (HOG) vectors determined based on images of the candidate 3D object and the real 3D object.

[0132] As described with reference to Figures 3A and 3B, two or more images of a candidate 3D object are generated (captured) at different camera locations. For each of the generated images, the pyramid HOG technique may be applied to generate a vector representation (feature vector) of each image.

[0133] The use of histogram of oriented gradients (HOG) feature vectors (descriptors) allows the appearance and shape of local 3D objects in an image to be captured in the distribution of intensity gradients or edge directions. The captured image is divided into connected regions (cells), and a histogram of gradient directions is calculated for the pixels contained within each cell. In some implementations, the feature vector is the concatenation of the calculated histograms. In some implementations where high accuracy is required, the local histograms can be contrast normalized by calculating an intensity measure over a larger region of the captured image, which is then used to normalize the cells within the larger region. Normalization may result in better classification and detection invariance to changes in lighting and shadowing. HOG descriptors have several important advantages over other feature vector descriptors. For example, because they operate on local cells, HOG descriptors are invariant to geometric and photometric transformations, except for object orientation.

[0134] The process of computing a Histogram of Oriented Gradients (HOG) vector involves first dividing an image into cells of a fixed pixel size. Then, for each cell, the gradient direction and magnitude are calculated for each pixel. These gradients are then quantized into a set of orientation bins, and a histogram magnitude is constructed for each cell. These histograms are concatenated to form a feature vector that summarizes the geometric information of the image. A variant of the HOG descriptor is the pyramidal HOG descriptor, which can extract features at different resolutions and capture finer details of the image.

[0135] The determination of pyramidal HOG feature vectors for each captured image of a candidate 3D object is described herein.

[0136] image_render α,eis an image generated from a camera location with azimuth angle α and elevation angle, the corresponding HOG for a particular resolution n of the image is given by the function HOG <n,n> ( ). The feature representation of the rendered image is then F α,e = [HOG <2,2> (image_render α,e ), H.O.G. <4,4> (image_render α,e ), H.O.G. <16,16> (image_render α,e )] It may be the case.

[0137] The particular resolution n refers to the number of pixels contained in the window utilized during the HOG calculation.

[0138] After the pyramid HOG vectors are calculated for each rendered image, the pyramid HOG vectors for each candidate 3D object are concatenated to create an asset feature vector that represents the complete geometry of the candidate 3D object (asset).

[0139] For example, if a candidate 3D object is identified by an asset identifier asset-id, the asset feature vector for the candidate 3D object whose images are taken at camera positions corresponding to 20 azimuth locations and 10 elevation locations may be: AssetFeatureVector <asset-id>< / asset-id> = [f 0,0 , f 0,1 ...f 19,9 ] is expressed by

[0140] These extracted features, represented by asset feature vectors (AssetFeatureVector), enable the mapping of candidate 3D objects (assets) into a multidimensional space, where the distance between assets in the multidimensional space is inversely proportional to the geometric similarity between the assets. For example, a short vector distance between two 3D objects indicates high geometric similarity between the two 3D objects, and a long vector distance between two 3D objects indicates low geometric similarity between the two 3D objects.

[0141] In this illustrative example, FIG. 4 depicts a series of images at different resolutions. The images are depicted at a size of 224×224 pixels and show details captured by a Histogram of Oriented Gradients (HOG) algorithm at different resolutions. The first image in the set depicts a HoG with a small window size, which can capture details at higher resolutions; as the window size is increased (drawn from left to right in the set), it can be observed that the features become more “generalized” or “aggregated.” By applying the HOG algorithm, both finer details and higher-level details of the geometry of a 3D object can be captured.

[0142] FIG. 5 is a flow diagram illustrating an example method for classifying candidate 3D objects according to some implementations.

[0143] In some implementations, method 500 may be implemented to classify candidate 3D objects on, for example, the virtual experience server 102 described with reference to FIG. 1 . In some other implementations, method 500 may be implemented on, for example, one or more servers described with reference to FIG. 1 . In the described example, the implementing system includes one or more digital processors or processing circuits (“processors”) and one or more storage devices (e.g., data store 120 or other storage). In some implementations, different components of one or more servers and / or clients may perform different blocks or other portions of method 500. In some examples, a first device is described as performing blocks of method 500. Some implementations may have one or more blocks of method 500 performed by one or more other devices (e.g., other client devices or server devices) that can send results or data to the first device.

[0144] The method 500 may begin at block 505 .

[0145] At block 505, a candidate 3D object is received. The candidate 3D object may be received from a developer device, such as, for example, developer device 130 described with reference to FIG. 1. In some implementations, the candidate 3D object may be a candidate 3D object previously received at a computing device associated with the virtual experience platform and / or stored in a storage device associated with the virtual experience platform.

[0146] In some implementations, candidate 3D objects may be received as part of a classification process and / or workflow. In some implementations, a uniform record locator (URL), token, or other asset identifier may be utilized to provide a link to the candidate 3D object, and the processor may utilize the provided link, token, URL, etc. to retrieve the candidate 3D object. The candidate 3D object may be implemented as a 3D model and may include a surface representation (also known as a skin or mesh) and a hierarchical collection of interconnected bones (also known as a skeleton or rig) used to render the object. The rig may be utilized to animate a character and simulate motion and actions by the object. The 3D model may be represented as a data structure, and one or more parameters of the data structure may be modified to change various characteristics (attributes) of the object and / or character, such as dimensions (height, width, diameter, girth, etc.), body type and / or shape, movement style, number / type of parts, proportions of the object or body part (e.g., shoulder-to-hip ratio), head size, etc.

[0147] In some implementations, receiving the candidate 3D object may include obtaining a 3D mesh of the candidate 3D object.

[0148] Block 505 may be followed by block 510 .

[0149] At block 510, multiple images (rendered images or rendered views) of the candidate 3D object are generated, each of the multiple images being captured from a respective one of the two or more camera positions. In some implementations, generating the multiple images of the candidate 3D object may include generating multiple images at multiple different azimuth and / or elevation angle settings.

[0150] In some implementations, two or more images of the candidate 3D object are generated, each image of the two or more images of the candidate 3D object being captured from a respective one of the two or more camera positions.

[0151] In some implementations, the multiple images of the candidate 3D object may include about 200 images captured from 200 different camera positions, each camera position corresponding to a particular azimuth and elevation setting among a plurality of azimuth and elevation settings. For example, in some implementations, the camera positions may correspond to about 20 different azimuth locations and about 10 elevation locations to provide a comprehensive set of views of the candidate 3D object. In some implementations, a modified Ritter algorithm may be utilized to determine the size (bounding circle) of the candidate 3D object and adjust the field of view of the camera.

[0152] In some implementations, during generation of an image of a candidate 3D object, the camera view may be adjusted during image capture so that the candidate 3D object occupies a predetermined area of ​​the image, e.g., the predetermined area may be specified as, for example, between 70 and 75% of the area of ​​the image.

[0153] In some implementations, during generation of the image of the candidate 3D object, the camera view may be adjusted during image capture so that the candidate 3D object occupies at least a (minimum) predetermined area of ​​the image. For example, the predetermined area may be specified as a minimum so that the candidate 3D object occupies at least 70% of the area of ​​the image.

[0154] In some implementations, a single camera position may be utilized, and a single image of the candidate 3D object may be captured for further analysis. This may allow for rapid classification and verification of authenticity of the candidate 3D object, and may be utilized in some scenarios where speed of classification results may be prioritized over accuracy.

[0155] In some implementations, before generating the multiple images of the candidate 3D object, the texture of the candidate 3D object may be replaced with a white plastic material (or other neutral material). In some implementations, the material that replaces the previous texture of the candidate 3D object may be selected such that it has a predetermined reflectance that allows the asset feature vector to provide a representation of the interior of the candidate 3D object and provide normalization. Block 510 may be followed by block 515.

[0156] One or more Histogram of Oriented Gradients (HOG) vectors are determined for each image of the plurality of images of the candidate 3D object at block 515. In some implementations, determining the one or more HOG vectors may include determining one or more pyramid HOG vectors, each of the one or more pyramid HOG vectors generated by concatenating HOG vectors of the candidate 3D object generated at multiple resolutions of a respective image of the candidate 3D object.

[0157] In some implementations, a pyramid HOG vector is determined that includes a concatenation of HOG vectors calculated based on different resolutions of each image. A pyramid HOG may be a vector that includes HOGs of multiple resolutions concatenated together.

[0158] In some implementations, the resolution of an HOG vector may refer to a window size that specifies the number of pixels per window. For example, HOG vectors may be calculated at resolutions of 2, 4, and 16, whereby the windows utilized to calculate the HOG vectors may include 2, 4, and 16 pixels per window, respectively. A 2-pixel HOG represents an HOG vector calculated at a relatively high resolution of the image, and a 16-pixel HOG vector represents an HOG vector calculated at a lower resolution of the image (when compared to a 2-pixel HOG). Block 515 may be followed by block 520.

[0159] At block 520, asset features (e.g., asset feature vectors) of the candidate 3D object are determined based on one or more HOG vectors of each of the multiple images of the candidate 3D object. In some implementations, the asset feature vectors are determined by concatenating the HOG vectors obtained from each image to generate a representation of the candidate 3D object that includes embeddings from all camera views of the candidate 3D object. Block 520 may be followed by block 525.

[0160] In block 525, the asset features of the candidate 3D object are compared with the asset features of the genuine 3D object. In some implementations, it may not be feasible to perform the comparison of asset feature vectors on their full calculated dimensions and still meet performance considerations (e.g., real-time or near-real-time detection of counterfeit objects). In such implementations, a dimensionality reduction of the asset feature vectors may be performed.

[0161] For example, the complete geometric asset feature vector generated from the above-described Pyramid HOG-based feature extraction at three hierarchical resolutions (of Pyramid HOG) using about 200 images has a dimension of about 2M float values. For a database containing about 100,000 3D objects, 700 GB of memory may be required. In some scenarios, this may be a computational challenge.

[0162] In some implementations, techniques such as incremental principal component analysis (PCA), principal coordinate analysis (PCoA), etc. may be utilized to reduce the dimensionality of the asset feature vectors of the candidate 3D object and the genuine 3D object. For example, a reduced dimensionality asset feature vector of 1024×1 may be determined by applying an appropriate PCA transformer to the full geometric asset feature vector. AssetFeatureVector[1024D] <assetid>< / assetid> = AssetFeatureVector <assetid>< / assetid> X PCA converter

[0163] In some implementations, prior to comparing the candidate 3D object with the real 3D object, a principal component analysis and / or principal coordinate analysis (PCoA) operation is performed on the asset feature vector to reduce the dimensionality of the asset feature vector. Determining whether asset features of the candidate 3D object match real asset features of the at least one real 3D object may include performing a comparison between the reduced-dimensional asset feature vector of the candidate 3D object and the reduced-dimensional asset feature vector of the at least one real 3D object.

[0164] In some implementations, before performing the comparison of the asset features of the candidate 3D object with the asset features of the genuine 3D object, it may be determined (verified) that the user identifier associated with the candidate 3D object does not match the user identifier associated with the genuine object. This may be performed to ensure that candidate 3D models from users who have previously submitted the same 3D object are not flagged as inauthentic. In some implementations, determining (verifying) that the user identifier associated with the candidate 3D object does not match the user identifier associated with the genuine object may be performed after comparing the n asset features of the candidate 3D object with the asset features of the genuine 3D object and before classifying or flagging the candidate 3D object as inauthentic. Block 525 may be followed by block 530.

[0165] At block 530, it is determined whether the asset features of the candidate 3D object match the authentic asset features of the at least one authentic 3D object. In some implementations, determining whether the asset features of the candidate 3D object match the authentic asset features of the at least one authentic 3D object may include calculating a vector distance between the asset features of the candidate 3D object and the at least one authentic 3D object, and if the vector distance meets a predetermined threshold, it is determined that the asset features of the candidate 3D object match the authentic asset features of the at least one authentic 3D object.

[0166] For example, in some implementations, a Euclidean distance may be calculated between the asset feature vector of the candidate 3D object and one or more real 3D objects to determine the geometric similarity between the candidate 3D object and one or more real 3D objects. In some other implementations, other distance measures, such as Manhattan distance, Hamming distance, cosine distance, etc., may be utilized to determine the similarity of the asset feature vectors.

[0167] In some implementations, a search index may be utilized to perform an approximate neighbors search based on asset feature vectors of the candidate 3D object and one or more real 3D objects.

[0168] In some implementations, determining whether asset features of the candidate 3D object match authentic asset features of at least one authentic 3D object may include generating a hash of the asset features of the candidate 3D object and detecting collisions between the asset features of the candidate 3D object and the asset features of multiple authentic objects within a unit hypercube.

[0169] In some implementations, the edge magnitude of the unit hypercube may be dynamically adjusted based on the region of interest, and smaller edge magnitudes may be selected for regions of more interest for classification. For example, portions of candidate 3D objects corresponding to regions of more interest, such as visually salient portions, may be compared by utilizing hypercubes with smaller edge magnitudes.

[0170] In some implementations, the edge size of the unit hypercube may be dynamically adjusted based on the type or class of object. For example, tuning for different classes of 3D objects may be performed by adjusting the dimensions of the unit hypercube used to determine the HOG values.

[0171] Since the hash function is essentially a custom partitioned space, additional parameters may be included in the hash function to increase accuracy in specific areas of the space based on the requirements or density of the clusters. Heterogeneous space partitions may be generated using hierarchical clustering and observing the density of the generated clusters.

[0172] In some scenarios, the orientation of the candidate 3D object may not match the orientation of one or more stored real 3D objects, which may lead to a non-real object being flagged as real based on a comparison of the candidate 3D object at its default (or received) orientation, while a comparison based on a different orientation of the candidate 3D object may lead to a determination that the candidate 3D object matches one or more real 3D objects.

[0173] In some implementations, determining whether asset features of the candidate 3D object match authentic asset features of the at least one authentic 3D object includes performing a rotation-invariant comparison of the asset features of the candidate 3D object with the asset features of the at least one authentic 3D object.

[0174] For example, even if AssetFeature <assetid>< / assetid> Even if represents the entire geometry of a 3D object, an initial change in the object's orientation may result in a change in the feature vector corresponding to the object. Because the asset feature vector is a concatenation of the individual image HoG feature vectors in camera viewpoint order, the original asset feature vector can be appropriately transformed to generate asset feature vectors corresponding to other orientations of the candidate 3D object.

[0175] In some implementations, performing a rotation-invariant comparison of the asset features of the candidate 3D object with the asset features of the at least one real 3D object may include generating a plurality of rotated asset feature vectors for the candidate 3D object based on the asset features of the candidate 3D object, where each rotated asset feature vector corresponds to a particular orientation of the candidate 3D object. After the rotated asset feature vectors are generated, each of the plurality of rotated asset feature vectors is compared with the asset features of the at least one real 3D object.

[0176] In some implementations, the rotated asset feature vector may be aligned with the orientation of the camera view used to generate the asset feature vector (e.g., the image used to generate the pyramid HOG vector of the candidate 3D object).

[0177] Although an infinite set of orientations is theoretically possible, by selecting a combination of camera viewpoints for multiple orientations, a good approximation of the asset feature vector for multiple orientations may be generated.

[0178] for example, <O α , O e is the orientation in which the asset feature vector should be generated, the asset feature vector is generated by the operation AssetFeatureVector[O α , O e ] <assetid>< / assetid> = [f 0-Oα,0-Oe , f 0-Oα,1-Oe ...] may be determined by

[0179] The rotation-invariant set includes 200 feature vectors, each of which is queried against a search index to determine its nearest neighbors. In some implementations, a smaller number of alternative orientations may be used to limit the computational load. For example, approximately 20 rotations of the original asset feature vector may be determined and used for comparison.

[0180] In some other implementations, a rotated asset feature vector may be associated with an orientation that is not aligned with the camera view utilized to generate the asset feature vector.

[0181] In some implementations, a rotation-invariant k-nearest neighbor (KNN) search is performed. In some other implementations, rotation invariance is performed by appropriate modification of the embedding of the asset feature vector by applying spherical harmonics techniques during generation of the asset feature vector. In some implementations, performing a rotation-invariant comparison of the asset features of the candidate 3D object with the asset features of the plurality of real 3D objects may include applying spherical harmonics to represent the asset feature vector such that the asset feature vector is specified as a set of functions on the surface of a sphere.

[0182] If it is determined that the asset characteristics of the candidate 3D object match the real asset characteristics of at least one real 3D object, block 530 may be followed by block 540; otherwise, block 530 may be followed by block 535.

[0183] At block 535, the candidate 3D object is classified as a genuine object. In some implementations, classifying the candidate 3D object as a genuine object may include assigning a flag to the candidate 3D object, the flag being readable by a game engine and causing the game engine to enable use of the candidate 3D object in a virtual environment hosted by the game engine. For example, an image of a candidate 3D object determined to be genuine may be displayed on a screen of a user device participating in a virtual experience within the virtual environment.

[0184] In some implementations, after classifying a candidate 3D object as a genuine object, the candidate 3D object may be stored in a different location on the platform used to store authenticated 3D objects, for example, in a different storage location.

[0185] In some implementations, the calculated asset feature vector of the candidate 3D object may be stored in a data store associated with the platform. In some implementations, the entire asset feature vector may be stored. In some implementations, after classifying the candidate 3D object as a real 3D object, the features of the candidate 3D object or its assets may be stored in a storage device and / or memory (e.g., in data store 120) and utilized to authenticate other candidate 3D objects that may be subsequently received. The stored real 3D object may be made available for use in the virtual environment. A user interface may be provided that includes the real 3D object in the virtual environment. Furthermore, if the virtual environment allows a user to purchase a real object (e.g., by paying virtual and / or real currency) or gain access to the real object through a subscription, the real 3D object (which is the candidate 3D virtual object after classification in block 535) may be made available to the user.

[0186] In block 540, the candidate 3D object is classified as a non-authentic object. For example, if the vector distance between the asset feature vector of the unauthenticated virtual candidate 3D object and the asset feature vector of the authentic 3D object meets a predetermined threshold (e.g., the distance is less than a predetermined threshold), the candidate 3D object is classified as a counterfeit 3D object.

[0187] In some implementations, classifying the candidate 3D object as an inauthentic object may further include assigning a flag to the candidate 3D object, the flag being readable by a game engine and causing the game engine to prevent use of the candidate 3D object in a virtual environment hosted by the game engine.

[0188] Upon detecting an inauthentic 3D object (classifying the 3D object as inauthentic), the platform may provide a notification to the developer user that the provided 3D object is inauthentic. For example, a notification may be provided via a user interface to alert the user that the content provided by the user, e.g., some or all portions of the candidate 3D object, may not be utilized.

[0189] In some implementations, suitable alternatives to the disallowed content may be automatically suggested to the user via a notification via a user interface to enable the developer user to select an appropriate alternative to the disallowed content.

[0190] In some implementations, the developer user may be provided with the option to submit an alternative candidate 3D object. Upon receiving the alternative candidate 3D object, the virtual experience platform may classify the newly received candidate 3D object and, upon verifying the authenticity of the newly received candidate 3D object, replace the inauthentic candidate 3D object with the newly received candidate 3D object.

[0191] Blocks 505-540 may be performed (or repeated) in a different order than described above, and / or one or more steps may be omitted. For example, block 510 may be performed multiple times, e.g., to generate asset features for multiple candidate 3D objects and generate images of those candidate 3D objects before performing a comparison with the real 3D objects.

[0192] In some implementations, received candidate 3D objects on the gaming platform may be scanned at a predetermined frequency (e.g., daily, every other day, hourly, etc.) to detect any inauthentic objects and mitigate user access to such objects. In some implementations, candidate 3D objects that are more likely to be copied may be scanned more frequently than 3D objects that are less likely to be copied. In some implementations, method 500 may be performed each time a new 3D object is received via user upload. In some implementations, method 500 may be performed when a stored object is modified.

[0193] In some implementations, user feedback regarding inauthentic objects the user encounters on the platform may be utilized to update the threshold distance, and method 500 may be performed for one or more previously authenticated 3D objects.

[0194] In some implementations, one or more parameters, such as the number of camera views, the threshold distance, the number of pyramid HOG levels, the number of asset feature vector orientations to use during comparison (inference), etc., may be updated (adjusted) based on previous detection results of the 3D object.

[0195] In some implementations, classification as a genuine 3D object may be used as a signal and combined with other signals (e.g., manual review of the 3D object, developer ratings associated with the developer uploading the 3D object, etc.) to further classify the 3D object.

[0196] In some implementations, after classifying a candidate 3D object as a genuine and / or true object, the asset characteristics of the candidate 3D object may be stored (e.g., in data store 120) and utilized to authenticate other 3D objects that may be subsequently received.

[0197] FIG. 6A is a schematic diagram illustrating an asset feature generator according to some implementations.

[0198] As depicted in FIG. 6A, the asset feature generator 610 includes an asset renderer 612, a feature generator 614, a rotation invariant generator 616, and a principal component analysis (PCA) transformer (618).

[0199] The asset feature generator 610 receives (or acquires) an asset model 620, e.g., a 3D model of a candidate 3D object. The asset renderer 612 is utilized to generate one or more images corresponding to different camera positions. The feature generator 614 is utilized to generate feature vectors, e.g., HOG vectors, corresponding to each of the one or more images. The rotation invariance generator 616 is utilized to determine multiple asset feature vectors, each corresponding to a different orientation of the candidate 3D object. The PCA transformer or other dimensionality reduction transformer 618 is utilized to reduce the dimensionality of the asset feature vectors.

[0200] Asset feature vectors 622 corresponding to different orientations (e.g., FV1 to FV N ) is provided as output by the asset feature generator.

[0201] FIG. 6B is a schematic diagram depicting the comparison of a candidate 3D object with a real object according to some implementations.

[0202] 6B , a candidate asset, e.g., a 3D model 630 of a candidate 3D object, is received at an asset feature generator 610, which generates a rotation-invariant combination of asset feature vectors. For example, in some implementations, approximately 200 feature vectors may be generated, each corresponding to a different orientation of the 3D mesh of the candidate 3D object. The search index 225 is queried with each of the generated asset feature vectors to determine rotation-invariant query results 640. In some implementations, a distributed search and analysis engine, such as MILVUS or ElasticSearch, may be utilized to perform the search.

[0203] Additionally, a single asset feature vector corresponding to the default orientation of the candidate 3D object is indexed and stored (645), for example, in the search index 225, for subsequent queries.

[0204] FIG. 7 is a schematic diagram depicting an exemplary generation of semantic feature vectors according to some implementations.

[0205] 7, a machine learning (ML) model 720 includes a text encoder 725 and an image encoder 730 that utilize one or more model parameters (weights) 735. A control pre-training 710 may be performed in which a training set of images 750 and corresponding text descriptions 740 is provided to the ML model.

[0206] The ML model is trained to combine knowledge of images and their associated textual descriptions, thereby enabling the ML model to learn rich and meaningful visual representations. The pre-training process involves training the model to predict matching image-text pairs while contrasting matching image-text pairs with mismatched pairs. The goal of this contrastive training is to enable the ML model to capture semantic similarities and differences between different images and their corresponding text. During training, model parameters 735 may be adjusted based on the contrastive training to configure the model to accurately predict matching and mismatched pairs. In some implementations, the ML model 720 may include a neural network with multiple layers, each layer including one or more nodes, and the model parameters 735 may include weights for one or more of the nodes.

[0207] After training, the trained image encoder 730 may be utilized to perform zero-shot prediction 760. In zero-shot prediction, the trained image encoder 730 is provided with images of a 3D object and generates corresponding semantic feature vectors 775.

[0208] By providing an image of a 3D object, e.g., a thumbnail of an asset, as input to an ML model, semantic features (semantic feature vectors) that capture the high-level concepts and visual semantics of the asset can be obtained. These features are represented as high-dimensional vectors, with each dimension encoding specific semantic information learned by the model during training. The semantic features extracted by the ML model provide a holistic representation of the asset's visual content.

[0209] In various implementations, the semantic feature vector may be generated from the image of the 3D object by utilizing other types of ML models, such as an image encoder.

[0210] FIG. 8 is a flow diagram illustrating an example method for classifying candidate 3D objects according to some implementations.

[0211] In some implementations, method 800 may be implemented to classify candidate 3D objects on, for example, the virtual experience server 102 described with reference to FIG. 1 . In some other implementations, method 800 may be implemented on, for example, one or more servers described with reference to FIG. 1 . In the described example, the implementing system includes one or more digital processors or processing circuits (“processors”) and one or more storage devices (e.g., data store 120 or other storage). In some implementations, different components of one or more servers and / or clients may perform different blocks or other portions of method 800. In some examples, a first device is described as performing blocks of method 800. Some implementations may have one or more blocks of method 800 performed by one or more other devices (e.g., other client devices or server devices) that can send results or data to the first device.

[0212] Method 800 may begin at block 810. At block 810, geometric asset features of the candidate 3D object are determined based on multiple images of the candidate 3D object. In some implementations, the geometric asset features may be asset feature vectors described with reference to FIG. 5.

[0213] In some implementations, before determining the geometric asset characteristics, method 800 may include generating a plurality of images of the candidate 3D object, where each image of the plurality of images of the candidate 3D object may be acquired based on a respective camera position of two or more camera positions.

[0214] In some implementations, determining the geometric asset features of the candidate 3D object may include determining one or more Histogram of Oriented Gradients (HOG) vectors for each image of the plurality of images of the candidate 3D object, and determining the geometric asset features of the candidate 3D object based on the one or more HOG vectors for each of the plurality of images of the candidate 3D object. Block 810 may be followed by block 815.

[0215] At block 815, a semantic feature vector of the candidate 3D object is determined. In some implementations, determining the semantic feature vector of the candidate 3D object may include obtaining one or more images of the candidate 3D object and applying a pre-trained machine learning model to the one or more images to determine the semantic feature vector of the candidate 3D object, where the machine learning model is trained by contrastive learning based on predicting matching pairs of images and associated text from a training dataset, and the semantic feature vector is a high-dimensional vector, with each dimension of the high-dimensional vector encoding a respective semantic information. In some implementations, the semantic feature vector may be a vector representation or embedding of semantic content in the image.

[0216] In some implementations, the one or more images of the candidate 3D object may include a thumbnail image of the candidate 3D object. In some implementations, the one or more images of the candidate 3D object may include an image used to determine geometric asset features, for example, one of multiple images used to determine an HoG vector for the candidate 3D object. Block 815 may be followed by block 820.

[0217] In block 820, a similarity between the candidate 3D object and the reference 3D object is determined based on a comparison of the geometric asset features and semantic feature vectors of the candidate 3D object with the geometric asset features and semantic feature vectors of the reference 3D object.

[0218] In some implementations, the similarity may be determined based on a fused vector distance that combines a first vector distance calculated between the geometric asset features of the candidate 3D object and the geometric asset features of the reference 3D object and a second vector distance (or angle) calculated between the semantic feature vector of the candidate 3D object and the semantic feature vector of the reference 3D object.

[0219] Thus, a first vector distance may be calculated between the geometric asset features of the candidate 3D object and the geometric asset features of the reference 3D object, and a second vector distance may be calculated between the semantic feature vector of the candidate 3D object and the semantic feature vector of the reference 3D object. A fused vector distance may be determined by combining the first vector distance and the second vector distance. Utilizing the fused vector distance by combining distances determined based on different types of attributes (e.g., geometric, semantic, etc.) for similarity analysis can provide a more comprehensive, accurate, and robust representation of similarity than any single modality alone could provide.

[0220] In some implementations, the first vector distance may be a Euclidean vector distance between the geometric asset features of the candidate 3D object and the reference 3D object, and the second vector distance may be a cosine similarity between the semantic feature vectors of the candidate 3D object and the reference 3D object.

[0221] In some implementations, combining the first vector distance and the second vector distance may include applying an appropriate transformation function to the first vector distance and the second vector distance. In some implementations, the transformation function is a monotonic function that maps the geometric distance to a transformed value that may be constrained to fall within a particular range. For example, in some implementations, the geometric distance and / or the semantic distance may be mapped by the transformation function such that the transformed value is within a constrained range, e.g., between 0 and 1.

[0222] In some implementations, the transformation function may be a normalization function, for example a min-max function, applied to the geometric and / or semantic distances.

[0223] The fused vector distance between two 3D objects is a measure of similarity between the two objects and may be used as a measure to classify 3D objects, determine non-real 3D objects, determine similar 3D objects, determine dissimilar 3D objects, etc.

[0224] The geometric distance between assets i and j can be denoted as D_geom(i, j), the semantic distance between assets i and j can be denoted as D_semantic(i, j), and the fused vector distance between assets i and j can be represented by D_fused(i, j).

[0225] The fused vector distance is equal to the geometric distance passed through a first normalization function combined with (added to) the semantic distance passed through a second normalization function. The fused vector distance may be determined as follows: D fused (i, j) = f geom (D geom (i, j)) + f semantic (D semantic (i, j) During the ceremony f geom ( .) is a monotonic function that maps geometric distances to transformed values. f geom ( .) may be specified as any suitable transformation function, such as a scaling, normalization, or function that adjusts geometric distances to a particular range or format; f semantic ( .) is a monotonic function that maps semantic distances to transformed values. geom ( .) as well as f semantic The (.) may be specified to be a function that modifies the semantic distance to a particular range or format.

[0226] A fused vector distance between representations of assets i and j may be determined by applying respective functions to the geometric distance and the semantic distance and summing them. This formula allows for the combination of geometric and semantic distances while taking into account their individual transformations.

[0227] In some implementations, f geom ( .) and f semantic The specific form of ( . ) may be tailored based on the nature of the distances, the desired characteristics of the fused vector distances, and the specific requirements of the application to which the fused vector distances will be applied. These functions may be defined based on domain knowledge, empirical analysis, or other considerations to optimize the fusion process and achieve the intended representation of similarity. Block 820 may be followed by block 825.

[0228] At block 825, it may be determined whether the similarity between the candidate 3D object and the reference 3D object meets one or more predetermined thresholds. In some implementations, the candidate 3D object is classified based on the similarity between the candidate 3D object and the reference 3D object.

[0229] In some implementations, the one or more predetermined thresholds may include multiple thresholds that can enable appropriate classification of the candidate 3D object. For example, the first predetermined threshold may be a similarity threshold that may be used to determine that the candidate 3D object is similar to a reference 3D object. As another example, the second predetermined threshold may be an authenticity threshold that may be used to determine whether the candidate 3D object is an inauthentic 3D object based on a comparison with a reference 3D object that is a genuine 3D object.

[0230] If it is determined that the similarity between the candidate 3D object and the reference 3D object meets one or more predetermined thresholds, block 825 may be followed by block 830; if not, block 825 may be followed by block 835.

[0231] For example, if the fused vector distance is determined to meet the non-authentic object threshold, the candidate 3D object may be classified as a non-authentic object, and if the fused vector distance does not meet the non-authentic object threshold, the candidate 3D object may be classified as a genuine object.

[0232] Similarly, if the fused vector distance is determined to meet the similarity threshold, the candidate 3D object may be classified as an object similar to the reference 3D object, and if the fused vector distance does not meet the similarity threshold, the candidate 3D object may be classified as an object dissimilar to the reference 3D object.

[0233] In block 830, the candidate 3D objects are classified as similar objects.

[0234] In block 835, the candidate 3D object is classified as a dissimilar object.

[0235] In some implementations, the fused vector distances may be utilized for additional applications on the virtual experience platform, such as upload fee prediction, asset monitoring metrics, market valuation, and the like.

[0236] In some implementations, the fused vector distance may be used to determine non-authentic objects that are copies of authentic objects but may have been modified beyond geometric or visual similarity, for example, copyrighted objects (characters) that have been modified to have a different shape. In some implementations, the fused vector distance may be used in a recommendation system for a user, for example, to suggest objects that are similar to other objects associated (liked) by the user.

[0237] In some implementations, the comparison of the 3D objects based on their respective objects may be performed as a comparison of a single candidate 3D object with one or more reference 3D objects, hi some other implementations, the comparison of the 3D objects may be performed as a batch process in which a fused vector distance is determined for each pair in the collection of 3D objects to determine multiple fused vector distances (similarity measures) between pairs of 3D objects in the collection of 3D objects.

[0238] For example, in some implementations, a fused vector distance matrix may be formed (constructed) that includes respective fused vector distances between a plurality of 3D objects. In some implementations, the plurality of 3D objects may include a candidate 3D object, for example, a 3D object that has just been uploaded to the virtual experience platform.

[0239] In some implementations, multidimensional scaling (MDS) may be applied to the fused vector distance matrix to determine multiple updated feature vectors for each of the multiple 3D objects. Multidimensional scaling is a technique for finding a lower-dimensional representation of the data while preserving pairwise distances as closely as possible.

[0240] Each updated feature vector represents a fused vector distance of the corresponding 3D object to other 3D objects of the plurality of 3D objects, and the dimensionality of the updated feature vector is lower than the dimensionality of the corresponding geometric asset feature and semantic feature vectors, which provides the technical advantage of determining a distance-preserving representation of the 3D objects, allowing for more efficient storage and / or distance calculation for future comparisons between the 3D objects.

[0241] In some implementations, the updated feature vector may be utilized in a recommender system, for example, a content recommendation system, to efficiently determine a set of similar 3D objects.

[0242] The fused vector distance matrix obtained from the distance fusion method that combines geometric distance and semantic distance is D fused The fused vector distance matrix D fused is an n x n matrix, where n is the number of assets included in the analysis.

[0243] In this case, the fused vector distance matrix D fusedMDS may be utilized to map x, y ...

[0244] The MDS algorithm computes the fused vector distance matrix D fused It takes as input a set of new features for each asset that capture the asset relationships in the reduced-dimensional space. These new features, also known as MDS coordinates, represent a transformed representation of the asset based on the asset's fused vector distances.

[0245] The MDS coordinates may be represented by a matrix X where each row corresponds to an asset and each column represents a dimension of the reduced dimensional space. The MDS coordinates X are then calculated using the fused vector distance matrix D fused can be obtained by applying the MDS algorithm to

[0246] By applying multidimensional scaling to the fused vector distance matrix, a new set of features (X) can be obtained that captures the essence of the fused vector distances.

[0247] In some implementations, the new set of features may be used to represent the 3D object for future comparison and may be stored and / or indexed, for example, in search index 225 or search index 282 described with reference to FIG. 2 .

[0248] In some implementations, classifying the candidate 3D object may include determining the uniqueness of the candidate 3D object. A corresponding fused vector distance may be determined between the candidate 3D object and each of the multiple reference 3D objects. A multiple neighboring 3D objects of the candidate 3D object may be determined based on the determined distances. A local density may be determined for the candidate 3D object by calculating the average distance between the candidate 3D object and the multiple neighboring 3D objects. A uniqueness score may be determined based on the local density of the candidate 3D object and the maximum local density of the multiple reference 3D objects.

[0249] In some implementations, before determining the similarity between the candidate 3D object and the reference 3D object, the method may further include obtaining a first plurality of semantically similar reference 3D objects, obtaining a second plurality of geometrically similar reference 3D objects, forming a combined pool of geometrically similar and semantically similar reference 3D objects based on the first plurality of semantically similar reference 3D objects and the second plurality of geometrically similar reference 3D objects, and selecting a reference 3D object from the combined pool of geometrically similar and semantically similar reference 3D objects.

[0250] This approach can be useful when the search and ranking process needs to be performed in real time (e.g., immediately as objects are uploaded to the platform) or when multiple algorithms provide complementary insights into the visual similarity of assets. This approach offers a way to incorporate the benefits of both algorithms and create a more sophisticated and comprehensive search and ranking mechanism.

[0251] Blocks 810-835 may be performed (or repeated) in a different order than described above, and / or one or more steps may be omitted. For example, block 825 may be performed multiple times, e.g., to perform comparisons of the candidate 3D object with multiple reference 3D objects.

[0252] FIG. 9 is a flow diagram illustrating an exemplary method for performing a similarity analysis, according to some implementations.

[0253] In some implementations, method 900 may be implemented to classify candidate 3D objects on, for example, the virtual experience server 102 described with reference to FIG. 1 . In some other implementations, method 900 may be implemented on, for example, one or more servers described with reference to FIG. 1 . In the described example, the implementing system includes one or more digital processors or processing circuits (“processors”) and one or more storage devices (e.g., data store 120 or other storage). In some implementations, different components of one or more servers and / or clients may perform different blocks or other portions of method 900. In some examples, a first device is described as performing blocks of method 900. Some implementations may have one or more blocks of method 900 performed by one or more other devices (e.g., other client devices or server devices) that can send results or data to the first device.

[0254] In some implementations, method 900 may be utilized to perform real-time similarity analysis for comparing 3D objects (assets). For a given asset, the closest set of assets based on separate geometric and semantic comparisons is determined, and then real-time fusion of distances is performed, followed by re-ranking of the set of assets. This method enables real-time fusion of similarity results from multiple algorithms, providing a flexible and dynamic approach to asset search and ranking.

[0255] Method 900 may start at block 905. At block 905, a request to determine 3D objects similar to a candidate 3D object is received. In some implementations, the request may be received from another method running on the virtual experience platform. For example, the request may be received in association with a method running on the virtual experience platform to suggest and / or determine prices for candidate 3D objects, where the predicted / suggested prices may be based on prices of similar 3D objects on the virtual experience platform. In another example, the request may be received in association with a method running on the virtual experience platform to suggest one or more new 3D objects to a user based on the user's preferences for the candidate 3D objects. Block 905 may be followed by block 910.

[0256] A first plurality of geometrically similar 3D objects may be obtained, for example, retrieved from a database or repository of 3D objects, in block 910. Block 910 may be followed by block 915.

[0257] At block 915, a second plurality of semantically similar 3D objects may be obtained, for example, retrieved from a database or repository of 3D objects. In some implementations, the first plurality of geometrically similar 3D objects and the second plurality of semantically similar 3D objects may include the same number (N) of 3D objects. In some other implementations, a greater weighting may be given to either geometrically similar 3D objects or semantically similar 3D objects, and accordingly, more 3D objects may be included. Block 915 may be followed by block 920.

[0258] A combined pool of nearby 3D objects is formed by combining the first plurality of geometrically similar 3D objects with the second plurality of semantically similar 3D objects in block 920. Block 920 may be followed by block 925.

[0259] In block 925, a fused vector distance matrix may be determined by calculating the fused vector distance between the candidate 3D object and each neighboring 3D object (from the combined pool). Block 925 may be followed by block 930.

[0260] In block 930, a third plurality of similar 3D objects (a re-ranked list) may be determined based on the ranking of the fused vector distances. The specific number of the third plurality of similar 3D objects may be configurable, for example, by a user, automatically by a method, etc. The re-ranked list represents a fusion of the top n assets from both algorithms, prioritizing assets that exhibit similarity in both geometric and semantic aspects. Block 930 may be followed by block 935.

[0261] A third plurality of similar 3D objects may be transmitted at block 935. By performing fusion and re-ranking on the fly, this approach leverages the strengths of both algorithms (semantic and geometric) and dynamically combines results from both algorithms based on the particular candidate 3D objects included in the query. It allows for adaptability and flexibility in considering the unique attributes of each algorithm and their contribution to the overall similarity assessment.

[0262] Method 900 may be particularly advantageous when the search and ranking process is to be performed in real time or when multiple algorithms provide complementary insights into the visual similarity of assets. Method 900 provides a way to incorporate the benefits of both algorithms to create a more sophisticated and comprehensive search and ranking mechanism.

[0263] Blocks 905-935 may be performed (or repeated) in a different order than described above, and / or one or more steps may be omitted.

[0264] FIG. 10 is a flow diagram illustrating an exemplary method for determining similarity, according to some implementations.

[0265] In some implementations, method 1000 may be implemented to determine similarities of candidate 3D objects on, for example, the virtual experience server 102 described with reference to FIG. 1 . In some other implementations, method 1000 may be implemented on, for example, one or more servers described with reference to FIG. 1 . In the described example, the implementing system includes one or more digital processors or processing circuits (“processors”) and one or more storage devices (e.g., data store 120 or other storage). In some implementations, different components of one or more servers and / or clients may perform different blocks or other portions of method 1000. In some examples, a first device is described as performing blocks of method 1000. Some implementations may have one or more blocks of method 1000 performed by one or more other devices (e.g., other client devices or server devices) that can send results or data to the first device.

[0266] Method 1000 may begin at block 1005. At block 1005, a candidate 3D object is received. The candidate 3D object may be received from a developer device, such as, for example, developer device 130 described with reference to FIG. 1. In some implementations, the candidate 3D object may be a candidate 3D object previously received at a computing device associated with the virtual experience platform and / or stored in a storage device associated with the virtual experience platform.

[0267] In some implementations, candidate 3D objects may be received as part of a classification process and / or workflow. In some implementations, a uniform record locator (URL), token, or other asset identifier may be utilized to provide a link to the candidate 3D object, and the processor may utilize the provided link, token, URL, etc. to retrieve the candidate 3D object. The candidate 3D object may be implemented as a 3D model and may include a surface representation (also known as a skin or mesh) and a hierarchical collection of interconnected bones (also known as a skeleton or rig) used to render the object. The rig may be utilized to animate a character and simulate motion and actions by the object. The 3D model may be represented as a data structure, and one or more parameters of the data structure may be modified to change various characteristics (attributes) of the object and / or character, such as dimensions (height, width, diameter, girth, etc.), body type and / or shape, movement style, number / type of parts, proportions of the object or body part (e.g., shoulder-to-hip ratio), head size, etc.

[0268] In some implementations, receiving the candidate 3D object may include obtaining a 3D mesh of the candidate 3D object. In some implementations, the candidate 3D object may be a 3D object created by a developer using on-platform tools and / or a 3D object uploaded to the platform after off-platform design.

[0269] Block 1005 may be followed by block 1010. In block 1010, geometric asset features of the candidate 3D object are generated.

[0270] Block 1010 may be followed by block 1015. In block 1015, semantic feature vectors of the candidate 3D objects are determined.

[0271] Block 1015 may be followed by block 1020. In block 1020, a fused vector distance between the candidate 3D object and the reference 3D object is determined.

[0272] In some implementations, the fused vector distances may be utilized for additional applications on the virtual experience platform, such as upload fee prediction, asset monitoring metrics, market valuation, and the like.

[0273] Blocks 1005-1020 may be performed (or repeated) in a different order than described above, and / or one or more steps may be omitted. For example, block 1020 may be performed multiple times to, for example, generate fused vector distances for the candidate 3D object using multiple reference 3D objects, thereby performing comparisons with multiple reference 3D objects.

[0274] FIG. 11A is a flow diagram illustrating an example method for determining a geometric uniqueness score of a 3D object, according to some implementations.

[0275] In some implementations, method 1100 may be implemented to determine a geometric uniqueness score of a 3D object on, for example, the virtual experience server 102 described with reference to FIG. 1 . In some other implementations, method 1100 may be implemented on, for example, one or more servers described with reference to FIG. 1 . In the described example, the implementing system includes one or more digital processors or processing circuits (“processors”) and one or more storage devices (e.g., data store 120 or other storage). In some implementations, different components of one or more servers and / or clients may perform different blocks or other portions of method 1100. In some examples, a first device is described as performing blocks of method 1100. Some implementations may have one or more blocks of method 1100 performed by one or more other devices (e.g., other client devices or server devices) that can send results or data to the first device.

[0276] 3D objects submitted to the virtual experience platform market may vary widely in their geometries. Quantifying the geometric uniqueness of 3D objects can be useful for several applications, such as upload fee prediction, asset monitoring index, and market evaluation.

[0277] The method 1100 may begin at block 1105. At block 1105, a candidate 3D object is obtained.

[0278] Block 1105 may be followed by block 1110. In block 1110, geometric asset characteristics of the candidate 3D object may be determined based on multiple images of the candidate 3D object. In some implementations, method 1100 may further include, prior to determining the geometric asset characteristics, generating multiple images of the candidate 3D object, each image of the multiple images of the candidate 3D object from a respective camera position of the two or more camera positions.

[0279] Block 1110 may be followed by block 1115. In block 1115, corresponding distances are determined between geometric asset features of the candidate 3D object and geometric asset features of each of a plurality of reference 3D objects. In some implementations, the distances may be pairwise Euclidean distances between the candidate 3D object and the reference 3D object. In some implementations, pairwise Euclidean distances may be determined between the MultiViewHoG feature vectors of all 3D objects in the dataset.

[0280] In some scenarios, the dataset may be large and it may be computationally infeasible to calculate pairwise distances for the entire dataset. In such scenarios, an approximate nearest neighbor solution, such as FAISS, ScaNN, or Elastic Search, may be utilized to efficiently find the N nearest neighbors of each object. Block 1115 may be followed by block 1120.

[0281] At block 1120, a plurality of neighboring 3D objects of the candidate 3D object is determined based on the determined distance. In some implementations, determining a plurality of neighboring 3D objects of the candidate 3D object may include applying an approximate nearest neighbor technique to geometric asset features of the candidate 3D object. For example, techniques such as FAISS, ScaNN, or Elastic Search may be utilized to efficiently find the closest set of neighbors for each 3D object.

[0282] In some implementations, the plurality of neighboring 3D objects may be determined by selecting from all available 3D objects on the virtual experience platform. In some other implementations, the plurality of neighboring 3D objects may be determined by selecting from available 3D objects on the virtual experience platform that are of the same type (category) as the candidate 3D object. Block 1120 may be followed by block 1125.

[0283] At block 1125, the local density of neighboring 3D objects of the candidate 3D object is determined by calculating the average distance between the candidate 3D object and multiple neighboring 3D objects. The local density (local density score) quantifies the sparseness of the space around the 3D object relative to the average sparseness around all 3D objects in the dataset.

[0284] For each 3D object, the average distance to its N nearest neighbors is determined. This gives a measure of the local density of each object. The local density LD of object i is i The determination of can be expressed by the following equation:

[0285]

number

[0286] where d(i, j) is the Euclidean distance between the feature vectors of objects i and j, and N is the number of nearest neighbors considered. i | is the norm of the feature vector of the query object. Block 1125 may be followed by block 1130.

[0287] At block 1130, a geometric uniqueness score is determined based on the local densities of the candidate 3D object's neighboring 3D objects and the maximum local densities of the plurality of reference 3D objects, where the maximum of all local densities across all 3D objects in the dataset is determined to determine the maximum local density MD, which is calculated using the formula MD = max{LD i} where i∈[0, M] can be expressed by where M is the total number of objects in the dataset.

[0288] In some implementations, the maximum local density of the reference 3D object may be predetermined for all 3D objects in the virtual experience platform. In some other implementations, the maximum local density of the reference 3D object may be predetermined for all 3D objects based on their type / category. For example, the maximum local density of the reference 3D object may be determined separately for hats, gloves, coats, a particular avatar, a particular character, etc.

[0289] Since asset type can usually have a strong influence on the geometry of an object (e.g., shirt, pants, etc.), the determination of MD is based on 3D objects of similar categories. In some implementations, only objects of the same asset type may be considered when calculating the maximum local density. The formula for calculating the maximum local density based on asset type is: MD assettype = max{LD j} where assettype(i) = assettype(j), j∈[0, M] It may be expressed by:

[0290] For each object, a geometry uniqueness score may be determined by calculating the ratio of the local density of the 3D object to the maximum local density across all 3D objects of the same asset type.

[0291]

number

[0292] In some implementations, a generalized uniqueness score may be determined based on, for example, the fused vector distance between the candidate 3D object and the reference 3D object, which allows for the determination of uniqueness based on the geometry and other attributes of the 3D object.

[0293] The geometric uniqueness score may be utilized within the virtual experience platform for various purposes, for example, a predicted or suggested price for a candidate 3D object may be determined based on the geometric uniqueness score and may be utilized upon uploading the candidate 3D object to the platform by a user.

[0294] In some implementations, the predicted or suggested price of the candidate 3D object may be displayed via a user interface, for example, on a display screen of a user device.

[0295] In some implementations, asset monitoring metrics or market valuation metrics may be determined for candidate 3D objects based on the geometric uniqueness score. For example, a user who uploads multiple 3D objects with a relatively high geometric uniqueness score may be recognized and / or rewarded. Similarly, a particular virtual experience that includes multiple 3D objects with a relatively high geometric uniqueness score may be placed in a separate category from virtual experiences that include 3D objects with a relatively low geometric uniqueness score. The metrics may then be further utilized when a list of virtual experiences (e.g., games) is displayed to a user during a virtual experience search or discovery process.

[0296] In some implementations, the uniqueness score may be used in a recommender system to increase the diversity of results. For example, the uniqueness score of a set of identified objects may be considered as an additional parameter for determining which subset of the set of identified objects should be displayed to a user. This may enable a user to view a richer and more diverse list of objects when the set of objects is displayed to a user, for example, when viewing recommendations on a landing page.

[0297] In some implementations, unexpected changes to the uniqueness score of a 3D object (e.g., a change in the uniqueness score threshold) may indicate a sudden surge in the creation of copies and uploads to the platform.

[0298] Blocks 1105-1130 may be performed (or repeated) in a different order than described above, and / or one or more steps may be omitted.

[0299] 11B depicts an example 3D object with a relatively high geometric uniqueness score. As can be seen, the shape and / or texture of the 3D object is different from the other displayed objects.

[0300] 11C depicts example 3D objects with relatively low geometric uniqueness scores. Each of the two rows depicts a collection of similar objects with low geometric uniqueness scores. As can be seen, the shape and / or texture of the "gift box" and "egg" objects are similar to the other objects in the row.

[0301] 12 is a block diagram of an exemplary computing device 1200 that may be used to implement one or more features described herein. In one example, device 1200 implements a computer device (e.g., 102, 110, and / or 130 of FIG. 1) and may be used to perform implementations of suitable methods described herein. Computing device 1200 can be any suitable computer system, server, or other electronic or hardware device. For example, computing device 1200 can be a mainframe computer, desktop computer, workstation, portable computer, or electronic device (portable device, mobile device, cell phone, smartphone, tablet computer, television, TV set-top box, personal digital assistant (PDA), media player, gaming device, wearable device, etc.). In some implementations, device 1200 includes a processor 1202, memory 1204, input / output (I / O) interface 1206, and audio / video input / output device(s) 1214.

[0302] Processor 1202 can be one or more processors, processing devices, and / or processing circuits for executing program code and controlling the basic operations of device 1200. A "processor" includes any suitable hardware and / or software system, mechanism, or component that processes data, signals, or other information. A processor may include a general-purpose central processing unit (CPU), multiple processing units, a system with dedicated circuitry for implementing functions, or other systems. Processing need not be limited to a particular geographic location or have temporal limitations. For example, a processor may perform its functions in "real time," "offline," "batch mode," etc. Portions of processing may be performed at different times and in different locations by different (or the same) processing systems. A computer may be any processor in communication with memory.

[0303] Memory 1204 is generally provided within device 1200 for access by processor 1202, is suitable for storing instructions for execution by the processor, and may be any suitable processor-readable storage medium, e.g., random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory, etc., located separately from and / or integrated with processor 1202. Memory 1204 may store software operated by processor 1202 on server device 1200, including an operating system 1208, one or more applications 1210, e.g., an audio spatialization application, a sound application, a content management application, and application data 1212. In some implementations, applications 1210 may include instructions that enable processor 1202 to perform (or control functions of) some or all of the functions described herein, e.g., the methods described in connection with FIGS. 5, 8, 9, 10, and 11A.

[0304] For example, application 1210 may include an audio spatialization module that can provide audio spatialization within an online virtual experience server (e.g., 102) as described herein. Any software in memory 1204 may alternatively be stored in any other suitable storage location or computer-readable medium. Additionally, memory 1204 (and / or other connected storage devices) may store instructions and data used in the features described herein. Memory 1204 and any other type of storage (such as magnetic disks, optical disks, magnetic tape, or other tangible media) may be considered "storage" or "storage devices."

[0305] The I / O interface 1206 may provide functionality to allow the server device 1200 to interface with other systems and devices. For example, network communication devices, storage devices (e.g., memory and / or data store 108), and input / output devices may communicate through the interface 1206. In some implementations, the I / O interface may connect to interface devices including input devices (keyboards, pointing devices, touchscreens, microphones, cameras, scanners, etc.) and / or output devices (display devices, speaker devices, printers, monitors, etc.).

[0306] The audio / video input / output devices 1214 may include user input devices (e.g., a mouse, etc.) that may be used to receive user input, display devices (e.g., a screen, a monitor, etc.) that may be used to provide graphical and / or visual output, and / or combined input and display devices.

[0307] For ease of illustration, FIG. 12 shows one block for each of processor 1202, memory 1204, I / O interface 1206, and software blocks 1208 and 1210. These blocks may represent one or more processors, processing devices or circuits, operating systems, memories, I / O interfaces, applications, and / or software engines. In other implementations, device 1200 may have less than all of the components shown and / or may have other elements, including other types of elements instead of or in addition to the elements shown herein. Although online virtual experience server 102 is described as performing the operations as described in some implementations herein, any suitable component or combination of components of online virtual experience server 102 or a similar system, or any suitable processor or processors associated with such a system, may perform the described operations.

[0308] A user device can also implement and / or be used with the features described herein. An exemplary user device can be a computing device that includes several components similar to device 1200, e.g., processor 1202, memory 1204, and I / O interface 1206. An operating system, software, and applications suitable for the user device can be provided in the memory and used by the processor. The I / O interface for the user device can be connected to a network communication device and input and output devices, e.g., a microphone for capturing sound, a camera for capturing images or video, a mouse for capturing user input, a gesture device for recognizing user gestures, a touchscreen for detecting user input, an audio speaker device for outputting sound, a display device for outputting images or video, or other output devices. For example, a display device within audio / video input / output devices 1214 may be connected to (or included in) device 1200 for displaying pre- and post-processing images as described herein, and such a display device may include any suitable display device, e.g., an LCD, LED, or plasma display screen, a CRT, a television, a monitor, a touchscreen, a 3-D display screen, a projector, or other visual display device. Some implementations may provide an audio output device, e.g., text-to-speech voice output or synthesis.

[0309] One or more methods described herein (e.g., methods 500, 800, 900, 1000, 1100, etc.) can be implemented by computer program instructions or code that can be executed on a computer. For example, the code can be executed by one or more digital processors (e.g., microprocessors or other processing circuits) and stored in a computer program product that includes a non-transitory computer-readable medium (e.g., a storage medium), such as a magnetic, optical, electromagnetic, or semiconductor storage medium, including semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), flash memory, rigid magnetic disks, optical disks, solid-state memory drives, etc. The program instructions can also be contained in and provided as electronic signals, for example, in the form of software as a service (SaaS) delivered from a server (e.g., a distributed system and / or a cloud computing system). Alternatively, one or more methods can be implemented in hardware (e.g., logic gates) or a combination of hardware and software. Exemplary hardware can be a programmable processor (e.g., a Field Programmable Gate Array (FPGA), a Complex Programmable Logic Device), a general-purpose processor, a graphics processor, an application-specific integrated circuit (ASIC), etc. One or more methods may be implemented as part of or a component of an application running on the system, or as an application or software running in conjunction with other applications and the operating system.

[0310] One or more methods described herein may be performed within a standalone program that may run on any type of computing device, a program running on a web browser, or a mobile application (“app”) running on a mobile computing device (e.g., a cell phone, a smartphone, a tablet computer, a wearable device (such as a watch, armband, jewelry, hat, goggles, glasses, etc.), a laptop computer, etc.). In one example, a client / server architecture may be used, e.g., a mobile computing device (as a user device) sends user input data to a server device and receives final output data for output (e.g., for display) from the server. In another example, all computations may be performed within a mobile app (and / or other apps) on the mobile computing device. In another example, computations may be split between the mobile computing device and one or more server devices.

[0311] Although the description has been given in connection with specific implementations thereof, these specific implementations are illustrative only and not limiting, and the concepts illustrated in the examples may be applied to other examples and implementations.

[0312] It should be noted that the functional blocks, operations, features, methods, devices, and systems described in this disclosure may be integrated or divided into different combinations of systems, devices, and functional blocks as known to those skilled in the art. Any suitable programming language and programming techniques may be used to implement the routines of a particular implementation. Different programming techniques, for example, procedural or object-oriented programming techniques, may be used. The routines may be executed on a single processing device or multiple processors. While steps, operations, or computations may be presented in a particular order, the order may be changed in different particular implementations. In some implementations, multiple steps or operations shown herein as sequential may be performed simultaneously. [Explanation of symbols]

[0313] 100 System Architecture 102 Virtual Experience Server 104 Virtual Experience Engine 106 Virtual experience applications and games 108 Graphics Engine 110 User Devices 110a User Device 110b User Device 110n User Devices 112 Virtual experience application, virtual experience engine 114 Input / Output (I / O) Interface 116 Audio Engine 120 data stores 122 Network 130 Developer Devices, User Devices 132 Virtual Experience Applications 134 Input / Output (I / O) Interface 140 Content Management Server 144 graphics engine, audio engine 146 Classification Controller 148 data stores 200 Object Classification System 202 Feature Extraction Module, Feature Extractor 204 Feature Encoder 210 Match Detector 212 Collision Detector 214 Vector Distance Calculator 220 Datastore 225 Search Index 230 Data Store, Object Repository 250 Similarity Analysis System 252 Geometric Feature Extractor 254 Semantic Feature Extractor 260 Analysis Module 262 Vector Distance Calculator 264 normalization module 266 Match Detector 268 Uniqueness Evaluator 270 Object Ranking Module 272 Pricing module, price predictor 280 candidate object storage 282 Search Index Storage 284 Reference Object Repository, Reference Object Storage 610 Asset Feature Generator 612 Asset Renderer 614 Feature Generator 616 Rotation Invariant Generator, Rotation Invariant Generator 618 Principal Component Analysis (PCA) transformers and other dimension reduction transformers 620 Asset Model 622 asset feature vectors 630 3D models 640 Rotation Invariant Query Results 710 Controlled Pre-Learning 720 Machine Learning (ML) Models 725 Text Encoder 730 Image Encoder 735 Model parameters (weights) 740 Text Description 750 images 760 Zero Shot Prediction 775 Semantic Feature Vectors 1200 computing devices 1202 processor 1204 memory 1206 Input / Output (I / O) Interface 1208 Operating Systems, Software Blocks 1210 Application, Software Block 1212 Application Data 1214 Audio / Video Input / Output Devices

Claims

1. determining geometric asset characteristics of the candidate 3D object based on a plurality of images of the candidate 3D object; determining a semantic feature vector of the candidate 3D object; determining a similarity between the candidate 3D object and the reference 3D object based on a comparison of the geometric asset features and the semantic feature vector of the candidate 3D object with the geometric asset features and semantic feature vector of a reference 3D object; classifying the candidate 3D object based on the similarity between the candidate 3D object and the reference 3D object; A computer-implemented method comprising:

2. determining a similarity between the candidate 3D object and the reference 3D object, calculating a first vector distance between the geometric asset features of the candidate 3D object and the geometric asset features of the reference 3D object; calculating a second vector distance between the semantic feature vector of the candidate 3D object and the semantic feature vector of the reference 3D object; generating a fused vector distance by combining the first vector distance and the second vector distance; 2. The computer-implemented method of claim 1, comprising:

3. classifying the candidate 3D objects, determining whether the fused vector distance meets a threshold for non-real objects; classifying the candidate 3D object as a non-real object if the fused vector distance meets the non-real object threshold; classifying the candidate 3D object as a real object if the fused vector distance does not meet the non-real object threshold; 3. The computer-implemented method of claim 2, comprising:

4. classifying the candidate 3D objects, determining whether the fused vector distances meet a similarity threshold; classifying the candidate 3D object as a similar object to the reference 3D object if the fused vector distance satisfies the similarity threshold; classifying the candidate 3D object as dissimilar to the reference 3D object if the fused vector distance does not satisfy the similarity threshold; 3. The computer-implemented method of claim 2, comprising:

5. 3. The computer-implemented method of claim 2, wherein combining the first vector distance and the second vector distance further comprises applying respective transformation functions to the first vector distance and the second vector distance.

6. constructing a fused vector distance matrix comprising respective fused vector distances between a plurality of 3D objects including the candidate 3D object; applying multidimensional scaling (MDS) to the fused vector distance matrix to determine a plurality of updated feature vectors for each of the plurality of 3D objects, each updated feature vector representing a fused vector distance of a corresponding 3D object to other 3D objects of the plurality of 3D objects, and a dimension of each updated feature vector being lower than a dimension of a corresponding geometric asset feature and a corresponding semantic feature vector; 3. The computer-implemented method of claim 2, further comprising:

7. determining geometric asset characteristics of the candidate 3D objects, determining one or more Histogram of Oriented Gradient (HOG) vectors for each image of the plurality of images of the candidate 3D object; calculating the geometric asset features of the candidate 3D object based on the one or more HOG vectors for each image of the plurality of images of the candidate 3D object; 2. The computer-implemented method of claim 1, comprising:

8. determining semantic feature vectors of the candidate 3D objects, obtaining one or more images of the candidate 3D object; analyzing the one or more images with a pre-trained machine learning model to obtain the semantic feature vector of the candidate 3D object, the machine learning model being trained by contrastive learning based on predicting matching pairs of images and associated text from a training dataset, the semantic feature vector being a high-dimensional vector, each dimension of the high-dimensional vector encoding a respective semantic information; 2. The computer-implemented method of claim 1, comprising:

9. classifying the candidate 3D objects includes determining uniqueness of the candidate 3D objects; determining the uniqueness of the candidate 3D objects by determining a corresponding fused vector distance between the candidate 3D object and each of a plurality of reference 3D objects; determining a plurality of neighboring 3D objects of the candidate 3D object; determining a local density of the candidate 3D object by calculating an average fused vector distance between the candidate 3D object and the plurality of neighboring 3D objects; determining a uniqueness score based on the local density of the candidate 3D object and a maximum local density of the plurality of reference 3D objects; 2. The computer-implemented method of claim 1, comprising:

10. prior to determining the similarity between the candidate 3D object and the reference 3D object, obtaining a first plurality of semantically similar reference 3D objects; obtaining a second plurality of geometrically similar reference 3D objects; forming a combined pool of geometrically similar and semantically similar reference 3D objects based on the first plurality of semantically similar reference 3D objects and the second plurality of geometrically similar reference 3D objects; selecting said reference 3D object from said combined pool of geometrically similar and semantically similar reference 3D objects; 10. The computer-implemented method of claim 1, further comprising:

11. 2. The computer-implemented method of claim 1, further comprising the step of generating the plurality of images of the candidate 3D object, each image of the plurality of images of the candidate 3D object being from a respective camera position of two or more camera positions.

12. generating a plurality of images of the candidate 3D object, each image of the plurality of images of the candidate 3D object from a respective one of two or more camera positions; determining geometric asset characteristics of the candidate 3D object based on the plurality of images of the candidate 3D object; determining respective distances between the geometric asset features of the candidate 3D object and geometric asset features of each of a plurality of reference 3D objects; determining a plurality of neighboring 3D objects of the candidate 3D object based at least in part on the respective distances; determining a local density of the candidate 3D object by calculating an average distance between the candidate 3D object and the plurality of neighboring 3D objects; determining a geometric uniqueness score based on the local density of the candidate 3D object and a maximum local density of the plurality of reference 3D objects; A computer-implemented method comprising:

13. 13. The computer-implemented method of claim 12, wherein determining a plurality of neighboring 3D objects of the candidate 3D object comprises applying an approximate nearest neighbor technique to the geometric asset features of the candidate 3D object.

14. The computer-implemented method of claim 12 , further comprising determining a price for the candidate 3D object based on the geometric uniqueness score.

15. The computer-implemented method of claim 14 , further comprising displaying the prices of the candidate 3D objects on a user interface.

16. The computer-implemented method of claim 12 , further comprising determining an asset monitoring index for the candidate 3D object based on the geometric uniqueness score.

17. 13. The computer-implemented method of claim 12, further comprising determining a market value index for the candidate 3D object based on the geometric uniqueness score.

18. A non-transitory computer-readable medium having instructions stored thereon, The instructions, upon execution by a processing device, cause the processing device to: determining geometric asset characteristics of the candidate 3D object based on a plurality of images of the candidate 3D object; determining a semantic feature vector of the candidate 3D object; determining a similarity between the candidate 3D object and the reference 3D object based on a comparison of the geometric asset features and the semantic feature vector of the candidate 3D object with the geometric asset features and semantic feature vector of a reference 3D object; classifying the candidate 3D object based on the similarity between the candidate 3D object and the reference 3D object; A non-transitory computer-readable medium for causing operations to be performed, including:

19. determining a similarity between the candidate 3D object and the reference 3D object, calculating a first vector distance between the geometric asset features of the candidate 3D object and the geometric asset features of the reference 3D object; calculating a second vector distance between the semantic feature vector of the candidate 3D object and the semantic feature vector of the reference 3D object; generating a fused vector distance by combining the first vector distance and the second vector distance; 20. The non-transitory computer-readable medium of claim 18, comprising:

20. classifying the candidate 3D objects, determining whether the fused vector distances meet a similarity threshold; classifying the candidate 3D object as a similar object to the reference 3D object if the fused vector distance satisfies the similarity threshold; classifying the candidate 3D object as dissimilar to the reference 3D object if the fused vector distance does not satisfy the similarity threshold; 20. The non-transitory computer-readable medium of claim 19, further comprising:

21. generating a plurality of images of the candidate 3D object, each image of the plurality of images of the candidate 3D object from a respective one of two or more camera positions; determining one or more Histogram of Oriented Gradient (HOG) vectors for each image of the plurality of images of the candidate 3D object; determining asset characteristics of the candidate 3D object based on the one or more HOG vectors for each image of the plurality of images of the candidate 3D object; determining whether the asset characteristics of the candidate 3D object match the real asset characteristics of at least one real 3D object; classifying the candidate 3D object as a non-authentic object if the asset features of the candidate 3D object match the authentic asset features of the at least one authentic 3D object; classifying the candidate 3D object as a real object if the asset features of the candidate 3D object do not match the real asset features of the at least one real 3D object; A computer-implemented method comprising:

22. 22. The computer-implemented method of claim 21, wherein determining whether the asset features of the candidate 3D object match the real asset features of at least one real 3D object includes calculating a vector distance between the asset features of the candidate 3D object and the real asset features of the at least one real 3D object, and if the vector distance meets a predetermined threshold, the asset features of the candidate 3D object are determined to match the real asset features of the at least one real 3D object.

23. 22. The computer-implemented method of claim 21, wherein determining whether the asset features of the candidate 3D object match the real asset features of at least one real 3D object includes performing a rotation-invariant comparison of the asset features of the candidate 3D object with the real asset features of the at least one real 3D object.

24. performing a rotation-invariant comparison of the asset features of the candidate 3D object with the real asset features of the at least one real 3D object, generating a plurality of rotated asset feature vectors of the candidate 3D object based on the asset features of the candidate 3D object, each rotated asset feature vector corresponding to a particular orientation of the candidate 3D object; comparing each of the plurality of rotated asset feature vectors with the real asset features of the at least one real 3D object; 24. The computer-implemented method of claim 23, comprising:

25. 22. The computer-implemented method of claim 21, wherein determining one or more HOG vectors includes determining one or more pyramidal HOG vectors, each of the one or more pyramidal HOG vectors generated by concatenating HOG vectors of the candidate 3D object generated at multiple resolutions for each image of the multiple images of the candidate 3D object.

26. 22. The computer-implemented method of claim 21, wherein generating a plurality of images of the candidate 3D object comprises generating the plurality of images at one or more azimuth and elevation points.

27. 22. The computer-implemented method of claim 21, wherein generating multiple images of a candidate 3D object includes adjusting a camera view during capture of the images so that the candidate 3D object occupies at least a predetermined area of ​​the images.

28. 22. The computer-implemented method of claim 21, further comprising replacing a texture of the candidate 3D object with a white plastic material prior to generating a plurality of images of the candidate 3D object.

29. 22. The computer-implemented method of claim 21, wherein classifying the candidate 3D object as a real object further comprises assigning a flag to the candidate 3D object, the flag being readable by a game engine and causing the game engine to enable use of the candidate 3D object in a virtual environment hosted by the game engine.

30. 22. The computer-implemented method of claim 21, wherein classifying the candidate 3D object as an inauthentic object further comprises assigning a flag to the candidate 3D object, the flag being readable by a game engine and causing the game engine to prevent use of the candidate 3D object in a virtual environment hosted by the game engine.

31. performing a principal component analysis of the asset features to reduce the dimensionality of the asset features; 22. The computer-implemented method of claim 21, wherein determining whether the asset features of the candidate 3D object match the real asset features of at least one real 3D object includes performing a comparison between the reduced dimensional asset features of the candidate 3D object and the reduced dimensional real asset features of the at least one real 3D object.

32. A non-transitory computer-readable medium having instructions stored thereon, The instructions, upon execution by a processing device, cause the processing device to: generating a plurality of images of the candidate 3D object, each image of the plurality of images of the candidate 3D object from a respective camera position of two or more camera positions; determining one or more Histogram of Oriented Gradient (HOG) vectors for each image of the plurality of images of the candidate 3D object; determining asset characteristics of the candidate 3D object based on the one or more HOG vectors for each image of the plurality of images of the candidate 3D object; determining whether the asset characteristics of the candidate 3D object match the real asset characteristics of at least one real 3D object; classifying the candidate 3D object as a non-authentic object if the asset features of the candidate 3D object match the authentic asset features of the at least one authentic 3D object; classifying the candidate 3D object as a real object if the asset features of the candidate 3D object do not match the real asset features of the at least one real 3D object; A non-transitory computer-readable medium for causing operations to be performed, including:

33. 33. The non-transitory computer-readable medium of claim 32, wherein determining whether the asset features of the candidate 3D object match the real asset features of at least one real 3D object includes calculating a vector distance between the asset features of the candidate 3D object and the real asset features of the at least one real 3D object, and if the vector distance meets a predetermined threshold, it is determined that the asset features of the candidate 3D object match the real asset features of the at least one real 3D object.

34. 33. The non-transitory computer-readable medium of claim 32, wherein determining one or more HOG vectors includes determining one or more pyramidal HOG vectors, each of the one or more pyramidal HOG vectors generated by concatenating HOG vectors of the candidate 3D object generated at multiple resolutions for each image of the multiple images of the candidate 3D object.

35. a memory storing instructions; a processing device coupled to the memory, the processing device configured to access the memory and execute the instructions; wherein the instructions cause the processing device to: generating a plurality of images of the candidate 3D object, each image of the plurality of images of the candidate 3D object from a respective camera position of two or more camera positions; determining one or more Histogram of Oriented Gradient (HOG) vectors for each image of the plurality of images of the candidate 3D object; determining asset characteristics of the candidate 3D object based on the one or more HOG vectors for each image of the plurality of images of the candidate 3D object; determining whether the asset characteristics of the candidate 3D object match the real asset characteristics of at least one real 3D object; classifying the candidate 3D object as a non-authentic object if the asset features of the candidate 3D object match the authentic asset features of the at least one authentic 3D object; classifying the candidate 3D object as a real object if the asset features of the candidate 3D object do not match the real asset features of the at least one real 3D object; A system that causes an operation including

36. 36. The system of claim 35, wherein determining whether the asset features of the candidate 3D object match the real asset features of at least one real 3D object includes calculating a vector distance between the asset features of the candidate 3D object and the real asset features of the at least one real 3D object, and if the vector distance meets a predetermined threshold, it is determined that the asset features of the candidate 3D object match the real asset features of the at least one real 3D object.

37. 36. The system of claim 35, wherein determining one or more HOG vectors includes determining one or more pyramid HOG vectors, each of the one or more pyramid HOG vectors generated by concatenating HOG vectors of the candidate 3D object generated at multiple resolutions of a respective image of the candidate 3D object.

38. 36. The system of claim 35, wherein determining whether the asset features of the candidate 3D object match the real asset features of at least one real 3D object comprises performing a rotation-invariant comparison between the asset features of the candidate 3D object and the real asset features of the at least one real 3D object.

39. 36. The system of claim 35, wherein the operations further include replacing a texture of the candidate 3D object with a white plastic material before generating the plurality of images of the candidate 3D object.

40. the operations further include reducing a dimensionality of the asset features; 36. The system of claim 35, wherein determining whether the asset features of the candidate 3D object match the real asset features of at least one real 3D object comprises performing a comparison between the reduced dimensional asset features of the candidate 3D object and the reduced dimensional real asset features of the at least one real 3D object.

Images