Automatic Detection of Prohibited Gaming Content

Abstract

Implementations described herein relate to methods, systems, and computer-readable media for automatically detecting prohibited content. For example, the method may include receiving data related to a game hosted on a gaming platform, the data indicating at least one developer, at least one user, and game content, automatically assigning a rating to the game using a machine learning model, the rating being based on the data indicating the at least one developer, the at least one user, and the game content, determining whether the rating of the game meets a safety threshold, and identifying the game as having prohibited content based on determining that the rating of the game does not meet the safety threshold.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of priority of U.S. Patent Application No. 17 / 172,866, filed on February 10, 2021, entitled "AUTOMATIC DETECTION OF PROHIBITED GAMING CONTENT", the entire content of which is incorporated herein by reference.

[0002] Embodiments generally relate to computer - based games, and more particularly, to methods, systems, and computer - readable media for automatically detecting and blocking prohibited gaming content.

Background Art

[0003] Some online gaming platforms enable users to access content such as games and game assets, connect with each other, communicate with each other (e.g., within a game), and share information with each other via the Internet. Users of online gaming platforms may participate in a multiplayer gaming environment where a game or part of a game is offered for communication and / or purchase.

[0004] When a user searches for game - related content (e.g., games, items for games, accessories for avatars, etc.) or other content, results based on factors such as popularity, represented by the number of downloads by the user, ratings by the user, etc., may be presented to the user. However, new or recently developed content that has not been rated may be presented. In these situations, inappropriate or prohibited content may be more easily accessed by users. Furthermore, as the number of users and developers increases, the amount of new, unrated content also increases.

[0005] The background description provided herein is for the purpose of presenting the context of the present disclosure. Neither the achievements of the inventors named herein, to the extent that they are within the scope described in this background section, nor aspects of the description that may not otherwise be eligible as prior art at the time of filing are admitted, either explicitly or implicitly, as prior art for the present disclosure.

Summary of the Invention

Means for Solving the Problems

[0006] Embodiments of the present application relate to automatically detecting prohibited content and preventing it from reaching a user. In some embodiments, a computer-implemented method includes receiving data related to a game hosted on a gaming platform, the data indicating at least one developer, at least one user, and game content; automatically assigning a rating to the game using a machine learning model, the rating being based on data indicating at least one developer, at least one user, and game content; determining whether the rating of the game meets a safety threshold; and identifying the game as having prohibited content based on a determination that the rating of the game does not meet the safety threshold.

[0007] In some embodiments, the data indicating at least one developer comprises one or more of the developer's operating period on the gaming platform, the number of games published by the developer on the gaming platform, user behavior data related to the developer on the gaming platform, the developer's moderation history on the gaming platform, or the average rating of games related to the developer on the gaming platform.

[0008] In some implementations, the data indicating at least one user comprises one or more of the user's attributes, the user's game play history, the behavioral data of other users associated with the user, the user's moderation history, the user's chat history, the user's transaction history, or the average rating of the games played by the user.

[0009] In some implementations, the data indicating game content comprises one or more of the textual description of the game, the game title, the game assets, the models that make up the game, the in-game actions, or the average rating of one or more other games having content within a threshold range of similarity to the game content.

[0010] In some implementations, the method further comprises restricting access to game content via a gaming platform based on a determination that the game content includes prohibited content.

[0011] According to another aspect, a computer-implemented method is provided. The computer-implemented method comprises receiving data related to a game hosted on a gaming platform, the data indicating at least one developer, at least one user, and game content; determining input features of a machine learning model based on the data indicating at least one developer, at least one user, and game content; providing the input features to the machine learning model; and obtaining an output of the machine learning model based on the input features, the output including a rating of the game.

[0012] In some implementations, the data indicating at least one developer includes one or more of the developer's operating period on the gaming platform, the number of games published by the developer on the gaming platform, user behavior data related to the developer on the gaming platform, the moderation history of the developer on the gaming platform, or the average rating of games related to the developer on the gaming platform.

[0013] In some implementations, the data indicating at least one user includes one or more of the user's attributes, the user's game play history, the behavior data of other users related to the user, the moderation history of the user, the chat history of the user, the transaction history of the user, or the average rating of games played by the user.

[0014] In some implementations, the data indicating game content includes one or more of the textual description of the game, the title of the game, the assets of the game, the models that make up the game, the actions within the game, or the average rating of one or more other games having content within the range of the similarity threshold of the game content.

[0015] In some implementations, the rating of a game is a value indicating the likelihood that the game contains prohibited content.

[0016] According to another aspect, a system is provided. The system includes a memory storing instructions and a processing device coupled to the memory. The processing device is configured to access the memory and execute the instructions. The instructions cause the processing device to receive data related to a game, where the data indicates at least one developer, at least one user, and game content; automatically assign a rating to the game using a machine learning model, where the rating is based on data indicating at least one developer, at least one user, and game content; determine whether the rating of the game meets a safety threshold; and identify the game as having prohibited content based on a determination that the rating of the game does not meet the safety threshold.

[0017] In some implementations, the data indicating at least one developer includes one or more of the developer's operating period on the gaming platform, the number of games published by the developer on the gaming platform, user behavior data related to the developer on the gaming platform, the developer's moderation history on the gaming platform, or the average rating of games related to the developer on the gaming platform.

[0018] In some implementations, the data indicating at least one user includes one or more of the user's attributes, the user's game play history, other user behavior data related to the user, the user's moderation history, the user's chat history, the user's transaction history, and the average rating of games played by the user.

[0019] In some implementations, the data indicating game content includes one or more of a textual description of the game, the game title, game assets, models that make up the game, in-game actions, or the average rating of one or more other games having content within a range of similarity thresholds of the game content.

[0020] In some implementations, the operation further includes restricting access to game content via a gaming platform based on a determination that the game content includes prohibited content.

[0021] According to another aspect, a non-transitory computer-readable medium is provided. Instructions are stored on the non-transitory computer-readable medium that, in response to execution by a processing device, cause the processing device to receive data related to a game, where the data indicates at least one developer, at least one user, and game content, determine input features of a machine learning model based on the at least one developer, at least one user, and the data indicating game content, provide the input features to the machine learning model, and obtain an output of the machine learning model based on the input features, where the output includes a rating of the game.

[0022] In some implementations, the data indicating at least one developer includes one or more of a developer's operating period on a gaming platform, the number of games published by the developer on the gaming platform, user behavior data related to the developer on the gaming platform, a moderation history of the developer on the gaming platform, or the average rating of games related to the developer on the gaming platform.

[0023] In some implementations, the data indicating at least one user comprises one or more of the user's attributes, the user's game play history, the behavior data of other users related to the user, the user's moderation history, the user's chat history, the user's transaction history, and the average rating of the games played by the user.

[0024] In some implementations, the data indicating game content comprises one or more of the textual description of the game, the game title, the game assets, the models constituting the game, the actions within the game, or the average rating of one or more other games having content within the range of the similarity threshold of the game content.

[0025] In some implementations, the game rating is a value indicating the likelihood that the game contains prohibited content.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

[0027] One or more implementations described herein relate to the detection and prevention of prohibited items related to an online gaming platform. Features may include automatically rating games and items based on developer data, game data, user profile data, behavioral parameters, and / or other factors.

[0028] The features described herein provide for the automatic detection and prevention of prohibited content through the intelligent rating of games and related game content, and the training of machine - learning models specialized in identifying prohibited content. For example, user profile data, user behavioral parameters, user game play data, developer profile data, developer behavioral data, and game data can be used to rate a game as containing potentially prohibited content. Further, the rating can be enhanced through feedback and management functions so that newly created content containing potentially prohibited content is quickly identified before it surfaces to the user.

[0029] A game rating can be generated such that only games that meet a threshold safety rating are presented to the user through the training of a machine - learning model based on user interactions and other data. The machine - learning model may be retrained periodically, thereby improving the rating of newly created content.

[0030] In addition, internal and external data sources such as social networks, external links to online gaming platforms, and other external data can be used to train a machine learning model to rate games according to game content, user activities, user behavior, groups of users previously associated with prohibited content, groups of developers previously associated with prohibited content, and other similar criteria.

[0031] As game content is created, prohibited content can be more easily restricted, obfuscated, and / or deleted by rating the game content based on multiple data signals. Accordingly, the disclosed features provide technical advantages including a reduction in the need to manually search for potentially prohibited game data, as well as a reduction in the time spent interpreting false positives of prohibited content, thereby leading to a reduction in the computing resources (e.g., server computer memory, processors, networking traffic, etc.) used to manage the online gaming platform. For example, when a piece of content is created, a machine learning model can quickly rate it based on external data, profile data, and behavioral data as to its likelihood of including prohibited content. Accordingly, there is no need for additional search calculations to manually examine new items, thereby saving computing resources when monitoring for prohibited content.

[0032] An online gaming platform (also referred to as a "user-generated content platform" or "user-generated content system") provides various ways for users to communicate with each other. For example, users of an online gaming platform can create games, or other content or resources (such as characters, graphics, items, etc. for gameplay within a virtual world) within the gaming platform.

[0033] Users of an online gaming platform may cooperate towards common goals in games or game creation, share various virtual gaming items, and send electronic messages to each other. Users of an online gaming platform may, for example, play games including characters (avatars) or other game objects and mechanisms. The online gaming platform may also enable users of the platform to communicate with each other. For example, users of an online gaming platform may communicate with each other using voice messages (e.g., via voice chat), text messaging, video messaging, or a combination of the above. Some online gaming platforms can provide a virtual three-dimensional environment in which users can play online games.

[0034] To help enhance the entertainment value of the online gaming platform, the platform can provide a search engine for games, game content, or other game-related resources, and users of the online gaming platform can use the search engine to search for game-related content or such other content. Since games and game resources are frequently created, modified, and added to the available game data by various users (including game creators / developers and / or game players), the search engine realizes an improvement in the discoverability of the available games and resources.

[0035] For example, a user can input a search query indicating a request for an item or resource sought by the user into the search engine. The search engine searches a set of game data and provides as output a search result that may include a list of items and / or game-related resources that are determined by the search engine to match the search query and ranked by a search algorithm.

[0036] However, in some situations, the search results provided by such search engines may include prohibited games and / or items. For example, games and / or items may be prohibited based on the user's age, the user's location, content usage license conditions, content type, user device, etc. The search engine may provide a number of search results that may include games or items that may contain inappropriate content in some cases. In some online gaming platforms, since users can create a variety of games, game types, resources, etc., a number of prohibited contents may surface. Furthermore, considering the large number of contents generated by users on the online gaming platform, it may be difficult to monitor all search results for prohibited contents.

[0037] In addition, an online gaming platform may have a large number of users, for example, millions of users, and those users can use external search functions outside the online gaming platform to find content that should otherwise be prohibited. Therefore, even if the search engine within the online gaming platform is configured to obfuscate known prohibited content, external resources may indicate such prohibited content before the prohibited content is identified through the online gaming platform.

[0038] Figures 1 and 2: System Architecture Figure 1 shows an exemplary network environment 100 according to some implementations of the present disclosure. The network environment 100 (also referred to herein as a "system") includes an online gaming platform 102, a first client device 110, a second client device 116 (collectively referred to herein as "client devices 110 / 116"), and a network 122. The online gaming platform 102 can include, among other things, a game engine 104, one or more games 105, a search engine 106, a game rating engine 107, and a data storage device 108. The client device 110 can include a game application 112. The client device 116 can include a game application 118. Users 114 and 120 can each use the client devices 110 and 116 to interact with the online gaming platform 102.

[0039] For purposes of illustration, a network environment 100 is provided. In some implementations, the network environment 100 can include the same, fewer, more, or different elements configured in the same or a different manner than shown in FIG. 1.

[0040] In some implementations, the network 122 can include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.11 network, a Wi-Fi® network, or a wireless local area network (WLAN)), a cellular network (e.g., a Long Term Evolution (LTE) network), a router, a hub, a switch, a server computer, or a combination thereof.

[0041] In some implementations, data storage device 108 can be a non-transitory computer-readable memory (e.g., random access memory), cache, drive (e.g., hard drive), flash drive, database system, or another type of component or device capable of storing data. Data storage device 108 can also include multiple storage components (e.g., multiple drives or multiple databases) that can span multiple computing devices (e.g., multiple server computers).

[0042] In some implementations, online gaming platform 102 can include a server having one or more computing devices (e.g., cloud computing system, rack-mounted server, server computer, cluster of physical servers, virtual server, etc.). In some implementations, the server can be included in online gaming platform 102, can be an independent system, or can be part of another system or platform.

[0043] In some implementations, the online gaming platform 102 may include one or more computing devices (such as rack-mounted servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, etc.), data storage devices (such as hard disks, memories, databases), networks, software components, and / or hardware components that can execute operations on the online gaming platform 102 and provide users with access to the online gaming platform 102. The online gaming platform 102 may also include a website (such as one or more web pages) or application backend software that can be used to provide users with access to the content provided by the online gaming platform 102. For example, a user can access the online gaming platform 102 using the game applications 112 / 118 on the client devices 110 / 116, respectively.

[0044] In some implementations, the online gaming platform 102 may include a social network of sorts that provides connections between users, or a user-generated content system of sorts that enables users (e.g., end users or consumers) to communicate with other users via the online gaming platform 102, and the communication may include voice chat (e.g., synchronous and / or asynchronous voice communication), video chat (e.g., synchronous and / or asynchronous video communication), or text chat (e.g., synchronous and / or asynchronous text-based communication). In some implementations of the present disclosure, a "user" may be represented as a single individual. However, other implementations of the present disclosure include the case where a "user" (e.g., a creator user) is an entity controlled by a set of users or an automated source. For example, a set of individual users associated as a community or group in a user-generated content system may be regarded as "users".

[0045] In some implementations, the online gaming platform 102 may be a virtual gaming platform. For example, the gaming platform may provide single-player or multiplayer games to a community of users who can access or interact with games (e.g., games generated by users or other games) using client devices 110 / 116 via network 122. In some implementations, a game (also referred to herein as a "video game", "online game", or "virtual game") may be, for example, a 2D game, a 3D game (e.g., a 3D game generated by a user), a virtual reality (VR) game, or an augmented reality (AR) game. In some implementations, users may search for games and game items and participate in gameplay with other users in one or more games. In some implementations, games may be played in real time with other users of the game.

[0046] In some implementations, other collaboration platforms may be used with, instead of, or in addition to, the rating functions described herein, in place of online gaming platform 102 and / or search engine 106. For example, a social networking platform, a purchasing platform, a messaging platform, a creation platform, etc. may be used with the rating functions so that prohibited content does not surface to users.

[0047] In some implementations, gameplay may refer to the interaction of one or more players using client devices (e.g., 110 and / or 116) within a game (e.g., 105), or the presentation of interactions on the display of client device 110 or 116 or other output devices.

[0048] One or more games 105 are provided by an online gaming platform. In some implementations, game 105 may include an electronic file that can be executed or loaded using software, firmware, or hardware configured to present game content (e.g., digital media items) to an entity. In some implementations, game application 112 / 118 may be executed and game 105 is rendered with respect to game engine 104. In some implementations, game 105 may have a common set of rules or a common goal, and the environment of game 105 shares that common set of rules or common goal. In some implementations, different games may have different rules or goals from each other.

[0049] In some implementations, the game may have one or more environments (also referred to herein as "gaming environments" or "virtual environments"), and the multiple environments may be connected. Examples of environments can be three-dimensional (3D) environments. One or more environments of game 105 may be collectively referred to herein as a "world" or "gaming world" or "virtual world" or "universe". An example of a world can be a 3D world of game 105. For example, a user may construct a virtual environment that is connected to another virtual environment created by another user. A character in a virtual game may enter a neighboring virtual environment across a virtual boundary.

[0050] It can be noted that 3D environments or 3D worlds use graphics that use a three-dimensional representation of geometric data representing game content (or at least present game content to appear as 3D content, regardless of whether a 3D representation of the geometric data is used). 2D environments or 2D worlds use graphics that use a two-dimensional representation of geometric data representing game content.

[0051] In some implementations, the online gaming platform 102 can host one or more games 105 and enable a user to interact with the game 105 (e.g., search for games, game-related content, or other content) using the game applications 112 / 118 on the client devices 110 / 116. Users (e.g., 114 and / or 120) of the online gaming platform 102 can play, create, interact with, or build the game 105, search for the game 105, communicate with other users, create and build objects of the game 105 (e.g., also referred to herein as "items" or "game objects" or "virtual game items"), and / or search for objects. For example, when generating virtual items generated by a user, the user can in particular create characters, decorations for the characters, one or more virtual environments for two-way games, or build structures used in the game 105.

[0052] In some implementations, a user can purchase, sell, or trade game virtual game objects such as in-platform currency (e.g., virtual currency) with other users of the online gaming platform 102. In some implementations, the online gaming platform 102 can send game content to a game application (e.g., 112). In some implementations, game content (also referred to herein as "content") can refer to any data or software instructions related to the online gaming platform 102 or the game application (e.g., game objects, games, user information, videos, images, commands, media items, etc.).

[0053] In some implementations, a game object (also referred to herein as an "item" or "object" or "virtual game item") can refer to an object that is used, created, shared, or otherwise depicted in a game application 105 of an online gaming platform 102 or a game application 112 or 118 of a client device 110 / 116. For example, game objects can include parts, models, characters, props, weapons, clothing, buildings, vehicles, currency, plants, animals, components of the foregoing (e.g., windows of a building), and the like.

[0054] It can be noted that the online gaming platform 102 hosting the game 105 is provided for purposes of illustration and not limitation. In some implementations, the online gaming platform 102 can host one or more media items that can include communication messages from one user to one or more other users. Media items can include, but are not limited to, digital videos, digital movies, digital photos, digital music, audio content, melodies, website content, social media updates, e-books, e-magazines, e-newspapers, digital audiobooks, e-journals, web blogs, real simple syndication (RSS) feeds, digital comics, software applications, and the like. In some implementations, a media item can be an electronic file that can be executed or loaded using software, firmware, or hardware configured to present the digital media item to an entity.

[0055] In some implementations, game 105 may be associated with a particular user or group of users (e.g., a private game), or may be made widely available to users of online gaming platform 102 (e.g., a public game). In some implementations where online gaming platform 102 associates one or more games 105 with a particular user or group of users, online gaming platform 102 may use user account information (e.g., user account identifiers such as a username and password) to associate game 105 with a particular user. Similarly, in some implementations, online gaming platform 102 may use developer account information (e.g., developer account identifiers such as a username and password) to associate a particular developer or group of developers with game 105.

[0056] In some implementations, the online gaming platform 102 or the client device 110 / 116 may include a game engine 104 or a game application 112 / 118. The game engine 104 may include game applications similar to the game applications 112 / 118. In some implementations, the game engine 104 may be used for the development or execution of the game 105. For example, the game engine 104 may include, among other things, a rendering engine (a "renderer") for 2D, 3D, VR, or AR graphics, a physics engine, a collision detection engine (and collision response), an audio engine, a scripting function, an animation engine, an artificial intelligence engine, a networking function, a streaming function, a memory management function, a threading function, a scene graph function, or video support for cinematics. The components of the game engine 104 may generate commands (such as rendering commands, collision commands, physics commands, etc.) that assist in calculating and rendering the game. In some implementations, the game applications 112 / 118 of the client devices 110 / 116 may each independently operate in cooperation with the game engine 104 of the online gaming platform 102 or in a combination of both.

[0057] In some implementations, both the online gaming platform 102 and the client devices 110 / 116 execute a game engine (104, 112, and 118 respectively). The online gaming platform 102 that uses the game engine 104 may execute some or all of the game engine functions (such as generating physics commands, rendering commands, etc.) or may offload some or all of the game engine functions to the game engine 104 of the client device 110. In some implementations, each game 105 may have a different ratio between the game engine functions executed on the online gaming platform 102 and the game engine functions executed on the client devices 110 and 116.

[0058] For example, the game engine 104 of the online gaming platform 102 may be used to generate physics commands when there is a collision between at least two game objects, but additional game engine functions (e.g., rendering generation commands) may be offloaded to the client device 110. In some implementations, the ratio of game engine functions executed on the online gaming platform 102 to those executed on the client device 110 may be changed (e.g., dynamically) based on game play conditions. For example, if the number of users participating in the game play of game 105 exceeds a threshold number, the online gaming platform 102 may execute one or more game engine functions that were previously executed by the client device 110 or 116.

[0059] For example, a user may play game 105 on client devices 110 and 116 and may send control instructions (e.g., user input such as right, left, up, down, user selection, or information on the position and speed of a character) to online gaming platform 102. After receiving the control instructions from client devices 110 and 116, online gaming platform 102 may send game play instructions (e.g., information on the position and speed of characters participating in group game play, or commands such as rendering commands, collision commands) to client devices 110 and 116 based on the control instructions. For example, online gaming platform 102 may perform one or more logical operations (e.g., using game engine 104) on the control instructions and generate game play instructions for client devices 110 and 116. In other cases, online gaming platform 102 may pass one or more of the control instructions from a certain client device 110 to other client devices (e.g., 116) participating in game 105. Client devices 110 and 116 may use the game play instructions and render the game play for presentation on the displays of client devices 110 and 116.

[0060] In some implementations, the control instructions may refer to instructions indicating in-game actions of the user's character. For example, the control instructions may include user inputs for controlling in-game actions such as right, left, up, down, user selection, gyroscope position and orientation data, force sensor data, etc. The control instructions may include the position and velocity information of the character. In some implementations, the control instructions are directly sent to the online gaming platform 102. In other implementations, the control instructions may be sent from the client device 110 to another client device (e.g., 116), and that other client device generates game play instructions using the local game engine 104. The control instructions may include instructions for playing an audio communication message or other sound from another user on an audio device (e.g., speaker, headphones, etc.).

[0061] In some implementations, the game play instructions may refer to instructions that enable the client device 110 (or 116) to render the game play of a game such as a multiplayer game. The game play instructions may include one or more of user inputs (e.g., control instructions), the position and velocity information of the character, or commands (e.g., physical commands, rendering commands, collision commands, etc.). As described in more detail herein, the game play instructions issued by the user may affect the rating or evaluation. This rating may be a numerical representation of the level of trust that the associated user is not operating the character inappropriately (e.g., not performing prohibited or adult-oriented interactions with the character). The associated rating may be stored in the data storage device 108 by the online gaming platform 102.

[0062] In some implementations, a character (or generally a game object) is constructed from components that are automatically combined to assist the user during editing, and one or more of these components may be selected by the user. One or more characters (also referred to herein as "avatars" or "models") may be associated with the user, and the user may control the character to facilitate the user's interaction with the game 105. In some implementations, a character may include components such as body parts (e.g., hair, arms, legs, etc.) and accessories (e.g., T-shirts, glasses, decorative images, props, etc.). In some implementations, customizable body parts of a character may include, among other things, head types, body part types (arms, legs, torso, and hands), face types, hair types, and skin types. In some implementations, customizable accessories may include clothing (e.g., shirts, pants, hats, shoes, glasses, etc.), weapons, or other props.

[0063] In some implementations, the user may also adjust the dimensions of the character (e.g., height, width, or depth) or the dimensions of the components of the character. In some implementations, the user may adjust the proportions of the character (e.g., stocky, anatomical, etc.). It can be noted that in some implementations, a character may not include character game objects (e.g., body parts, etc.), but the user may still adjust the character (without character game objects) to facilitate the user's interaction with the game (e.g., a puzzle game where there is no rendered character game object, but the user controls a character to control actions within the game).

[0064] In some implementations, components such as parts of the body may be in basic geometric shapes such as blocks, cylinders, spheres, or any other basic shape such as wedges, toruses, tubes, grooves, etc. In some implementations, the authoring module may issue the user's character for viewing or use by other users of the online gaming platform 102. In some implementations, creating, modifying, or customizing a character, other game object, game 105, or game environment may be performed by the user using or without using scripting (or with or without using an application programming interface (API)) using a user interface (e.g., a developer interface). It can be noted that, for purposes of illustration and not limitation, a character is described as having a humanoid form. It can further be noted that the character can have any form, such as a vehicle, an animal, an inanimate object, or other creative forms.

[0065] In some implementations, the online gaming platform 102 may store characters created by users in the data storage device 108. In some implementations, the online gaming platform 102 maintains a character catalog and a game catalog that may be presented to users via the search engine 106, the game engine 104, the game 105, and / or the client devices 110 / 116. In some implementations, the game catalog includes images of games stored in the online gaming platform 102. Additionally, a user may select a character (e.g., a character created by the user or another user) from the character catalog and participate in the selected game. The character catalog includes images of characters stored in the online gaming platform 102. In some implementations, one or more of the characters in the character catalog may be created or customized by the user. In some implementations, the selected character may have a character setting that defines one or more of the components of the character.

[0066] In some implementations, the user's character can include a component configuration, and the component configuration and appearance, and more generally the character's overview, may be defined by a character setting. In some implementations, the character setting of the user's character may be at least partially selected by the user. In other implementations, the user may select a character with a default character setting or a character setting selected by another user. For example, the user may select a default character with a predefined character setting from the character catalog, and the user may further customize the default character by changing a part of the character setting (e.g., adding a customized logo to the shirt). The character setting may be associated with a specific character by the online gaming platform 102.

[0067] As described in more detail herein, each part of a game, including characters, character appearances, avatars, clothing, components, items, and other parts, may include a rating or evaluation. This rating may be a numerical representation of the level of trust that the associated part does not contain prohibited content. The associated rating may be stored in the data storage device 108 by the online gaming platform 102.

[0068] In some implementations, each of the client devices 110 or 116 may include a computing device such as a personal computer (PC), a mobile device (e.g., a laptop, a cellular phone, a smart phone, a tablet computer, or a netbook computer), a network-connected television, a game console, etc. In some implementations, the client devices 110 or 116 may also be referred to as "user devices". In some implementations, one or more client devices 110 or 116 may be connected to the online gaming platform 102 at any given moment. It should be noted that the number of client devices 110 or 116 is given by way of example and not limitation. In some implementations, any number of client devices 110 or 116 may be used.

[0069] In some implementations, each client device 110 or 116 may respectively include instances of game applications 112 or 118. In one implementation, game applications 112 or 118 enable a user to interact with them using online gaming platform 102, such as searching for games or other content, controlling virtual characters in virtual games hosted by online gaming platform 102, or browsing or uploading content such as games 105, images, video items, web pages, documents, etc. In one example, the game application can be a web application (e.g., an application that operates in conjunction with a web browser) that can access, retrieve, present, or manipulate content (e.g., virtual characters in a virtual environment, etc.) served by a web server. In another example, the game application can be a native application (e.g., a mobile application, app, or gaming program) installed on client device 110 or 116 and executed locally, enabling the user to interact with online gaming platform 102. The game application can render, display, or present content (e.g., web pages, user interfaces, media viewers) to the user. In certain implementations, the game application may also include an embedded media player (e.g., a Flash® player) embedded in a web page.

[0070] According to aspects of the present disclosure, the game application 112 / 118 can be an online gaming platform application for a user to build, create, edit content, upload it to the online gaming platform 102, and interact with the online gaming platform 102 (e.g., play a game 105 hosted by the online gaming platform 102). Thus, the game application 112 / 118 can be provided to the client device 110 or 116 by the online gaming platform 102. In another example, the game application 112 / 118 can be an application downloaded from a server.

[0071] In some implementations, the user can log in to the online gaming platform 102 via the game application. The user may access the user account by providing user account information (e.g., username and password), and the user account is associated with one or more characters available for participating in one or more games 105 of the online gaming platform 102.

[0072] Generally, functions described as being performed by the online gaming platform 102 can also be performed by the client device 110 or 116, or the server, if appropriate in other implementations. Additionally, functions resulting from a particular component can be performed by different or multiple components operating together. Since the online gaming platform 102 can also be accessed as a service provided to other systems or devices through an appropriate application programming interface (API), it is not limited to use on a website.

[0073] In some implementations, the online gaming platform 102 may include a search engine 106. In some implementations, the search engine 106 may be a system, application, or module that enables the online gaming platform 102 to provide a search function to users, and the search function enables users to search for games, game-related content, items, lists of items, or other content within the online gaming platform 102.

[0074] In some implementations, the online gaming platform 102 may include a game rating engine 107. In some implementations, the game rating engine 107 may be a system, application, or module that determines ratings for games or related gaming content, users, and / or developers. The rating may be a numerical measure of the likelihood that the relevant component includes prohibited content (or, in the case of a user or developer, is likely to generate prohibited content). The rating may be determined through a machine learning model (supervised or unsupervised) that takes feature quantities (or feature vectors) as input and provides an output value indicating the likelihood that the input feature quantities are associated with prohibited content.

[0075] As used herein, the term "prohibited content" generally refers to any content that is considered inappropriate for an online gaming platform. For example, prohibited content may include adult-oriented content, extremist content, and content related to any dangerous, illegal, and / or unethical activities. By way of example, adult-oriented content may include, but is not limited to, adult-oriented conversations between avatars / characters, adult-oriented clothing, adult-oriented accessories, adult-oriented language, and / or other inappropriate content. Other prohibited content may include game sessions or scenarios with a modified physics engine that allows adult-oriented conversations between characters and / or objects. Still further, other prohibited content may include euphemistic language and other avoidance techniques utilized by users attempting to promote prohibited content.

[0076] Generally, search engine 106 may utilize a filter configured to determine whether the rating value of any searched content meets a safety threshold (e.g., is unlikely to include prohibited content) before providing results. Additionally, online gaming platform 102 may also provide recommendations to some users, for example, via a home screen, interface, or other similar feature. In these scenarios, online gaming platform 102 may also utilize a method and / or filter configured to determine whether the rating value of any content being promoted meets a safety threshold before providing the recommendation. Still further, online gaming platform 102 may also utilize the rating value when indexing content such that the content is evaluated by the rating value to determine whether any accessed content meets a safety threshold.

[0077] However, some prohibited content may be linked outside the online gaming platform 102, for example, through links from external data sources. Thus, although the search engine 106 is described herein as functioning to surface content queried by a user, the ratings generated by the game rating engine 107 may also be used to restrict access to any link to any content that may be considered inappropriate or prohibited such that external access can be reduced or restricted.

[0078] Accordingly, the rating values generated by the game rating engine 107 may also be used to manage the online gaming platform 102 in an automated manner such that games, content, or other media with ratings below a threshold can be isolated or held for further investigation before access is permitted on the client device 110 / 116 or through any external link. The operation of the rating engine 107 is more fully described below with respect to FIG. 2.

[0079] FIG. 2 shows an exemplary network environment 200 according to some implementations of the present disclosure. The network environment 200 (also referred to herein as a “system”) includes an online gaming platform 102, a user data source 202, a developer data source 204, a game data source 206, and a network 122. The online gaming platform 102 may include, among other things, a game engine 104, one or more games 105, a search engine 106, a game rating engine 107, and a data storage device 108.

[0080] As briefly described above, user account information (e.g., user account identifiers such as a username and password) may be maintained by the online gaming platform 102. The user account information may enable the online gaming platform to identify a user data source 202 to be used by the user using appropriate user permissions granted by the user. In this way, user data 221 may be retrieved from publicly available external data sources as well as from the online gaming platform 102 itself. Publicly available external data sources may include, but are not limited to, social networking websites, news forums, newsgroups, online bulletin boards, or other web pages where users of the online gaming platform 102 provide links to content provided by the online gaming platform 102. User data 221 may also include any available user data regarding the use of the online gaming platform 102, including previous use of prohibited content, gatherings with other users of prohibited content, and previous communications with the developers of prohibited content.

[0081] For each user, the data to be stored can be based on user-specific permissions provided by the user. The user is given options to choose the specific data that can be stored, and the parameters related to the storage of such data (e.g., the period for which the data can be stored), and how the data can be used (e.g., aggregated, processed, or otherwise analyzed). Further, the storage of user data is carried out in accordance with applicable regulations (e.g., in the user's region). Some data can be processed so that personally identifiable information (PII) of the user is not stored or cannot be retrieved from the stored data. Access to data for various programs of the platform is restricted such that only specific programs that perform specific actions permitted by the user can access the data. For example, the data can be aggregated (e.g., across multiple user accounts) or otherwise processed before being provided to a program.

[0082] User data is not accessed without explicit permission from the user. The user is provided with information about how the user data can be stored and / or utilized, about features / functions enabled based on the user data, and options to restrict access to the user data (e.g., no access to some parts of the data, restricted access, etc.). The user can change the permission for data access at any time, view the user data that can be accessed, and / or delete a part of the user data.

[0083] Developer account information (e.g., developer account identifiers such as a username and password) may also be maintained by the online gaming platform 102. The developer account information may enable the online gaming platform 102 to identify developer data sources 204 used by the developer using appropriate user permissions granted by the developer. In this way, developer data 241 may be retrieved from publicly available external data sources as well as from the online gaming platform 102 itself. Publicly available external data sources may include, but are not limited to, social networking websites, news forums, newsgroups, online bulletin boards, or other web pages where developers of game content for the online gaming platform 102 provide links to content provided by the online gaming platform 102. Developer data 241 may also include any available data regarding users of the online gaming platform 102, including previous development of prohibited content, facilitation of user actions to avoid rules / procedures for avoiding prohibited content, and previous interactions with users of prohibited content.

[0084] For each developer, the data stored may be based on developer-specific permissions provided by the developer. The developer is given options to choose the specific data that may be stored, and the parameters related to the storage of such data (e.g., the period for which the data may be stored), and how the data may be used (e.g., aggregated, processed, or otherwise analyzed). Further, the storage of developer data is carried out in accordance with applicable regulations (e.g., in the developer's region). Some data may be processed so that personally identifiable information (PII) of the developer is not stored or cannot be retrieved from the stored data. Access to data for various programs of the platform is restricted such that only specific programs that perform actions permitted by a specific developer can access the data. For example, the data may be aggregated (e.g., across multiple developer accounts) or otherwise processed before being provided to a program.

[0085] Developer data is not accessed without explicit permission from the developer. The developer is provided with information about how developer data may be stored and / or utilized, about features / functions enabled based on developer data, and options to restrict access to developer data (e.g., no access to some portions of the data, restricted access, etc.). The developer can change the permission for data access at any time, view the developer data that may be accessed, and / or delete some of the developer data.

[0086] Game data 261 can also be retrieved from game data sources 206, such as a new website, forum, rating website, and other publicly available data sources and / or external game data sources. Game data 261 can also include data retrieved from within the online gaming platform 102 itself. Data source 206 can include an evaluation or textual description of potentially prohibited content, an euphemistic representation of prohibited content (e.g., Condo game, HOA, etc.), and other similar data. Additionally, data source 206 can include links (e.g., hyperlinks) to access potentially prohibited game content.

[0087] Game data is not accessed without explicit permission from the developer. The developer is provided with information on how the game data can be stored and / or used, features / functions enabled based on the game data, and options to restrict access to the game data (e.g., no access to some portions of the data, restricted access, etc.). The developer can change the permission for data access at any time, view the game data that can be accessed, and / or delete a portion of the game data.

[0088] In some implementations, the game rating engine 107 may provide one or more of user data 221, developer data 241, and game data 261 as inputs to a machine learning model configured to output a value, such as a numerical value or other value, indicating the likelihood that the user data 221, developer data 241, and game data 261 are related to prohibited content. For example, user data 221 may be used to determine the frequency with which a particular user accesses prohibited content. Developer data 241 and game data 261 may also reflect the number of users who frequently access, aggregate, or access a particular game, and who access prohibited content. In this example, the input of this data may cause the machine learning model to output the likelihood that the accessed content includes prohibited content. In additional examples, the machine learning model may determine that a large number of relatively new user accounts frequently access prohibited content. Thus, user data 221 reflecting the status of new users and accessing the same game, as shown in game data 261, may also cause an output of the likelihood that the game includes prohibited content. Further, user data 221 reflecting the history of moderation or abnormal behavior on the online gaming platform 102 (e.g., relatively old and inactive accounts that have revived after a relatively long period of time) may also cause an output of the likelihood that the accessed game includes prohibited content. These and other examples are non-exhaustive and are for illustrative purposes only, as the machine learning model may further identify other trends or relationships related to prohibited content.

[0089] Accordingly, the output may be used to generate a rating 271. Although described as a general rating, the rating 271 may be a "user rating", "game rating", and / or "developer rating" depending on the data being rated such that access is prohibited for some users or developers due to a history of using / creating prohibited content.

[0090] Once generated, the rating 271 can be used to restrict access to the content (including access from external access through links). Additionally, any restricted or obfuscated content may be edited by the user or developer to remove any prohibited content. In this example, a new rating can be generated after the editing. If the new rating exceeds the threshold, the associated content will surface normally and can be accessed by the user. In some implementations, the game rating engine 107 can execute one or more of the operations described below in connection with the flowcharts shown in FIGS. 6 and 7, using the features shown in FIGS. 3, 4, and 5.

[0091] Figure 3: User Data FIG. 3 is a schematic diagram showing user data 221 according to some implementations. In some implementations, with the permission of the user, the user data 221 can be accessed by the online gaming platform 102 and / or stored, for example, in the data storage device 108. The user data 221 can include, but is not limited to, a user ID 301, profile data features 303, behavioral data features 305, player data features 307, and / or other data features. Although shown in tabular form, it should be noted that any suitable data storage format may be used to organize the user data 221.

[0092] For each user, the data to be stored can be based on user-specific permissions provided by the user. The user is given options to choose the specific data that can be stored, the parameters related to the storage of such data (e.g., the period for which the data can be stored), and how the data can be used (e.g., aggregated, processed, or otherwise analyzed). Further, the storage of user data is carried out in accordance with applicable regulations (e.g., in the user's region). Some data can be processed so that personally identifiable information (PII) of the user is not stored or cannot be retrieved from the stored data. Access to data for various programs of the platform is restricted such that only specific programs that perform specific actions permitted by the user can access the data. For example, the data can be aggregated (e.g., across multiple user accounts) or otherwise processed before being provided to a program.

[0093] The profile data feature 303 can include relatively static features about the user. For example, in some implementations, the profile data feature 303 can include age, registration / sign-up date, gender, language, region / location, type of client device, time of access, whether the user is a registered user of the platform, moderation history (e.g., any warnings or bans in the player history), and / or other appropriate profile data features. Other appropriate profile data features may also be applicable, which can include nickname, second language, most used character / avatar, awards / achievements, connections with other players (e.g., social graph), links to other platforms, and other data features.

[0094] The behavioral data feature 305 can include dynamic features as well as static features. For example, in some implementations, the behavioral data feature 305 can be purchase history (e.g., at least one item purchased last week) such as stored by or otherwise available to an online gaming platform, the type of item purchased, the type of item viewed, game play history (e.g., played "pizza place" every day for a month), the type of game played, engagement metrics (e.g., engagement level, engagement time, etc.), designation as a new user (e.g., no previous behavioral data), previous search data (e.g., searched for a specific type of item such as clothing items, weapons, decorative items for an avatar), and other suitable behavioral data features. Additionally, other suitable behavioral data can include "collective group behavior" of players in prohibited games. For example, in addition to examining game play and purchase history at the user level, "crowd behavior" can be analyzed (e.g., did the game suddenly attract a large number of users coming from an off-site link?). Other suitable behavioral data features may also be applicable.

[0095] The player data feature 307 can include dynamic features. For example, in some implementations, the player data feature 307 can include data related to a user who plays a game as a player, including available chat history, voice transcriptions, the type of game played, other players communicated with, the age of the account, the duration of time of the game type played, IP address (e.g., if the user is playing from a particular external service, it may be an automated or "bot" account), and other suitable player data features. Other suitable player data features may also be applicable.

[0096] Therefore, the user data 221 may include, at least, the user's age, user attributes, user gameplay history, behavioral data of other users related to the user, the user's moderation history, the user's chat history, the user's transaction history, moderation history, the number of alternative accounts (if available), newness of activity (e.g., has the player become active after a long period on the platform?), and the average rating of the games played by the user. Other user data 221 is also applicable. The user data 221 can be used to identify groupings of similar users and demonstrate that similar users may also be involved in similar behaviors. For example, a neighborhood clustering function can be used to determine whether a user related to the user data 221 is similar to a user identified as being involved with prohibited content. Similarly, a distance function can be used to determine the level of similarity between a user related to the user data 221 and an exemplary user who is (or was previously) involved with prohibited content. Additionally, the user data 202 can be used to determine external relationships between users so as to make other inferences about the likelihood of involvement with prohibited content.

[0097] Figure 4: Developer Data Figure 4 is a schematic diagram showing developer data 241 according to some implementations. In some implementations, with the permission of the developer, the developer data can be stored by the online gaming platform 102, for example, in the data storage device 108. The developer data 241 can include, without limitation, a developer ID 401, profile data features 403, behavioral data features 405, and / or other data features. Note that although shown in tabular form, any suitable data storage format may be used to compile the developer data 241.

[0098] For each developer, the data to be stored may be based on developer-specific permissions provided by the developer. The developer is given options to choose the specific data that may be stored, and the parameters related to the storage of such data (e.g., the period for which the data may be stored), and how the data may be used (e.g., aggregated, processed, or otherwise analyzed). Further, the storage of developer data is carried out in accordance with applicable regulations (e.g., in the developer's region). Some data may be processed so that personally identifiable information (PII) of the developer is not stored or cannot be retrieved from the stored data. Access to data for various programs of the platform is restricted such that only specific programs that perform specific actions permitted by the developer can access the data. For example, the data may be aggregated (e.g., across multiple developer accounts) or otherwise processed before being provided to a program.

[0099] The profile data feature 403 may include relatively static features about the developer. For example, in some implementations, the profile data feature 403 may include company / employer affiliation, language, region / location, device type, access time, whether the developer is associated with a larger group of developers, and / or other suitable profile data features. Other suitable profile data features may also be applicable and may include other data features such as nickname, second language, famous character / avatar, awards / achievements included, connections with other developers or players (e.g., social graph), links to other platforms, and other data features that are stored by the online gaming platform or are otherwise available to the online gaming platform.

[0100] The behavioral data feature 405 may include dynamic features as well as static features. For example, in some implementations, the behavioral data feature 405 may include the type of item created, the type of game created, the frequency of new content creation, a measure of engagement (e.g., engagement level, engagement time, etc.), designation as a new developer (e.g., no previous behavioral data), the age of the developer account, the presence / number of alternative accounts (if available), moderation history, previous search data (e.g., searched for a specific type of item such as clothing items, weapons, decorative items for an avatar), and other suitable behavioral data features. Other suitable behavioral data features may also be applicable.

[0101] Accordingly, the developer data 241 may include at least the developer's operating period, account verification status, presence of alternative accounts (if available), moderation history, the number of games published by the developer, the frequency of game publication, user behavioral data related to the developer, the developer's moderation history, and the average rating of games related to the developer. Other developer data 241 may also be applicable. The developer data 241 may be used to identify the level of similarity between the developer associated with the developer data 241 and any user involved (or previously involved) with prohibited content. Additionally, the developer data 204 may be used to determine the external relationship between the developer and the user such that other inferences can be made about the likelihood of creation of prohibited content / engagement with prohibited content.

[0102] Figure 5: Game Data FIG. 5 is a schematic diagram showing game data 261 according to some implementation forms. In some implementation forms, the game data 261 can be stored by an online gaming platform 102 in, for example, a data storage device 108. The game data 261 can include, but is not limited to, a game ID 501, content data features 503, game play data features 505, and / or other data features. Although shown in tabular form, it should be noted that any suitable data storage format may be used to organize the game data 261.

[0103] In some implementations, game data 261 includes descriptions, assets, locations, or game levels, objectives, actions, and developer reputations related to game 105. Thus, game data 261 can include textual descriptions of game content, titles of game content, assets of game content, models (e.g., game levels, models, etc.) that make up game content, actions within game content, and one or more of the average ratings of similar games related to the game content. Other game data 261 is also applicable. Game data 261 can be used to determine the likelihood that a related game includes prohibited content. Additionally, external game data 106 can include additional text data, descriptions, and euphemisms useful for determining the likelihood that a related game includes prohibited content. For example, without limitation, text analysis of game descriptions, such as syntax, patterns of word usage, and other linguistic elements, can provide an indication that a game includes prohibited content. Additionally, games repeatedly described on websites known to host explicit or prohibited content can be an indication that a game includes prohibited content. Further, additional analysis can include analysis of reports of violations against games. For example, the analysis can include reports about games where a user describes keywords / expressions indicating that the user has played a game with potentially prohibited content (e.g., "Please remove this game. It has adult content or explicit images").

[0104] Details regarding the training of the machine learning model and rating are described below with respect to FIG. 6.

[0105] FIG. 6: Exemplary method for training a model to rate content FIG. 6 is a flowchart of an exemplary method 600 for training a machine learning model to rate content according to some implementations. In some implementations, method 600 may be implemented, for example, in a server system, such as an online gaming platform 102 as shown in FIG. 1. In some implementations, some or all of method 600 may be implemented in a system such as one or more client devices 110 and 116 as shown in FIG. 1, and / or both in a server system and one or more client systems. In the example being described, the system implementing the method includes one or more processors or processing circuits, and one or more storage devices such as a database or other accessible storage. In some implementations, various components of one or more servers and / or clients can execute different blocks or other portions of method 600. Method 600 may begin at block 602.

[0106] Blocks 602 - 604 illustrate the creation of input features for a machine learning model. In block 402, user data for multiple users, developer data related to a game or content, and game data related to a game or content are obtained, for example, at a server, with permission from the corresponding data owners (e.g., game players, other users, game developers, etc.). The data can include user data (e.g., 221), developer data (e.g., 241), and game data (e.g., 261). User data can include age, gender, language, region / location, type of client device, time of access, and other appropriate user data. User data can also include purchase history, type of purchased items, viewing benefits, type of viewed items, game play history, type of played games, engagement metrics, designation as a new user, previous search data, and other appropriate user data. Developer data can include development history, previous management issues, previous prohibited content, and other appropriate developer data. Game data can include text data that describes items, actions, and other content. Game data can also include external data that describes game play or other attributes. There may be a block 604 after block 602.

[0107] In block 604, the input feature amount can be determined from the acquired user data 221, developer data 241, and game data 261. For example, the input feature amount may include feature amounts 303, 305, 307, 403, 405, 503, and 505 compiled by user ID 301, developer ID 401, and game ID 501. The input feature amount may relate to user activities, static user feature amounts, developer activities, static developer feature amounts, and game feature amounts for input to a machine learning algorithm. FIGS. 3 to 5 show the input feature amount in tabular form, but it can be understood that the acquired user data 221, developer data 241, and game data 261 can be processed in various ways to obtain the input feature amount. For example, in some implementations, a multi-dimensional vector and / or matrix representation with numerical values for each dimension can be obtained based on the acquired user data 221, developer data 241, and game data 261, for example, by converting the input data into a multi-dimensional vector. The conversion can be performed by a machine learning model or other appropriate techniques.

[0108] In some implementations, as shown by operation 622, there may be a block 602 after block 604. For example, the determination of the input feature amount can be performed multiple times until the input feature amounts for a threshold number of users, developers, or games are determined, or until the data available for determining the input feature amount is analyzed. In some implementations, blocks 602 to 604 can be executed periodically, for example, every hour, every day, etc., or when at least a threshold amount of user / developer / game data that has not been processed so far becomes available. When the input feature amount is determined, there may be a block 606 after block 604.

[0109] Blocks 606 to 612 show the training of a machine learning model. In block 606, the method may include the step of providing the input feature amount to the machine learning model. The machine learning model can be configured to generate a likelihood of being associated with content for which a related game or game content item is prohibited based on the input feature amount.

[0110] A machine learning model may include coefficients for nodes (e.g., neural network nodes). The machine learning model may be initialized and trained.

[0111] Generally, training can be performed using supervised or unsupervised learning. Further, according to some implementations, when implemented using a single node and function, an administrator can manually investigate the machine learning model (e.g., node coefficients) and / or model inputs (feature values of input features) and output them along with model performance (whether the output generated by the model meets a criterion of accuracy or a threshold of basic safety). For example, according to one implementation, historical user data, developer data, and game data may be input at block 606. If the model output indicates that the game meets a predetermined or desired safety threshold (e.g., the model outputs a rating that accurately demonstrates whether the associated content includes prohibited content), the model training can be considered complete. Alternatively, the model can be retrained based on additional historical user data, developer data, and game data until it exceeds a predetermined or desired safety threshold. After block 606, there may be a block 608.

[0112] At block 608, output features including numerical values for each user, developer, and game being considered are obtained from the machine learning model. For example, the machine learning model generates a set of user, developer, and game rating values as output. Generally, the output values are based on input features, which in some cases may include previous ratings of users and developers. According to some implementations, the output of the machine learning model is a single value representing the rating of a game or content within the game based on the developers associated with the game and the users interacting with the game. After block 608, there may be a block 610.

[0113] In block 610, a rating is generated for each game based on the output feature amount. In some implementations, the rating can be generated to be within a defined numerical scale, for example, between 1 and 10 or between 1 and 100. In this way, the safety threshold can be defined as a number (for example, 5 or 55) within the defined numerical scale. Therefore, for ratings below the threshold, an inference can be made about the possibility that the item contains prohibited content. There may be a block 612 after block 610.

[0114] In block 612, at least one model parameter for the machine learning model is adjusted based on a comparison of the generated rating with the known rating for each item. In some implementations, the known rating is based on historical data indicating whether a game, user, or developer is associated with prohibited content.

[0115] For example, the generated rating can be compared with a subset of users with prohibited content, a subset of developers, and / or the actual observed behavior of the actual game. These comparisons can be used as feedback to train the machine learning model, for example, by adjusting the weights for the nodes of the machine learning model.

[0116] In some implementations, the training can be repeated periodically as indicated by operation 624. The period for the next round of training can be a predetermined or desired period, such as daily, weekly, monthly, or others. The training can also be repeated until the generated rating and the known rating come within the distance of the threshold.

[0117] Furthermore, in some implementations, various portions of method 600 may be repeated periodically. For example, blocks 602-604 may be executed, for example, once a week, once a month, as needed, when a number of new users join the platform, when other changes to the platform become available, such as new games, new game items, new avatar accessories, etc., to generate new features. Further, in some implementations, training (blocks 606-612) may be performed separately from the generation of input features. For example, the model may be trained without changing the input features, for example, when the model performance drops below a threshold, when prohibited content surfaces after initial training, or when any other change to the platform occurs. In this way, updated user data, updated developer data, and updated game data based on user profile data and behavior parameters can be considered by performing operations 622 and 624 to fine-tune performance.

[0118] Blocks 602-612 may be executed (or repeated) in an order different from that described above and / or one or more blocks may be omitted. For example, feature generation (blocks 602-604) may be performed independently of model training (blocks 606-612). Further, training of multiple models (for example, one model for generating user ratings, one model for generating developer ratings, one model for generating game ratings) may be performed in parallel, for example, by executing blocks 606-612 in parallel for different models.

[0119] After training an initial set of machine learning models or implementing a pre-trained or default-initialized machine learning model (e.g., prior to training), the model may be used to generate item ratings in online gaming platform 102, as more fully described with respect to FIG. 7.

[0120] Figure 7: Surfacing of Content Based on Rating FIG. 7 is a flowchart of an exemplary method 700 for providing content to a user based on a rating, according to some implementations. In some implementations, method 700 may be implemented, for example, in a server system, such as in an online gaming platform 102 as shown in FIGS. 1 and 2. In some implementations, some or all of method 700 may be implemented in a system such as one or more client devices 110 and 116 as shown in FIG. 1, and / or in both a server system and one or more client systems. In the example being described, implementing the system includes one or more processors or processing circuits, and one or more storage devices such as a database or other accessible storage. In some implementations, different components of one or more servers and / or clients may execute different blocks or other portions of method 700. Method 700 may begin at block 702.

[0121] At block 702, a request for game content available on an online gaming platform (e.g., 102) may be received from a user (e.g., 114 / 120). For example, the user may utilize a search engine or other interface (e.g., a browsing interface) provided by the platform. In some implementations, a list of available items may be stored in the online gaming platform 102, for example, through data storage device 106. The request may also be embodied as the activation or selection of a hyperlink (e.g., a link from an external source such as a website, social network, or newsgroup) to content available on the online gaming platform. The hyperlink or “link” may include a direct link, or a query including identification data such as a game ID (e.g., 501) or other data. Block 704 may follow block 702.

[0122] In block 704, a particular game or content is identified based on a request. For example, if the request is a search query, a list of matching games can be returned through a search engine on an online gaming platform. Similarly, a database query can be executed to identify one or more games or other content items. There may be a block 706 after block 704.

[0123] In block 706, a rating (or ratings of multiple games) of the game can be received, for example, by applying a machine learning model. The rating can be compared to a predetermined or desired safety threshold to determine whether the associated content exceeds the threshold. If the rating does not exceed the threshold, the method can include steps of identifying other games or content until a safety threshold is exceeded. There may be a block 708 after block 706.

[0124] In block 708, other games or content identified by the method can be determined to exceed the threshold. There may be a block 710 after block 708.

[0125] In block 710, a list of items (or in some implementations, a single item) that exceed the safety threshold is generated. Thus, the list of items is adjusted to include only non-prohibited content based on the trained machine learning model of FIG. 6. There may be a block 712 after block 710.

[0126] In block 712, a user interface including a list of games and / or game content is provided to the user. In this way, a user interface without prohibited content is provided. Note that if the request in block 702 is a link to a specific game or content, only the relevant game or content can be returned if its associated rating exceeds the safety threshold.

[0127] As described above, the machine learning model (or a separate model) can be retrained periodically. Additionally, the safety threshold may also be changed periodically to ensure that a reasonable percentage of prohibited content is restricted and that the desired safety threshold is met or exceeded. In this regard, if too many non-prohibited games or content are restricted, the safety threshold may be lowered. Similarly, if one or more prohibited games or content surface to the user, the safety threshold may be raised. In some implementations, user feedback and developer feedback may be used to monitor the ratings and retrain the machine learning model. Additionally, an administrator of the online gaming platform 102 may manually adjust the machine learning model, the ratings, or other attributes to change the surfacing of non-prohibited content that is rated as prohibited.

[0128] Blocks 702-712 may be executed (or repeated) in an order different from that described above, and / or one or more blocks may be omitted. Methods 600 and / or 700 may be executed on a server (e.g., 102) and / or a client device (e.g., 110 or 116). Further, according to any desired implementation, some of methods 600 and 700 may be combined, and may be executed sequentially or in parallel.

[0129] As described above, rating techniques based on user data, developer data, and game data can be used to rate content on an online gaming platform before surfacing the content to users. In this way, users are not provided with prohibited content even when accessing the online gaming platform using external links designed to avoid other security measures. Additionally, automated rating may be used to enhance the standard management and / or moderation of the online gaming platform so that parental controls and other restrictions on content are more able to identify prohibited content and prevent its surfacing to users.

[0130] More detailed descriptions of various computing devices that can be used to implement the various devices shown in FIGS. 1 and 2 are given with respect to FIG. 8 below.

[0131] FIG. 8 is a block diagram of an exemplary computing device 800 that may be used to implement one or more features described herein according to some implementations. In one example, device 800 may be used to implement a computer device (e.g., 102, 110, and / or 116 of FIG. 1) and to execute implementations of suitable methods described herein. The computing device 800 may be any suitable computer system, server, or other electronic or hardware device. For example, the computing device 800 may be a mainframe computer, a desktop computer, a workstation, a portable computer, or an electronic device (portable device, mobile device, cellular phone, smartphone, tablet computer, television, TV set-top box, personal digital assistant (PDA), media player, game device, wearable device, etc.). In some implementations, device 800 includes a processor 802, a memory 804, an input / output (I / O) interface 806, and an audio / video input / output device 814 (e.g., a display screen, touch screen, display goggles or glasses, audio speaker, microphone, etc.).

[0132] The processor 802 can be one or more processors and / or processing circuits for executing program code and controlling the basic operations of the device 800. A "processor" includes any suitable hardware and / or software system, mechanism, or component that processes data, signals, or other information. The processor can include a general-purpose central processing unit (CPU), multiple processing units, a system with dedicated circuitry for achieving functions, or other systems. Processing need not be limited to a specific geographical location or have temporal constraints. For example, the processor may execute its functions in "real time", "offline", "batch mode", etc. Some parts of the processing may be executed at different times and in different locations by different (or the same) processing systems. A computer can be any processor that communicates with a memory.

[0133] The memory 804 is typically provided in the device 800 for access by the processor 802 and is suitable for storing instructions for execution by the processor, and can be any suitable processor-readable storage medium located remotely from and / or integrated with the processor 802, such as random access memory (RAM), read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), flash memory, etc. The memory 804 can store software that operates on the server device 800 by the processor 802, including an operating system 808, a search engine application 810, and related data 812. In some implementations, the search engine application 810 can include instructions that enable the processor 802 to execute some or all of the functions described herein, such as the methods of FIGS. 6 and 7. In some implementations, as described herein, the search engine application 810 may also include one or more machine learning models for generating content ratings and providing a user interface and / or other features of the platform based on the ratings.

[0134] For example, memory 804 may include software instructions for a search engine 810 that can provide a search while observing content ratings (e.g., while restricting prohibited content) within an online gaming platform (e.g., 102). Alternatively, any of the software in memory 804 may be stored in any other suitable storage location or computer-readable medium. Additionally, memory 804 (and / or other connected storage devices) can store the instructions and data used in the features described herein. Memory 804 and any other type of storage (magnetic disk, optical disk, magnetic tape, or other tangible media) may be considered a "storage" or "storage device".

[0135] The I / O interface 806 can provide functions to enable the server device 800 to interface with other systems and devices. For example, network communication devices, storage devices (e.g., memory and / or data storage device 108), and input / output devices can communicate via interface 806. In some implementations, the I / O interface can be connected to interface devices that include input devices (such as keyboards, pointing devices, touchscreens, microphones, cameras, scanners, etc.) and / or output devices (such as display devices, speaker devices, printers, motors, etc.).

[0136] To simplify the illustration, FIG. 8 shows one block for each of processor 802, memory 804, I / O interface 806, software blocks 808 and 810, and database 812. These blocks may represent one or more processors or processing circuits, an operating system, memory, an I / O interface, an application, and / or a software module. In other implementations, device 800 may not have all of the components shown, and / or may have other elements including other types of elements instead of, or in addition to, those shown herein. Online gaming platform 102 is described as performing operations as described in some implementations herein, but online gaming platform 102 or any suitable component or combination of components of a similar system, or any suitable one or more processors associated with such a system, may perform the described operations.

[0137] The user device may also implement and / or be used with the features described herein. Exemplary user devices may be computing devices that include some components similar to device 800, such as processor 802, memory 804, and I / O interface 806. Operating systems, software, and applications suitable for client devices may be provided in the memory and used by the processor. The I / O interface for the client device may be connected to network communication devices as well as input and output devices, such as a microphone for capturing sound, a camera for capturing images or video, an audio speaker device for outputting sound, a display device for outputting images or video, or other output devices. The display device within the audio / video input / output device 814 may be connected to (or included in) device 800, for example, to display pre - processing and post - processing of images as described herein, and such display devices may include any suitable display device, such as an LCD, LED, or plasma display screen, CRT, television, monitor, touch screen, 3D display screen, projector, or other visual display device. Some implementations may be able to provide an audio output device, such as voice output or synthesis for speaking text.

[0138] The methods, blocks, and / or operations described herein may, as appropriate, be executed in an order different from that shown or described, and / or may be executed (partially or fully) concurrently with other blocks or operations. Some blocks or operations may be executed for some portions of the data and, for example, may be executed again later for other portions of the data. In various implementations, not all of the described blocks and operations need to be executed. In some implementations, the blocks and operations may be executed multiple times, in different orders, and / or at different times in the method.

[0139] In some implementations, some or all of the method may be implemented in a system such as one or more client devices. In some implementations, one or more of the methods described herein may be implemented, for example, in a server system and / or in both a server system and a client system. In some implementations, different components of one or more servers and / or clients can execute different blocks, operations, or other parts of the method.

[0140] One or more of the methods described herein (e.g., method 600 and / or 700) can be implemented by computer program instructions or code that can be executed on a computer. For example, the code may be implemented by one or more digital processors (e.g., a microprocessor or other processing circuitry), and may be stored in a computer program product that includes a non-transitory computer-readable medium (e.g., a storage medium), such as a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, or a semiconductor storage medium, including, for example, semiconductor or solid-state memory, magnetic tape, removable computer diskette, random access memory (RAM), read-only memory (ROM), flash memory, fixed magnetic disk, optical disk, solid-state memory drive, etc. The program instructions may also be included in, or provided as, an electronic signal in the form of software as a service (SaaS), for example, delivered from a server, or such an electronic signal (e.g., in a distributed system and / or a cloud computing system). Alternatively, one or more of the methods can be implemented in hardware (such as logic gates, etc.) or in a combination of hardware and software. Exemplary hardware can include a programmable processor (e.g., a field programmable gate array (FPGA), a complex programmable logic device), a general-purpose processor, a graphics processor, an application specific integrated circuit (ASIC), etc. One or more of the methods can be executed as part of an application or component running on a system, or as an application or software that runs in conjunction with other applications and an operating system.

[0141] One or more of the methods described herein can be executed in a stand-alone program that can be executed on any type of computing device, a program executed on a web browser, or a mobile application (an "app") executed on a mobile computing device (e.g., a cellular phone, smartphone, tablet computer, wearable device (such as a wristwatch, armband, jewelry, headgear, goggles, glasses, etc.), laptop computer, etc.). In one example, a client / server architecture may be used, for example, a mobile computing device (as a client device) sends user input data to a server device and receives final output data from the server for output (e.g., for display). In another example, all calculations can be performed within a mobile application (and / or other applications) on the mobile computing device. In another example, the calculations can be split between the mobile computing device and one or more server devices.

[0142] Although specific implementations have been described, these specific implementations are merely illustrative and not limiting. The concepts shown in the examples can be applied to other examples and implementations.

[0143] In situations where some of the implementations discussed herein may obtain or use user data (e.g., user attributes, user behavior data on the platform, user search history, purchased and / or viewed items, friends of the user on the platform, etc.), the user is given the option to control whether such information is collected, stored, or used, or how they are done. That is, the implementations discussed herein collect, store, and / or use user information upon receiving clear user approval and in accordance with applicable regulations.

[0144] The user is given the right to control whether a program or function collects user information about that particular user or other users related to that program or function. Each user about whom information is to be collected is presented with options (e.g., via a user interface) to enable the user to exercise control over the collection of information related to that user and provide permission or approval as to whether information is to be collected and which portions of the information are to be collected. Additionally, some data may be modified in one or more ways before storage or use so that personally identifiable information is deleted. As an example, the user's identifying information may be modified (e.g., by substitution using pseudonyms, numbers, etc.) so that personally identifiable information cannot be determined. In another example, the user's geographical location may be generalized to a broader area (e.g., city, zip code, state, country, etc.).

[0145] Note that the functional blocks, operations, features, methods, devices, and systems described in this disclosure may be integrated or divided into different combinations of systems, devices, and functional blocks that may be known to those skilled in the art. Any suitable programming language and programming technique may be used to implement the routines of a particular implementation. Various programming techniques, such as procedural programming techniques or object-oriented programming techniques, may be utilized. The routines may be executed on a single processing device or on multiple processors. Steps, operations, or calculations may be presented in a particular order, but the order may be changed in different particular implementations. In some implementations, multiple steps or operations shown as sequential herein may be executed simultaneously.

Description of Reference Numerals

[0146] 102 Online gaming platform 104 Game engine 105 Game 106 Search Engine 107 Game Rating Engine 108 Data Storage Device 110 Client Device A 112 Game Application 114 User A 116 Client Device n 118 Game Application 120 User n 122 Network 202 User Data Source 204 Developer Data Source 221 User Data 241 Developer Data 261 Game Data 271 Rating 301 User ID 303 Profile Data Feature 305 Behavior Data Feature 307 Player Data Feature 401 Developer ID 403 Player Data Feature 405 Behavior Data Feature 501 Game ID 503 Content Data Feature 505 Gameplay Data Feature 800 Computing Device 802 Processor 804 Memory 806 I / O Interface 808 Operating System 810 Search Engine Application 812 Database 814 I / O Device

Claims

1. Receiving data related to a game hosted on a gaming platform, the data indicating at least one developer, at least one player, and game content; Automatically assigning a rating to the game using a machine learning model, the rating being based on the data indicating the at least one developer, at least one player, and game content; Determining whether the rating of the game meets a safety threshold; Identifying the game as having prohibited content based on a determination that the rating of the game does not meet the safety threshold. A computer-implemented method comprising the steps of.

2. The method according to claim 1, wherein the data indicating the at least one developer comprises one or more of the developer's operating period on the gaming platform, the number of games published by the developer on the gaming platform, player behavior data related to the developer on the gaming platform, the developer's moderation history on the gaming platform, or the average rating of games related to the developer on the gaming platform. A computer-implemented method.

3. The method according to claim 1, wherein the data indicating at least one player comprises one or more of the player's attributes, the player's gameplay history, other player behavior data related to the player, the player's moderation history, the player's chat history, the player's transaction history, or the average rating of games played by the player. A computer-implemented method.

4. The method according to claim 1, wherein the data indicating the game content comprises one or more of a textual description of the game, the title of the game, the assets of the game, the models constituting the game, the actions within the game, or the average rating of one or more other games having content within a similarity threshold range of the game content. A computer-implemented method.

5. The method implemented on a computer according to claim 1, further comprising the step of restricting access to the game content via the gaming platform based on a determination that the game content includes prohibited content.

6. Receiving data related to a game hosted on a gaming platform, the data indicating at least one developer, at least one player, and game content; Determining input features of a machine learning model based on the at least one developer, the at least one player, and the data indicating the game content; Providing the input features to the machine learning model; Obtaining an output of the machine learning model based on the input features, the output including a rating of the game; The method implemented on a computer, wherein the rating of the game is a value indicating the possibility that the game includes prohibited content.

7. The method implemented on a computer according to claim 6, wherein the data indicating the at least one developer comprises one or more of the developer's operating period on the gaming platform, the number of games published by the developer on the gaming platform, player behavior data related to the developer on the gaming platform, the moderation history of the developer on the gaming platform, or the average rating of games related to the developer on the gaming platform.

8. The method implemented on a computer according to claim 6, wherein the data indicating the at least one player comprises one or more of the player's attributes, the player's game play history, other player behavior data related to the player, the player's moderation history, the player's chat history, the player's transaction history, or the average rating of games played by the player.

9. The method implemented by a computer according to claim 6, wherein the data indicating the game content includes one or more of the average ratings of one or more other games having content within the range of the threshold of similarity of the text description of the game, the title of the game, the assets of the game, the models constituting the game, the actions within the game, or the game content.

10. A system comprising: a memory storing instructions; and a processing device coupled to the memory, the processing device being configured to access the memory and execute the instructions, the instructions causing the processing device to receive data related to a game, the data indicating at least one developer, at least one player, and game content, the receiving; automatically assign a rating to the game using a machine learning model, the rating being based on the data indicating the at least one developer, at least one player, and game content, the assigning; determine whether the rating of the game meets a safety threshold; and identify the game as having prohibited content based on a determination that the rating of the game does not meet the safety threshold and execute operations including.

11. The system according to claim 10, wherein the data indicating the at least one developer includes one or more of the operating period of the developer on the gaming platform, the number of games published by the developer on the gaming platform, player behavior data related to the developer on the gaming platform, the moderation history of the developer on the gaming platform, or the average rating of games related to the developer on the gaming platform.

12. The system according to claim 10, wherein the data indicating at least one player comprises one or more of the player's attributes, the player's game play history, the behavior data of other players related to the player, the player's moderation history, the player's chat history, the player's transaction history, or the average rating of the games played by the player.

13. The system according to claim 10, wherein the data indicating the game content comprises one or more of a textual description of the game, the title of the game, the assets of the game, the models constituting the game, the actions within the game, or the average rating of one or more other games having content within a threshold range of similarity to the game content.

14. The system according to claim 10, wherein the operation further comprises restricting access to the game content via a gaming platform based on a determination that the game content includes prohibited content.

15. A non-transitory computer-readable medium storing instructions that, in response to execution by a processing device, cause the processing device to receive data related to a game, the data indicating at least one developer, at least one player, and game content, the receiving determine input features of a machine learning model based on the data indicating the at least one developer, the at least one player, and the game content provide the input features to the machine learning model obtain an output of the machine learning model based on the input features, the output including a rating of the game, the obtaining to perform operations including A non-transitory computer-readable medium, wherein the rating of the game is a value indicating the likelihood that the game includes prohibited content.

16. The non-transitory computer-readable medium according to claim 15, wherein the data indicating the at least one developer comprises one or more of the developer's operating period on the gaming platform, the number of games published by the developer on the gaming platform, player behavior data related to the developer on the gaming platform, the moderation history of the developer on the gaming platform, or the average rating of games related to the developer on the gaming platform.

17. The non-transitory computer-readable medium according to claim 15, wherein the data indicating at least one player comprises one or more of the player's attributes, the player's game play history, the behavior data of other players related to the player, the moderation history of the player, the player's chat history, the player's transaction history, or the average rating of games played by the player.

18. The non-transitory computer-readable medium according to claim 15, wherein the data indicating the game content comprises one or more of the textual description of the game, the title of the game, the assets of the game, the models constituting the game, the actions within the game, or the average rating of one or more other games having content within a threshold range of similarity of the game content.

Images