Graph-based feature engineering for machine learning models

Abstract

Methods and systems are presented for assisting a user in identifying and evaluating features for use in a machine learning model configured to perform tasks. A user interface is provided on the device based on graph data associated with the graph data structure. Feature candidates for the machine learning model are determined based on user input received via the user interface. The feature candidates are associated with a particular manner of traversing the graph data structure to obtain attribute values associated with one or more vertices and / or one or more edges in the graph data structure. Based on the attribute values, values corresponding to the feature candidates may be calculated. This value may be used to assess the validity of the feature candidates in performing the task. The feature candidates may then be incorporated into a machine learning model as one of the input features.

Description

Technical Field

[0001] The present specification relates generally to machine learning models, and more specifically to providing a tool for analyzing and evaluating features used in machine learning models according to various embodiments of the present disclosure. Background Art

[0002] Machine learning models have been widely used to perform various tasks for organizations. For example, machine learning models can be used to classify data (e.g., determine whether a transaction is a legitimate transaction or a fraudulent transaction, determine whether a merchant is a high-value merchant, determine whether a user is a high-risk user, etc.). In order to construct a machine learning model, a set of input features related to performing tasks associated with the machine learning model is identified and defined. Training data can be provided to train the machine learning model, the training data including attribute values ​​corresponding to the set of input features and labels corresponding to predetermined prediction results. Based on the training data and the labels, the machine learning model can learn patterns associated with the training data and provide predictions based on the learned patterns. For example, new data corresponding to the set of input features (e.g., transaction data associated with a new transaction) can be provided to the machine learning model. The machine learning model can perform predictions on the new data based on the patterns learned from the training data (e.g., whether the new transaction is a legitimate transaction, etc.).

[0003] While machine learning models are effective at identifying patterns and making predictions, they are limited to the input features that are identified at construction time. When a set of input features selected for a machine learning model are highly relevant to the task performed by the machine learning model (e.g., highly relevant to the predictions performed by the model), the machine learning model can be trained to perform the task with high accuracy. Conversely, when a set of input features selected for a machine learning model are irrelevant to the task, the machine learning model may not perform the task well (e.g., make inaccurate predictions) even with a large amount of training data. Further complicating the problem is that data trends and patterns change frequently, such that features that were determined to be highly relevant to performing a task some time ago may no longer be relevant, while features that were previously determined to be irrelevant to performing a task may become relevant. Thus, there is a need for a computer-based tool for efficiently identifying and evaluating features for machine learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is a block diagram illustrating a networked system including an electronic transaction system according to an embodiment of the present disclosure;

[0005] Figure 2 An example diagram according to an embodiment of the present disclosure is illustrated;

[0006] Figure 3-Figure 5An exemplary user interface provided by a feature engineering system for identifying and evaluating feature candidates according to an embodiment of the present disclosure is illustrated;

[0007] Figure 6 is a block diagram illustrating a feature engineering module according to an embodiment of the present disclosure;

[0008] Figure 7 A flowchart illustrating a process of identifying and evaluating feature candidates according to an embodiment of the present disclosure is illustrated; and

[0009] Figure 8 is a block diagram of a system for implementing a device according to an embodiment of the present disclosure.

[0010] Embodiments of the present disclosure and their advantages are best understood by referring to the following detailed description. It should be understood that like reference numerals are used to identify like elements illustrated in one or more of the drawings, wherein the showings are for illustrating embodiments of the present disclosure and not for limiting thereof. DETAILED DESCRIPTION

[0011] The present disclosure includes methods and systems for providing a computer-based tool for identifying and evaluating features for use in machine learning models. As discussed herein, selecting high-quality input features for machine learning models - a process also referred to as "feature engineering" - is critical to improving the performance of machine learning models. However, feature engineering is a complex process that traditionally requires experts in different fields (e.g., data scientists, professional software programmers, etc.) to work together for a long period of time to generate and test different feature candidates. This process is further complicated when the data that can be used to identify features is embedded within a graph structure.

[0012] Many organizations have adopted graph structures to store data. A graph structure is a data structure that uses vertices to represent different entities. Vertices can be connected to each other based on one or more types of relationships. In this way, storing data in a graph structure is beneficial for illustrating the relationships between different entities. For example, a social media platform can use a graph structure to represent different users and the relationships between different users and the social media platform. In this example, each user can be represented by a different vertex in the graph, and the relationship between two users (e.g., friendship, spouse, sibling, etc.) can be represented by an edge between two corresponding vertices in the graph.

[0013] In another example, an online payment provider may use a graph structure to represent user accounts associated with the payment provider. In this example, each user account may be represented by a different vertex in the graph, and each payment transaction performed between two accounts may be represented by an edge connecting two corresponding vertices. In some embodiments, in addition to user accounts, or different from user accounts, online payment providers may also use vertices to represent different assets or attributes. For example, online payment providers may also use vertices to represent assets or attributes that may be shared by different user accounts, such as network addresses (e.g., Internet Protocol addresses, email addresses, etc.), device identifiers, physical locations (e.g., residential addresses, etc.), sources of funds (e.g., credit card accounts, debit card accounts, etc.), and other data related to payment transactions. Vertices may then be connected based on different types of relationships. For example, when a user account uses a specific device for a transaction, an edge may connect a vertex representing a user account and a vertex representing a specific device. When two user accounts use the same device for a payment transaction, two vertices representing the two user accounts will be connected to a vertex representing the same device. In this way, a graph structure may be used to represent the relationship between different entities and / or assets in a clearer manner than using a traditional data structure (e.g., a relational database, etc.).

[0014] The disadvantage of using a graph structure to store data is that accessing graph data in a graph structure (e.g., traversing a graph structure, getting values ​​in a graph structure, creating new vertices or edges in a graph structure, etc.) requires a specialized graph query language (e.g., Gremlin, Cypher, etc.). These graph query languages ​​are more complex and more difficult to learn and use than conventional query languages ​​(e.g., SQL, etc.). Traditionally, data scientists have the knowledge and skills to analyze data to determine the features of a machine learning model, and they rely mainly on professional software programmers to identify and evaluate different feature candidates for a machine learning model.

[0015] For example, based on the pattern of the graph structure, the data scientist can identify possible feature candidates. Feature candidates may be associated with graph traversal logic and calculations. In a specific example, the data scientist can determine feature candidates for a machine learning model configured to predict the risk of a new user account (e.g., an account that has just been registered). Feature candidates can be defined as the average transaction amount associated with an existing user account that shares the same network address with the new user account. Thus, in order to generate a value corresponding to this feature candidate based on a seed account (e.g., a new user account), people will make a first jump from a vertex representing a seed account to another connected vertex representing a network address (e.g., a vertex representing an IP address, etc.) in the graph structure, and then make a second jump from a vertex representing a network address to another connected vertex representing an existing user account in the graph structure. Transaction data associated with an existing user account can be obtained, and the average transaction amount can be calculated. In this example, the feature candidate requires a two-hop traversal from a seed vertex within the graph structure to reach a target vertex (e.g., a vertex representing an existing user account), and an average calculation is performed based on the attribute value associated with the target vertex.

[0016] Professional software programmers can (based on communication from data scientists) convert graph traversal logic and calculations into programming code using a graph query language (e.g., Gremlin, Cypher, etc.). By executing the programming code, professional software programmers can obtain sample values ​​corresponding to feature candidates. Data scientists can evaluate the quality of feature candidates based on the sample values ​​(e.g., the relevance of feature candidates to the tasks performed by the machine learning model). This traditional feature engineering process is very time-consuming and error-prone. For example, any disconnect between data scientists and software programmers (e.g., due to poor communication, etc.) may result in delays or even errors in evaluating feature candidates. In addition, the inability to directly access the graph structure further hinders data scientists from identifying feature candidates that may be relevant to the machine learning model.

[0017] In this way, according to various embodiments of the present disclosure, the feature engineering system can assist the user in generating and evaluating features for the machine learning model, and incorporate the features into the machine learning model. In some embodiments, the feature engineering system can access the graph data corresponding to the graph structure from the data storage device. For example, the feature engineering system can use programming code corresponding to one of the graph query languages ​​to access different parts of the graph data. Based on the different parts of the graph data, the feature engineering can generate a representation of at least a part of the graph structure, and present the representation of the graph structure on the user interface. In some embodiments, the feature engineering system can enable the user to specify parameters for generating the representation of the graph. For example, via the user interface of the feature engineering system, the user can specify a seed account and several jumps (connections) from the seed account. The feature engineering system can identify the vertex in the graph structure corresponding to the seed account, and can traverse a specified number of jumps from the vertex. The feature engineering system can obtain the graph data based on the traversal, and generate a representation of the portion of the graph to be displayed via the user interface.

[0018] In some embodiments, the feature engineering system can enable a user to define feature candidates based on inputs provided via a user interface. These inputs may include interactions with representations of graphs displayed on the user interface. For example, a user may define the number of jumps (connections) from a seed vertex to one or more target vertices (e.g., by providing inputs on a representation of a graph), may define the type of edge to be traversed (e.g., the type of relationship) and / or the type of vertex to be traversed (e.g., a vertex representing a specific entity and / or asset), may identify attributes associated with a target vertex and / or an edge connecting a seed vertex to a target vertex, which attributes may be used to calculate feature values ​​corresponding to feature candidates, and may define calculations for feature candidates based on attributes (e.g., average, sum, difference, count, maximum, minimum, etc.). Using the example described above, a user may define feature candidates for predicting the risk of a new user account. Feature candidates may be associated with jumping from a seed vertex to a connection vertex representing a network address and from the vertex to another vertex representing a target account. Feature candidates may also be associated with obtaining transaction amounts for transactions conducted by a target account and calculating the average value of the transaction amounts.

[0019] Based on the input provided by the user via the user interface, the feature engineering system can implement the graph traversal logic using a graph query language. For example, the feature engineering system can generate programming code for performing a traversal of the graph from a given seed vertex representing a seed account, and calculating feature values ​​corresponding to feature candidates of the seed account. In some embodiments, the feature engineering system can perform simulations on feature candidates by repeatedly executing programming code based on different seed accounts (e.g., different seed vertices), and obtain feature values ​​corresponding to the feature candidates from the execution. Using the feature values, the feature engineering system can determine the degree of correlation between the feature candidates and the tasks performed by the machine learning model (e.g., the degree of correlation between the average transaction amount of user accounts with a common network address and the risk prediction of another account). In some embodiments, the feature engineering system can generate a distribution of feature values ​​and present the distribution on a user interface so that the user can determine the degree of correlation between the feature candidates and the tasks performed by the machine learning model.

[0020] When the correlation between the feature candidate and the task performed by the machine learning model is high (e.g., above a threshold), or when a confirmation is received from the user via a user interface, the feature engineering system may incorporate the feature candidate into the machine learning model as one of the input features of the machine learning model. For example, the feature engineering system may modify the structure of the machine learning model (e.g., add additional input nodes to the input layer of the machine learning model when the machine learning model is implemented as an artificial neural network, etc.). In some embodiments, the feature engineering system may also incorporate programming code associated with the definition of the feature candidate into the machine learning model (and / or any software module using the machine learning model).

[0021] In some embodiments, the machine learning model may be configured to dynamically use input features defined by the feature engineering system. For example, the feature engineering system may store programming code as a template for features, rather than incorporating the programming code into the machine learning model (or (one or more) software modules that use the machine learning model). Based on input received from a user via a user interface, the feature engineering system may generate and store different programming codes corresponding to definitions of different features of the machine learning model. The feature engineering system may include a configuration file for the machine learning model. The configuration file may specify which programming codes stored in the feature engineering system are to be used to obtain input values ​​when performing a task (e.g., predicting risks associated with a user account).

[0022] When a request to perform a task is received, the machine learning model can dynamically obtain input values ​​corresponding to features defined (or stored) by the feature engineering system. For example, the machine learning model can access programming code for obtaining input values ​​from the feature engineering system based on a configuration file, and can execute the programming code based on the seed account. In another example, the feature engineering system can execute programming code based on the seed account to obtain values ​​corresponding to the features, and provide these values ​​as input values ​​to the machine learning model.

[0023] The ability of the feature engineering system to generate and manage features for different machine learning models enables the feature engineering system to dynamically modify the features of any one machine learning model without modifying the internal structure of the machine learning model. For example, based on input received from a user via a user interface, the feature engineering system can generate programming code for implementing new features defined by the user. The feature engineering system can add new input features and / or replace existing input features with new features by modifying a configuration file. After modifying the configuration file, the machine learning model (or feature engineering system) can automatically determine the input values ​​corresponding to a set of features including the new input features of the machine learning model when receiving a request to perform a task. The machine learning model can perform tasks using the input values ​​without knowing about changes in the input features.

[0024] In some embodiments, the feature engineering system may also generate training data for training the machine learning model based on the input features defined for the machine learning model. For example, the feature engineering system may traverse the graph structure and obtain (or otherwise calculate) input values ​​corresponding to the features defined for the machine learning model based on the data within the graph structure. The feature engineering system may then use the obtained values ​​to train the machine learning model.

[0025] By enabling users (e.g., an organization's data scientists, or even a machine, such as an AI-based system) to define and evaluate feature candidates, and to incorporate the selected feature candidates into various machine learning models, the process of feature engineering becomes more efficient and less error-prone. In addition, using templates to store different programming codes corresponding to different features of a machine learning model, the input features of a machine learning model can be modified seamlessly and dynamically without accessing and adjusting the internal structure of the machine learning model.

[0026] Figure 1The figure shows a networked system 100, in which a feature engineering system can be implemented according to an embodiment of the present disclosure. However, it is to be noted that the present technology can be applied in many different computing and technical environments, not limited to those shown in the figure. The networked system 100 includes a service provider server 130 and user devices 110, 170 and 180, which can be communicatively coupled to each other via a network 160. In one embodiment, the network 160 can be implemented as a single network or a combination of multiple networks. For example, in various embodiments, the network 160 may include the Internet and / or one or more intranets, landline networks, wireless networks, and / or other appropriate types of communication networks. In another example, the network 160 may include a wireless telecommunications network (e.g., a cellular telephone network) that is suitable for communicating with other communication networks such as the Internet.

[0027] In one embodiment, user device 110 may be utilized by user 140 to interact with service provider server 130 via network 160. For example, user 140 may log into a user account to access account services or conduct electronic transactions (e.g., account transfers or payments, purchases of goods and / or services, etc.) with service provider server 130. In various embodiments, user device 110 may be implemented using any suitable combination of hardware and / or software configured for wired and / or wireless communications via network 160. In various implementations, user device 110 may include at least one of a wireless cellular phone, a wearable computing device, a PC, a laptop, and the like.

[0028] In one embodiment, the user device 110 includes a user interface (UI) application 112 (e.g., a web browser, a mobile payment application, etc.) that the user 140 can utilize to interact with the service provider server 130 via the network 160. In one implementation, the user interface application 112 includes a software program (e.g., a mobile application) that provides a graphical user interface (GUI) for the user 140 to interact and communicate with the service provider server 130 via the network 160. In another implementation, the user interface application 112 includes a browser module that provides a network interface to browse information available on the network 160. For example, the user interface application 112 can be partially implemented as a web browser to view information available on the network 160.

[0029] In one embodiment, the user device 110 may include at least one identifier 114, which may be implemented, for example, as an operating system registry entry, a cookie associated with the user interface application 112, an identifier associated with the hardware of the user device 110 (e.g., a media control access (MAC) address), or various other suitable identifiers. In various implementations, the identifier 114 may be communicated to the service provider server 130 along with the user login request via the network 160, and the identifier 114 may be used by the service provider server to associate the user 140 with a specific user account, a specific digital wallet, and / or a specific profile.

[0030] In various implementations, user 140 can enter data and information into an input component (e.g., a keyboard or microphone) of user device 110. For example, user 140 can use the input component to interact with UI application 112 (e.g., retrieve content from a third-party server (e.g., service provider server 130), etc.).

[0031] Each of user devices 170 and 180 may include similar hardware and software components as user device 110 to enable their respective users to interact with service provider server 130 through user devices 170 and 180. For example, users of user devices 110, 170, and 180 may use their respective devices to conduct electronic transactions (e.g., login transactions, data access transactions, payment transactions, funds transfer transactions, etc.) through different user accounts of service provider server 130.

[0032] In one embodiment, the service provider server 130 may be maintained by a transaction processing entity or online service provider that can provide processing for electronic transactions between different entities (e.g., between users of user devices 110, 170, and 180), electronic transactions between users and one or more business entities, or other types of electronic transactions between payees. As such, the service provider server 130 may include a service application 138 that may be adapted to interact with the user devices 110, 170, and 180 via the network 160 to facilitate searching, selecting, purchasing, paying for items, and / or other services provided by the service provider server 130. In one example, the service provider server 130 may be provided by PayPal, Inc. of San Jose, California, USA. The Company provides, and / or is provided by, one or more service entities or respective intermediaries, which may provide a plurality of point-of-sale devices at various locations to facilitate transaction routing between merchants and, for example, service entities.

[0033] In some embodiments, the service applications 138 may include a payment processing application (not shown) for processing purchases and / or payments for electronic transactions between users and merchants or between any two entities (e.g., between two users, etc.). In one implementation, the payment processing application assists in resolving electronic transactions through verification, delivery, and settlement. Thus, the payment processing application resolves debts between users and / or between users and merchants, where funds can be directly and / or automatically deducted from accounts and / or deposited.

[0034] The service provider server 130 may also include an interface server 134, which is configured to provide content (e.g., web content) to the user and interact with the user. For example, the interface server 134 may include a web server configured to provide web content in response to an HTTP request. In another example, the interface server 134 may include an application server, which is configured to interact with a corresponding application (e.g., a service provider mobile application) installed on the user device 110 via one or more protocols (e.g., REST API, SOAP, etc.). In this way, the interface server 134 may include pre-generated electronic content, ready to be provided to the user. For example, the interface server 134 may store a login page, and is configured to provide a login page to the user, so as to log in to the user's user account, to access the various services provided by the service provider server 130. The interface server 134 may also include other electronic pages associated with the different services (e.g., electronic transaction services, etc.) provided by the service provider server 130. Thus, a user (eg, user 140 , users of user devices 170 , 180 , and 190 , etc.) may access a user account associated with the user and access various services provided by service provider server 130 by generating an HTTP request to service provider server 130 .

[0035] In one embodiment, the service provider server 130 may be configured to maintain one or more user accounts in the account database 136, each account may be associated with a profile, and may include account information associated with one or more individual users (e.g., user 140 associated with user device 110, users associated with user devices 170 and 180). The account information may include an identifier of the user account, as well as other information associated with the user, such as gender, age, demographic information, device information associated with the device associated with the user (e.g., device identifier, Internet Protocol (IP) address of the device, residential address, operating system version, screen attributes of the device, etc.). In one implementation, the user may have credentials for identity authentication or verification with the service provider server 130. Thus, the service provider server may store the user's credentials in a corresponding record of the account database 136 associated with the user account. In some embodiments, the account database 136 may also store transaction data associated with transactions that have been conducted by users of the service provider server 130. The transaction data may include transaction amounts, device information associated with the device used by the user to conduct transactions, merchant (or payee) information, items purchased, and the like.

[0036] In some embodiments, data associated with various user accounts of the service provider server 130 and transaction data associated with transactions conducted through the various user accounts can be stored in a graph data structure 142. In some embodiments, the graph data structure 142 can be part of the account database 136. The graph data structure 142 is different from a typical data structure (e.g., a relational database, etc.) because it does not store data in a linear manner (e.g., by rows and columns), but rather stores data by vertices (also called "nodes") and edges. As such, the graph data structure 142 may include multiple vertices and edges connecting the vertices. Each vertex and edge can be implemented as a different data structure (e.g., an object within a programming code, etc.), which can store data associated with each vertex and edge.

[0037] As discussed herein, in terms of representing the relationship between different objects, graph data structures have advantages over traditional data structures (e.g., relational databases). Various objects can be represented by corresponding vertices, and the relationship between objects can be represented by the edges connecting the corresponding vertices. In this example, the vertices in the graph data structure 142 can be used to represent various entities, attributes, data and / or assets related to the service provider server 130, and the edges within the graph data structure 142 can be used to represent the relationship between different entities, attributes, data and / or assets related to the service provider server 130. For example, the service provider server 130 can use vertices to represent each user account associated with the service provider server 130. Thus, each user account can have a corresponding vertex in the graph data structure 142. The vertex representing the user account can store account data associated with the corresponding user account. For example, each vertex representing the user account can include an account identifier and the attributes of the individual / entity associated with the user account, such as name, gender, physical address, email address, and other information associated with the individual / entity. In some embodiments, the service provider server 130 can use edges to represent transactions conducted between two user accounts. Thus, when a transaction is conducted between two user accounts represented by two vertices, an edge may be generated to connect the two vertices within the graph data structure 142 .

[0038] In some embodiments, the service provider server 130 may use vertices to represent other assets or attributes related to the service provider server 130. For example, the service provider server 130 may also use vertices to represent assets or attributes that may be shared by different user accounts, such as network addresses (e.g., Internet Protocol addresses, email addresses, etc.), device identifiers, physical locations, sources of funds (e.g., credit card accounts, debit card accounts, etc.), and other data related to payment transactions. When vertices are used to represent different types of entities and / or assets, the vertices may also be connected via edges based on different types of relationships. For example, as shown above, when an edge connects two vertices representing that two user accounts have conducted one or more transactions with each other, the edge may represent a transaction relationship.

[0039] In another example, when a user account uses a specific device to conduct a transaction, based on the usage relationship, an edge can connect the vertex representing the user account and the vertex representing the specific device (or specific device identifier) ​​in the graph data structure 142. When two user accounts use the same device to conduct a payment transaction, the two vertices representing the two user accounts will be connected to the same vertex representing the device in the graph data structure 142.

[0040] Similarly, when a specific funding instrument (e.g., a specific credit card, etc.) is used to conduct a transaction through a user account, based on the usage relationship, an edge may connect the vertex representing the user account and the vertex representing the specific funding instrument in the graph data structure 142. When two user accounts conduct a payment transaction using the same financial instrument, the two vertices representing the two user accounts will be connected to the same vertex representing the financial instrument in the graph data structure 142.

[0041] In various embodiments, the service provider server 130 also includes a feature engineering module 132 that implements a feature engineering system as discussed herein. Specifically, the feature engineering module 132 can assist a user in identifying and evaluating various feature candidates for use in one or more machine learning models. For example, the feature engineering module 132 can access the graph data within the graph data structure 142. Based on the graph data, the feature engineering module 132 can generate a representation (e.g., a graphical representation) of at least a portion of the graph data structure 142, and can present the representation via a user interface of a device (e.g., device 190, which can be a personal computer, a mobile device, or a computer server). The feature engineering module 132 can enable a user to define feature candidates based on input provided via a user interface. The input may include user interaction with a representation of the graph data structure 142 displayed on the user interface. Based on these inputs, the feature engineering module 132 can generate a definition of feature candidates for use in one or more machine learning models.

[0042] Figure 2 An example graph 200 based on the graph data structure 142 according to various embodiments of the present disclosure is illustrated. In some embodiments, the graph 200 may represent a portion or all of the graph data structure 142. As shown, the graph 200 includes a plurality of vertices 202, 204, 206, 208, 210, 212, and 214 representing various entities, attributes, data, and / or assets. For example, the vertices 202, 204, 206, 208, and 210 represent various user accounts associated with the service provider server 130 (e.g., a user account associated with the user 140, a user account associated with the user of the user devices 170 and 180, etc.). The vertex 212 represents a physical address, and the vertex 214 represents a financial instrument (e.g., a credit card, a debit card, a gift card, a bank account, etc.).

[0043] Graph 200 also includes a plurality of edges 222, 224, 232, 234, 236, 238, and 240 connecting the vertices in graph 200. Different edges in graph 200 may represent different types of relationships. For example, edges 222 and 224 represent transaction relationships between user accounts. Specifically, edge 222 represents one or more transactions conducted between user accounts represented by vertices 202 and 204, and edge 224 represents one or more transactions conducted between user accounts represented by vertices 202 and 208. Edges 232, 234, and 236 may represent residential relationships between user accounts and residential addresses represented by vertex 212. Specifically, edge 232 indicates that the user of the user account represented by vertex 206 lives at the address represented by vertex 212. Similarly, edge 234 indicates that the user of the user account represented by vertex 208 lives at the address represented by vertex 212, and edge 236 indicates that the user of the user account represented by vertex 204 lives at the address represented by vertex 212. By analyzing graph 200, it may be determined that the user accounts represented by vertices 204, 206, and 208 are related to each other because the users of these user accounts reside at the same address.

[0044] Edges 238 and 240 represent usage relationships between user accounts and the financial instruments represented by vertex 214. Specifically, edge 238 indicates that the user of the user account represented by vertex 210 has traded using the financial instrument represented by vertex 214. Similarly, edge 240 indicates that the user of the user account represented by vertex 202 has traded using the same financial instrument represented by vertex 214. By analyzing graph 200, it can be determined that the user accounts represented by vertices 202 and 210 are related to each other because the users of these user accounts have traded using the same financial instrument.

[0045] In some embodiments, the representation of the graph data structure 142 generated by the feature engineering module 132 may look similar to Figure 2 Based on viewing and analyzing graph 200, a user (e.g., a user of device 190) can determine various relationships between different entities and / or assets related to service provider server 130, and can identify feature candidates that one or more machine learning models can use to perform tasks for service provider server 130.

[0046] The service provider server 130 may use one or more machine learning models to perform different tasks related to the operation of the service provider server 130. For example, since the service provider server 130 is configured to manage user accounts of users and process electronic payment transactions (and other electronic services) through user accounts, the service provider server 130 may generate and / or configure a machine learning model to predict the risk associated with the payment transaction. The predicted risk may be used by the service application 138 to process the payment transaction (e.g., authorize the payment transaction when the risk is below a threshold, deny the payment transaction when the risk exceeds a threshold, etc.). In another example, the service provider server 130 may generate and / or configure another machine learning model to predict the risk associated with the user account. The predicted risk may be used by the service application 138 to modify the access level of different user accounts (e.g., when the risk exceeds a threshold, limit certain functions, such as payment transactions exceeding a certain amount or certain data access).

[0047] In order to configure the machine learning model to perform a task, a set of input features is determined to be relevant to performing the task. The set of input features can be associated with attributes stored by the service provider server 130 (e.g., stored in the account database 136, the graph data structure 142, etc.), or associated with attributes that can be calculated based on other attributes stored by the service provider server 130. For example, an input feature of a machine learning model configured to predict the risk of an electronic payment transaction can be an average transaction amount associated with a user account. Since the service provider server 130 stores transaction data for user accounts (e.g., stored in the account database 136 and / or the graph data structure 142, associated with edges representing transaction relationships between user accounts, etc.), a value corresponding to the input feature can be calculated based on the stored transaction data (e.g., calculating the average transaction amount of transactions conducted through the user account, etc.).

[0048] The machine learning model is then trained using training data corresponding to the set of input features. By learning patterns based on the training data, the trained machine learning model can then use the learned patterns to perform tasks (e.g., predicting the risk of a user account, predicting the risk of an electronic payment transaction, etc.). In this way, selecting high-quality input features (also known as "feature engineering") is critical to the performance of the machine learning model in performing the task (e.g., how accurately the machine learning model can predict risk). In addition, the types of features that are relevant to performing the task may change over time as trends come and go. As such, it is very important to frequently re-evaluate the input features of the machine learning model and identify any new features that may be relevant to the task so that the machine learning model can keep up with the changes. Determining which input features to re-evaluate and when to re-evaluate can be based on various factors, including but not limited to when new input features are identified, when the accuracy of the prediction drops below a certain threshold, or when a new type of prediction is required.

[0049] In some embodiments, the feature engineering module 132 can assist a user (e.g., a user of the device 190) in the feature engineering process by enabling the user to identify feature candidates, evaluate feature candidates, and incorporate feature candidates into one or more machine learning models via a user interface provided on the device without requiring the user to enter any programming code. For example, the feature engineering module 132 can access graph data associated with the graph data structure 142 by using a graph query language. The graph data may include information associated with vertices and edges within the graph data structure 132. Based on the graph data, the feature engineering module 132 can construct a graphical representation of at least a portion of the graph data structure 142. The feature engineering module 132 can present the graphical representation on a user interface of the device (e.g., the device 190). In some embodiments, the graphical representation may look similar to Figure 2 A graph 200 including icons representing vertices and lines representing edges of the graph data structure 142 is shown.

[0050] In some embodiments, the feature engineering module 132 can construct a graphical representation of a portion of the graph data structure 142 based on one or more parameters provided by a user. For example, a user can provide an identifier of a seed account (e.g., a user account associated with the service provider server 130) and a number of hops from the seed account via a user interface on the device 190. Based on the one or more parameters, the feature engineering module 132 can first access the graph data associated with the vertex representing the seed account (e.g., the seed vertex). The feature engineering module 132 can then traverse the graph data structure 142 from the seed vertex and obtain graph data associated with edges and other vertices from the traversal of the graph data structure 142. In some embodiments, the feature engineering module 132 can limit the traversal of the graph data structure 142 according to the number of hops specified in the one or more parameters. In one specific example, Figure 2 The graph 200 may be constructed by the feature engineering module 132 based on the seed account represented by the vertex 202 and the two-hop traversal. Thus, the graph 200 includes the vertex 202 representing the seed account, and edges and other vertices within two hops from the vertex 202 (e.g., vertices 212 and 214 within one hop from the seed vertex 202, and vertices 204, 206, 208, and 210 within two hops from the seed vertex 202).

[0051] By viewing and analyzing the graphical representation of graph data structure 142, a user of device 190 may identify attributes that may be obtained from graph data structure 142 and that may be relevant to performing tasks associated with a machine learning model. For example, a user may determine that the residential address of a seed account would be a feature candidate for predicting the risk of a seed account. A user may also determine that the average transaction amount associated with other user accounts that share a financial instrument with the seed account would be a feature candidate for predicting the risk of a seed account.

[0052] Traditionally, a user may have to communicate the logic associated with a feature candidate to a professional software programmer proficient in a graph query language (e.g., Gremlin, Cypher, etc.) in order to implement the feature candidate for the graph data structure 142 for testing and evaluation, and ultimately incorporation into one or more machine learning models. However, as discussed herein, the traditional feature engineering process based on back-and-forth communication between data scientists and software programmers may be inefficient and error-prone. Thus, the feature engineering module 132 may enable a user to define the logic of a feature candidate via a user interface, and automatically implement the logic of the feature candidate for a graph data structure (e.g., graph data structure 142).

[0053] In some embodiments, the user can provide input indicating how to traverse from the seed vertex to obtain data for feature candidates. For example, the user can indicate the number of hops to traverse from the seed vertex and the type(s) of edges to traverse from the seed vertex. Figure 3 An example user interface 300 provided by the feature engineering module 132 according to various embodiments of the present disclosure is illustrated, which enables a user to define feature candidates for one or more machine learning models. The user interface 300 may be provided on a user device (e.g., device 190, etc.) associated with the service provider server 130 (e.g., a device operated by an agent of a service provider associated with the service provider server 130). During the feature engineering process of a user defining a feature candidate, the feature engineering module 132 may present an icon (e.g., icon 302) representing an arbitrary seed vertex. The icon may not correspond to any particular vertex in the graph data structure 142, but is used as a tool for defining attributes associated with the feature candidate.

[0054] Via the user interface 300, the user of the device 190 can indicate the number of hops to be traversed from the seed vertex. For example, the user can select an icon 302 displayed on the user interface 300. After receiving the selection of the icon 302, the feature engineering module 132 can enable the user to specify the number of hops to be traversed from the seed vertex. In one example, the feature engineering module 132 can display a drop-down menu 304, which provides different options associated with different hops. Then, the user can select one of the options from the drop-down menu 304 to specify the number of hops to be traversed from the seed vertex. In a specific example, when the user wishes to define a feature candidate associated with the average transaction amount of accounts sharing the same residential address with the same seed account, the user can specify a two-hop traversal from the seed vertex for the feature candidate-a first hop traversal from the seed vertex to the vertex representing the residential address associated with the seed account, and a second hop traversal from the vertex representing the residential address to one or more vertices representing other accounts sharing the same residential address associated with the seed account.

[0055] Figure 4The user interface 300 is illustrated after a user specifies the number of hops to traverse from a seed vertex according to various embodiments of the present disclosure. In some embodiments, after receiving user input indicating the number of hops to traverse from a seed vertex, the feature engineering module 132 may enable the user to specify which edge type(s) (e.g., relationship type) may be traversed from the seed vertex via which the graph data structure 142 may be traversed. For example, the feature engineering module 132 may analyze the edges present in the graph data structure 142, and may determine different types of edges (e.g., different types of relationships) associated with the edges. The types of edges (e.g., relationships) included in the graph data structure 142 may include a "payment transaction" relationship representing one or more payment transactions conducted between two accounts, a "fund instrument use" relationship representing one or more payment transactions conducted through a user account using a specific funding instrument, a "residence" relationship representing a location where a user associated with the user account resides, and other types of relationships.

[0056] The feature engineering module 132 can then present different types of relationships as options via the user interface 300. In this example, the feature engineering module 132 can present a window 402 that includes different check boxes associated with different relationships found in the graph data structure 142. The user can select the type(s) of edges to traverse from the seed vertex 302 by selecting one or more check boxes within the window 402. Using the above example, the user can select the "residential" relationship for the first hop traversal. However, although not shown in this example, other options associated with edge characteristics can also be presented and selected by the user, such as the time when the edge was created (e.g., a time range), the minimum (or maximum) number of connections between two vertices (e.g., indicating the minimum (or maximum) number of transactions conducted between the two vertices), or other edge characteristics.

[0057] Figure 5 The user interface 300 after the user specifies the type of edge traversed from the seed vertex according to various embodiments of the present disclosure is illustrated. As shown, the feature engineering module 132 presents, in addition to the icon 302 representing the seed vertex, an icon 502 representing one or more first-hop vertices and a dotted line 504 connecting the icons 302 and 502. The dotted line 504 represents one or more first-hop edges for traversing from the seed vertex to the one or more first-hop vertices. Since the icon 302 represents an arbitrary seed vertex (which can be any vertex in the graph data structure 142), during the feature engineering process, it is impossible to determine how many vertices can be reached based on a one-hop traversal from the seed vertex at this time. Thus, the icon 502 represents any vertex that can be reached from any seed vertex based on a one-hop traversal using an edge of the (one or more) type represented by the dotted line 504 and indicated by the user.

[0058] If the user only specifies a one-hop traversal of the feature candidate, the feature engineering process can end here. However, if the user specifies additional hops to traverse for the feature candidate (e.g., two hops, three hops, etc.), the user can continue to specify the type(s) of edges to traverse for the subsequent hop(s) (e.g., the second hop, the third hop, etc.) in the same manner as disclosed herein. Using the above example, the user can select the "residence" relationship for the second hop edge from the seed vertex, so that based on the traversal from the seed vertex, the vertex representing the user account that shares the same residential address as the seed account can be reached.

[0059] After defining the traversal logic for the feature candidates, the feature engineering module 132 can enable the user to define, via the user interface 300, the type of attribute associated with the target vertex to be obtained, and the mathematical operation (e.g., addition, subtraction, average, etc.) used to calculate the value corresponding to the feature candidate. Using the above example, the user can specify to obtain data corresponding to the transaction amount attribute associated with each target vertex. The data can indicate the transaction amount associated with the transaction conducted in the past through the user account represented by the target vertex. The user can also specify an "average" operation to calculate the value corresponding to the feature candidate based on the data.

[0060] In some embodiments, based on feature candidates defined by a user via the user interface 300 , the feature engineering module 132 may implement the feature candidates, evaluate the feature candidates, and incorporate the feature candidates into one or more machine learning models as one of the input features of the one or more machine learning models. Figure 6The diagram illustrates a computing environment 600 within which the feature engineering module 132 can implement, evaluate, and incorporate feature candidates into one or more machine learning models. In some embodiments, based on user input received from a user of the device 190, the feature engineering module 132 can generate programming code that implements logic associated with feature candidates in a graph query language (e.g., Gremlin, Cypher, etc.). The programming code, when executed based on a specified seed vertex (e.g., when an identifier of a vertex is provided as an input value, etc.), may cause the machine to traverse a graph data structure (e.g., graph data structure 142) from the specified seed vertex using user-specified traversal logic, obtain data associated with a target vertex and / or edge, and calculate a value corresponding to the feature candidate based on the data. For example, when a feature candidate is associated with the average transaction amount of transactions conducted through (one or more) user accounts that share the same residential address as the seed account, the programming code may cause the machine to jump (e.g., the first jump) from the seed vertex to the adjacent vertex representing the residential address, and then jump (e.g., the second jump) from the vertex representing the residential address to one or more other vertices (e.g., the target vertex) representing other accounts associated with the residential address. The programming code may further cause the machine to obtain data from the vertices and / or edges along the traversed path. For example, the programming code may cause the machine to obtain the transaction amount of transactions conducted through the user account represented by the target vertex. The programming code may then cause the machine to calculate a value corresponding to the feature candidate based on the data (e.g., calculating an average based on the transaction amount).

[0061] The feature engineering module 132 may store the programming code in the data storage device 610 in association with the feature candidate so that the programming code can be retrieved and executed when needed. In some embodiments, the feature engineering module 132 may perform simulations on the feature candidate based on different seed accounts (e.g., different seed vertices). In this way, the feature engineering module 132 may execute the programming code associated with the feature candidate based on different seed vertices in the graph data structure 142. By executing the programming code based on different seed vertices in the graph data structure 142, different values ​​(e.g., average transaction amounts) corresponding to the feature candidate may be calculated for different seed accounts. In some embodiments, the feature engineering module 132 may present the results of the simulation on a user interface (e.g., user interface 300). In some embodiments, the feature engineering module 132 may evaluate the feature candidate based on the simulation results (e.g., values ​​corresponding to the feature candidate determined based on different seed accounts) and the tasks performed by the machine learning model. For example, when generating feature candidates for a machine learning model 602 configured to predict the risk of a user account, the feature engineering module 132 may determine whether there is a correlation between the value corresponding to the feature candidate generated from the simulation and the risk score previously determined for each seed account used in the simulation. In some embodiments, when the feature engineering module 132 determines that there is a correlation between the value corresponding to the feature candidate and the risk score (or based on confirmation from the user of the device 190), the feature engineering module 132 can use the techniques described herein to incorporate the feature candidate into the machine learning model 602 as one of the input features of the machine learning model 602. In some embodiments, the feature engineering module 132 can analyze the simulation results based on the designation provided by the user via the user interface. For example, the user can specify a set of requirements and / or definitions for the correlation between the value corresponding to the feature candidate and the risk score.

[0062] In some embodiments, the feature engineering module 132 may add input features to the machine learning model 602 by modifying the internal structure of the machine learning model 602. For example, if the machine learning model 602 is implemented as an artificial neural network, the feature engineering module 132 may modify the input layer of the artificial neural network (e.g., adding additional input nodes in the input layer, etc.).

[0063] In some embodiments, the feature engineering module 132 may also incorporate programming code into the machine learning model 602, so that when the machine learning model 602 is used by another software module (e.g., a risk assessment module of the service provider server 130, etc.), the machine learning model 602 may use the programming code to retrieve input values ​​corresponding to feature candidates to perform tasks.

[0064] In some embodiments, the feature engineering module 132 can also generate new training data to train the machine learning model 602 based on the data retrieved from the graph data structure 142 and / or the account database 136. The new training data may include values ​​corresponding to the feature candidates, so that the modified machine learning model 602 can be trained to learn patterns based on the feature candidates (e.g., new input features) and the existing input features. The feature engineering module 132 can then train the machine learning model 602 using the new training data.

[0065] In some embodiments, feature engineering module 132 may keep programming code under its control rather than incorporating programming code into machine learning model 602 and / or other software modules. For example, feature engineering module 132 may establish a protocol with machine learning model 602 such that whenever machine learning model 602 is requested to perform a task (e.g., provide a risk prediction for a user account, etc.), machine learning model 602 will request (e.g., via an application programming interface (API) call, etc.) feature engineering module 132 to obtain a value corresponding to a feature to be used as an input value to machine learning model 602.

[0066] The ability of the feature engineering module 132 to store and manage programming code associated with various input features for the machine learning model 602 enables flexibility regarding the implementation of various input features for the machine learning model 602. For example, the feature engineering module 132 can modify the implementation of various input features of the machine learning model 602 without modifying the machine learning model 602. In some embodiments, the feature engineering module 132 can re-evaluate existing input features of the machine learning model 602 (e.g., automatically or upon request from a user of the device 190), and can also identify and evaluate new feature candidates for the machine learning model 602 (e.g., periodically, etc.). Based on the effectiveness of the input features and feature candidates (e.g., the correlation between the input features and the task, etc.), the feature engineering module 132 can change the input features of the machine learning model 602 to improve the performance of the machine learning model 602 based on the evaluation. The change can include replacing existing input features with new features, adjusting parameters of existing input features (e.g., adjusting different weights that can be applied to different values ​​to calculate the input features, etc.), adjusting the implementation of existing input features (e.g., adding or removing the number of hops traversed from the seed vertex, etc.), and the like. The feature engineering module 132 may change the input features by modifying and / or replacing programming code associated with the input features and stored in a data storage device 610 associated with the feature engineering module 132 without requiring any changes to the machine learning model 602.

[0067] In some embodiments, the feature engineering module 132 can manage existing input features and use the same techniques as disclosed herein to assist the user of the device 190 in creating new features for other machine learning models (e.g., machine learning models 604 and 606). The machine learning models 604 and 606 can be configured to perform tasks different from those performed by the machine learning model 620. For example, the machine learning model 604 can be configured to predict the risk associated with electronic transactions, while the machine learning model 606 can be configured to predict product recommendations for users. In this way, the feature engineering module 132 can determine feature candidates for each of the machine learning models 602, 604, and 606 based on user input via the user interface 300. The feature engineering module 132 can generate programming code for each feature candidate, and can evaluate the feature candidates by performing simulations on the graph data structure 142. The feature engineering module 132 can also incorporate one or more feature candidates into the machine learning models 602, 604, and 606 (by changing the internal structure of the machine learning models 602, 604, and 606 or changing the configuration files of the machine learning models 602, 604, and 606).

[0068] By assisting a user of the device 190 (e.g., a data scientist of a service provider associated with the service provider server 130) to define and evaluate feature candidates, and incorporating the selected feature candidates into various machine learning models, the process of feature engineering becomes more efficient and less error-prone. For example, a user (e.g., a data scientist) can perform the entire feature engineering process without requiring the participation of other personnel (e.g., a professional software programmer). In addition, by managing the features of various machine learning models outside of the machine learning model (e.g., within the feature engineering module 132, etc.), the input features of the machine learning model can be seamlessly and dynamically modified without accessing and / or adjusting the internal structure of the machine learning model. Since the input features of the machine learning model may be frequently changed to maintain and / or improve the performance of the machine learning model, the feature engineering module 132 can continuously re-evaluate existing input features and evaluate new feature candidates for the machine learning model, and implement changes to the input features of the machine learning model without accessing and / or adjusting the internal structure of the machine learning model.

[0069] Figure 7A process 700 for providing tools to assist a user in defining, evaluating, and incorporating features into a machine learning model according to various embodiments of the present disclosure is illustrated. In some embodiments, at least a portion of the process 700 may be performed by a feature engineering module 132. The process 700 may begin (at step 705) by presenting a graphical representation of a graph on a user interface. For example, the feature engineering module 132 may access graph data associated with a graph data structure 142, and may generate a representation of at least a portion of the graph data structure 142. The feature engineering module 132 may then present the representation on the user interface 300 of the device 190. The representation may include vertices and edges of the graph data structure 142.

[0070] Then, process 700 (at step 710) receives user input via the user interface, the user input including manipulation of the representation. For example, a user of device 190 may provide user input via user interface 300. The user input may include manipulation of the representation of graph data structure 142 (e.g., selecting one or more vertices in the representation). Based on the manipulation of the representation, different graph data associated with different vertices and / or edges within graph data structure 142 may be presented via user interface 300. The user may also perform an action via user interface 300 to indicate a desire to define a new feature candidate. In some embodiments, upon detecting an action via user interface 300, feature operation module 132 may present an icon on user interface 300 that represents any seed vertex of graph data structure 142. The user may continue to provide input based on the icon to specify logic associated with feature candidates. The logic may include traversal logic, such as the number of hops traversed from the seed vertex during traversal, and the type (or types) of edges traversed during traversal.

[0071] Process 700 determines a feature based on the user input (at step 715), the feature being defined by a calculation based on one or more properties associated with one or more hops from a seed vertex within the graph. For example, feature engineering module 132 may implement logic for feature candidates using a graph query language (e.g., generating programming code in the graph query language, etc.). Feature engineering module 132 may store the programming code in data storage device 610.

[0072] Then, the process 700 performs simulation based on the features (at step 720), and incorporates the features into the machine learning model based on the simulation results (at step 725). Using the programming code generated for the feature candidate, the feature engineering module 132 can traverse the graph data structure 142 and calculate the value corresponding to the feature candidate based on the given seed vertex. In this way, the feature engineering module 132 can perform multiple simulations by executing the programming code based on different vertices (as seed vertices) in the graph data structure 142. By executing the programming code, the value corresponding to the feature candidate can be calculated for the corresponding seed vertex. The feature engineering module 132 can also determine the label corresponding to the machine learning model based on the seed vertex (e.g., the risk score determined for the user account represented by the seed vertex, etc.). Then, the feature engineering module 132 can determine whether there is a correlation between the calculated value corresponding to the feature candidate and the label. If there is a correlation (or user request), the feature engineering module 132 can incorporate the feature candidate into the machine learning model 602 as an input feature of the machine learning model 602.

[0073] Figure 8 800 is a block diagram of a computer system 800 suitable for implementing one or more embodiments of the present disclosure, including a service provider server 130, user devices 110, 170, 180, and device 190. In various implementations, each of the devices 110, 170, 180, and 190 may include a mobile cellular phone suitable for wireless communication, a personal computer (PC), a laptop, a wearable computing device, etc., and the service provider server 130 may include a network computing device, such as a server. Thus, it should be understood that the devices / servers 110, 130, 170, 180, and 190 may be implemented as the computer system 800 in the following manner.

[0074] The computer system 800 includes a bus 812 or other communication mechanism for communicating information data, signals, and information between the various components of the computer system 800. These components include an input / output (I / O) component 804, which processes user (i.e., sender, receiver, service provider) actions, such as selecting a key from a keypad / keyboard, selecting one or more buttons or links, etc., and sends corresponding signals to the bus 812. The I / O component 804 may also include output components, such as a display 802 and a cursor control 808 (e.g., a keyboard, a keypad, a mouse, etc.). The display 802 may be configured to present a login page for logging into a user account or a checkout page for purchasing items from a merchant. An optional audio input / output component 806 may also be included to allow a user to input information using voice by converting audio signals. The audio I / O component 806 may allow a user to hear audio. The transceiver or network interface 820 communicates with the user via a network 822 (e.g., Figure 1 The computer system 800 and the communication link 824 may be used to transmit and receive signals between the computer system 800 and other devices (e.g., another user device, a merchant server, or a service provider server) using a network 160. In one embodiment, the transmission is wireless, although other transmission media and methods may also be suitable. The processor 814, which may be a microcontroller, a digital signal processor (DSP), or other processing component, processes these various signals, such as for display on the computer system 800 or for transmission to other devices via the communication link 824. The processor 814 may also control the transmission of information (e.g., cookies or IP addresses) to other devices.

[0075] The components of the computer system 800 also include a system memory component 810 (e.g., RAM), a static storage component 816 (e.g., ROM), and / or a disk drive 818 (e.g., solid-state drive, hard disk drive). The computer system 800 performs specific operations by the processor 814 and other components by executing one or more instruction sequences contained in the system memory component 810. For example, the processor 814 can perform the feature engineering functions described herein according to the process 700.

[0076] The logic may be encoded in a computer readable medium, which may refer to any medium that participates in providing instructions to the processor 814 for execution. Such media may take many forms, including but not limited to non-volatile media, volatile media, and transmission media. In various implementations, non-volatile media include optical or magnetic disks, volatile media include dynamic memory, such as the system memory component 810, and transmission media include coaxial cables, copper wires, and optical fibers, including the wires that make up the bus 812. In one embodiment, the logic is encoded in a non-transitory computer readable medium. In one example, the transmission medium may take the form of sound waves or light waves, such as those generated during radio waves, optical communications, and infrared data communications.

[0077] Some common forms of computer readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media with patterns of holes, RAM, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or cartridges, or any other media suitable for reading by a computer.

[0078] In various embodiments of the present disclosure, execution of the sequences of instructions implementing the present disclosure may be performed by a computer system 800. In various other embodiments of the present disclosure, multiple computer systems 800 coupled to a network (e.g., such as a LAN, WLAN, PSTN, and / or various other wired or wireless networks, including telecommunications, mobile, and cellular telephone networks) by a communication link 824 may execute the sequences of instructions to implement the present disclosure in coordination with each other.

[0079] When applicable, the various embodiments provided by the present disclosure may be implemented using hardware, software, or a combination of hardware and software. In addition, when applicable, the various hardware components and / or software components recorded herein may be combined into composite components including software, hardware, and / or both, without departing from the spirit of the present disclosure. When applicable, the various hardware components and / or software components recorded herein may be separated into subcomponents including software, hardware, or both, without departing from the scope of the present disclosure. In addition, when applicable, it is contemplated that software components may be implemented as hardware components, and vice versa.

[0080] According to the present disclosure, software, such as program code and / or data, may be stored on one or more computer-readable media. It is also contemplated that the software identified herein may be implemented using one or more general or special-purpose computers and / or computer systems, networked and / or otherwise. Where applicable, the ordering of the various steps described herein may be changed, combined into composite steps, and / or separated into sub-steps to provide the features described herein.

[0081] The various features and steps described herein may be implemented as a system comprising one or more memories storing the various information described herein and one or more processors coupled to the one or more memories and a network, wherein the one or more processors are operable to perform the steps as described herein; as a non-transitory machine-readable medium comprising a plurality of machine-readable instructions that, when executed by one or more processors, are suitable for causing the one or more processors to perform a method comprising the steps described herein; and as a method performed by one or more devices, such as hardware processors, user devices, servers, and other devices described herein.

Claims

1. A system comprising: non-transitory memory; as well as One or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations including: providing a user interface on a device based on graph data associated with the graph; receiving user input via the user interface; Based on the user input, determining feature candidates for a machine learning model configured to perform a task, wherein the feature candidates are associated with: a traversal of the graph starting from a seed vertex and a computation based on one or more properties associated with vertices along the traversal; and The machine learning model is configured to perform the task using the feature candidates as input features.

2. The system of claim 1, wherein: The operations also include: Computer programming code is generated to implement the feature candidates.

3. The system of claim 2, wherein: The operations also include: traversing the graph starting from a particular vertex within the graph based on executing the computer programming code; Based on the traversal, obtaining one or more attribute values; and For a specific user account corresponding to the specific vertex, a value corresponding to the feature candidate is calculated based on the one or more attribute values.

4. The system of claim 2, wherein: The operations also include incorporating the computer programming code into the machine learning model.

5. The system of claim 1, wherein: The operations also include: performing a plurality of simulations on the feature candidates based on using different vertices in the graph as the seed vertices; and Based on simulation results from the executing, correlations between the feature candidates and the tasks are determined.

6. The system of claim 5, wherein: The operations also include: Determining that the correlation exceeds a threshold, wherein configuring the machine learning model to use the feature candidate as an input feature is responsive to determining that the correlation exceeds the threshold.

7. The system of claim 1, wherein: The user input specifies the number of hops to be traversed from the seed vertex for the feature candidate.

8. A method comprising: receiving, by the one or more hardware processors via a user interface of the device, a user interaction with a graphical element, the graphical element representing at least a portion of a graph associated with a service provider, wherein the graph includes a plurality of vertices and a plurality of edges; determining, by the one or more hardware processors, feature candidates for a machine learning model configured to perform a task based on a user interaction with the graphical element, wherein the feature candidates are associated with: a traversal of the graph starting from a seed vertex and a computation based on one or more properties associated with vertices along the traversal; and The one or more hardware processors configure the machine learning model to perform the task using the feature candidates as input features.

9. The method of claim 8, wherein: The calculation is based on at least one of a sum, an average, a maximum, a minimum, or a count.

10. The method of claim 8, wherein: The user interaction specifies the type of edges to be traversed from the seed vertex.

11. The method of claim 8, further comprising: designating a specific vertex from among a plurality of vertices of the graph as the seed vertex; Based on traversing the graph from the specific vertex, calculating a value corresponding to the feature candidate; and The value is provided to the machine learning model.

12. The method of claim 11, further comprising: receiving a request to perform the task based on a specific user account associated with the service provider; and A determination is made that the particular user account is represented by the particular vertex, wherein the designation of the particular vertex as the seed vertex is in response to a determination that the particular user account is represented by the particular vertex.

13. The method of claim 8, further comprising: Computer programming code is generated for the feature candidates, the code, when executed, calculating values ​​corresponding to the feature candidates for different user accounts associated with the service provider.

14. The method of claim 13, wherein: The computer programming code is associated with a graph query language.

15. A non-transitory machine-readable medium having machine-readable instructions stored thereon, the instructions being executable to cause a machine to perform operations, the operations comprising: accessing graph data associated with a graph from a data storage device, wherein the graph represents relationships between a plurality of user accounts associated with a service provider; providing a user interface on a device based on the graph data; receiving user input via the user interface; Based on the user input, determining feature candidates for a machine learning model configured to perform a task, wherein the feature candidates are associated with: a traversal of the graph starting from a seed vertex and a computation based on one or more properties associated with vertices along the traversal; and The machine learning model is configured to perform the task using the feature candidates as input features.

16. The non-transitory machine-readable medium of claim 15, wherein: The operations also include: Computer programming code is generated for the feature candidates for calculating values ​​corresponding to the feature candidates for different user accounts associated with the service provider.

17. The non-transitory machine-readable medium of claim 16, wherein: The operations also include: Based on executing the computer programming code, traverse the graph from a specific vertex within the graph to obtain one or more property values ​​corresponding to the one or more properties; and For a specific user account corresponding to the specific vertex, a value corresponding to the feature candidate is calculated based on the one or more attribute values.

18. The non-transitory machine-readable medium of claim 16, wherein: The operations also include incorporating the computer programming code into the machine learning model.

19. The non-transitory machine-readable medium of claim 1, wherein: The operations also include: performing a plurality of simulations on the feature candidates based on using different vertices in the graph as the seed vertices; and Based on simulation results from executing the plurality of simulations, it is determined whether there is a correlation between the feature candidates and the task.

20. The non-transitory machine-readable medium of claim 19, wherein: The operations also include: It is determined based on the simulation result that the correlation exists between the feature candidate and the task, wherein configuring the machine learning model to use the feature candidate as an input feature is based on the correlation.

Images