Early pattern detection in data for improved business operations

Abstract

To provide an early pattern detection platform to improve operations of enterprises, and a computer-implemented method for generating events and actions.SOLUTION: A method includes: receiving a goal; providing a problem-specific knowledge graph that is responsive to at least a portion of the goal; determining a set of events from the problem-specific knowledge graph; processing data representative of events in the set of events associated with individual events in the set of events through a first machine learning (ML) model to provide a set of event scores; determining a subset of events based on the set of event scores; for each event in the subset of events, determining at least one action by processing a sequence of actions through a second ML model; and outputting the subset of events and a set of actions for execution of at least one action in the set of actions.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] Technical Field This application relates generally to an early pattern detection platform for improving business operations and a computer-implemented method for generating events and actions based on pattern recognition in data from connected networks.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 164,159, filed March 22, 2021, the entire disclosure of which is expressly incorporated herein by reference. [Background technology]

[0003] Businesses operate in connected environments that may include networks of multiple businesses and customers. By way of example, businesses may operate in a business-to-business (B2B) context and / or a business-to-consumer (B2C) context, among other contexts. In a B2B context, businesses provide goods and / or services to other businesses. In a B2C context, businesses provide goods and / or services to individuals.

[0004] In today's digitalized and connected world, interactions between businesses and between businesses and customers span multiple networks and multiple platforms. Furthermore, such connectivity results in increased interaction dynamics and the generation of vast amounts of data representing the interactions. As an example, in a B2C context, a business may offer its available products across multiple e-commerce platforms, and each e-commerce platform provides product reviews that can affect the business's operations. As an example, product reviews can affect not only the business offering the product, but also businesses in the supply chain that offer the product. That is, interactions in a B2C context can affect a B2B context and vice versa.

[0005] With this in mind, companies analyze data in an attempt to detect dynamics (e.g., emerging market trends) that may affect their operations. While various computer-implemented tools have been developed to support so-called big data analytics, these tools suffer from technical shortcomings. For example, traditional tools attempt to detect dependencies between companies, but are unable to glean useful signals from noisy data and / or recognize patterns in instances of incomplete information. Furthermore, the models used by such tools are unable to keep up with the dynamics arising from connectivity and real-time interactions between increasingly connected companies and their customers. Summary of the Invention [Means for solving the problem]

[0006] Implementations of the present disclosure are generally directed to an early pattern detection platform for improving business operations. In particular, implementations of the present disclosure are directed to an early pattern detection platform that builds and updates a knowledge graph, leverages artificial intelligence (AI) to extract web-based data, and detects domain-related patterns based on the knowledge graph and the web-based data. In some implementations, the early pattern detection platform identifies events and outputs actions that can be taken by the business to improve its operations.

[0007] In some implementations, the actions include receiving a goal, providing a problem-specific knowledge graph responsive at least in part to the goal, determining a set of events from the problem-specific knowledge graph, processing data representing events in the set of events through a first machine learning (ML) model to provide a set of event scores, where each event score in the set of event scores is associated with a respective event in the set of events, determining a subset of events based on the set of event scores, determining at least one action for each event in the subset of events by processing a sequence of actions through a second ML model, and outputting the subset of events and the set of actions for performance of at least one action in the set of actions. Other implementations of this aspect include corresponding systems, apparatuses, and computer programs configured to perform the actions of the methods and encoded on a computer storage device.

[0008] Each of these and other implementations may optionally include one or more of the following features: determining the set of events from the problem-specific knowledge graph includes mapping at least a portion of the target entities to nodes in the problem-specific knowledge graph and identifying paths in the problem-specific knowledge graph that include the nodes; the events are determined as distinct instances of at least one node in the problem-specific knowledge graph; the first ML model processes at least a portion of the sparse feature set through an embedding layer and processes the dense feature set through a hidden layer to provide an event score for the distinct event; the second ML model receives a sequence of actions associated with the distinct events and predicts a next action in the sequence of actions for the distinct event; the second ML model includes a set of transformers that process the sequence of actions; and the action further includes, for each event in the set of events, extracting data representing the event through web sensing.

[0009] It should be understood that methods according to the present disclosure can include any combination of the aspects and features described herein, that is, by way of example, the apparatus and methods according to the present disclosure are not limited to the combinations of aspects and features specifically described herein, but can also include any combination of the aspects and features shown.

[0010] The details of one or more implementations of this disclosure are set forth in the accompanying drawings and the description below. Other features and advantages of this disclosure will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0011] [Figure 1] 1 illustrates an example system in which implementations of the present disclosure can be performed. [Figure 2] 1 illustrates an example conceptual architecture including an early pattern detection platform according to an implementation of the present disclosure. [Figure 3A]1 illustrates an example representation of building a knowledge graph and updating parts of it. [Figure 3B] 1 illustrates an example representation of building a knowledge graph and updating parts of it. [Figure 3C] 1 illustrates an example representation of building a knowledge graph and updating parts of it. [Figure 3D] 1 illustrates an example representation of building a knowledge graph and updating parts of it. [Figure 4] 1 illustrates an example architecture for web sensing using artificial intelligence (AI) in accordance with an implementation of the present disclosure. [Figure 5] 1 illustrates an example machine learning (ML) model for scoring events, according to implementations of the present disclosure. [Figure 6] 1 illustrates an example ML model for determining an action for an event, according to implementations of the present disclosure. [Figure 7] 1 illustrates an example process that can be performed by implementations of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The same reference numbers and designations in the various drawings indicate like elements.

[0013] Implementations of the present disclosure are generally directed to an early pattern detection platform for improving business operations. In particular, implementations of the present disclosure are directed to an early pattern detection platform that builds and updates a knowledge graph, leverages artificial intelligence (AI) to extract web-based data, and detects domain-related patterns based on the knowledge graph and the web-based data. In some implementations, the early pattern detection platform identifies events and outputs actions that can be taken by the business to improve its operations. In some implementations, the actions include receiving a goal; providing a problem-specific knowledge graph responsive to at least a portion of the goal; determining a set of events from the problem-specific knowledge graph; processing data representing events in the set of events through a first machine learning (ML) model to provide a set of event scores, wherein each event score in the set of event scores is associated with a respective event in the set of events; determining a subset of events based on the set of event scores; determining at least one action for each event in the subset of events by processing a sequence of actions through a second ML model; and outputting the subset of events and the set of actions for execution of at least one action in the set of actions.

[0014] To provide further context for the implementation of the present disclosure, as presented above, businesses operate in connected environments that may include networks of multiple businesses and customers. By way of example, businesses may operate in a business-to-business (B2B) context and / or a business-to-consumer (B2C) context, among other contexts. In a B2B context, businesses provide goods and / or services to other businesses. In a B2C context, businesses provide goods and / or services to individuals.

[0015] In today's digitalized and connected world, interactions between businesses and between businesses and customers span multiple networks and multiple platforms. Furthermore, such connectivity results in increased interaction dynamics and the generation of vast amounts of data representing the interactions. As an example, in a B2C context, a business may offer its available products across multiple e-commerce platforms, and each e-commerce platform provides product reviews that can affect the business's operations. As an example, product reviews can affect not only the business offering the product, but also businesses in the supply chain that offer the product. That is, interactions in a B2C context can affect a B2B context and vice versa.

[0016] With this in mind, companies analyze data in an attempt to detect dynamics (e.g., emerging market trends) that may affect their operations. While various computer-implemented tools have been developed to support so-called big data analytics, these tools suffer from technical shortcomings. For example, traditional tools attempt to detect dependencies between companies, but are unable to glean useful signals from noisy data and / or recognize patterns in instances of incomplete information. Furthermore, the models used by such tools are unable to keep up with the dynamics arising from connectivity and real-time interactions between increasingly connected companies and their customers.

[0017] In consideration of the above, implementations of the present disclosure provide an early pattern detection platform for improving business operations. In particular, implementations of the present disclosure are directed to an early pattern detection platform that overcomes technical deficiencies of traditional approaches by, for example, building and updating a knowledge graph, leveraging AI to extract web-based data, and detecting domain-related patterns based on the knowledge graph and the web-based data. In some implementations, the early pattern detection platform identifies events from domain-related patterns and outputs actions that can be taken by the business to improve its operations.

[0018] As described in further detail herein, the disclosed early pattern detection platform enables the construction of a knowledge graph representing dependencies between companies and / or between companies and customers, a database of associated symptoms (e.g., representative in the data) provided from network analysis (e.g., web sensing), and the application of inference and pattern recognition models for event identification, enrichment, and prioritization. Implementations of the present disclosure enable users to more effectively and accurately detect subtle patterns in emerging events compared to using manual approaches and / or computer-implemented tools that have technical deficiencies (e.g., that do not effectively handle noisy data and / or incomplete information, or that use models that are less or slow to respond to dynamics). Implementations of the present disclosure further enable seamless adaptation to market trend dynamics (e.g., new emergence of trends) by periodically refreshing the knowledge graph and continuously updating the underlying models.

[0019] Implementations of the present disclosure are described in further detail herein with reference to example contexts. Example contexts include generating leads as events and recognizing patterns to provide actions for event execution to improve business operations. In example contexts, leads can include sales leads (e.g., opportunities to interact with customers) and actions can include offering one or more items to the leads. It is contemplated that implementations of the present disclosure can be implemented in any suitable context. By way of example, although implementations of the present disclosure are described herein with reference to a commercial context, implementations of the present disclosure can be implemented in non-commercial contexts.

[0020] 1 illustrates an example system 100 in which implementations of the present disclosure can be performed. The example system 100 includes a computing device 102, a backend system 108, and a network 106. In some examples, the network 106 includes a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, connecting websites, devices (e.g., the computing device 102), and backend systems (e.g., the backend system 108). In some examples, the network 106 is accessible over wired and / or wireless communication links.

[0021] In some examples, computing device 102 may include any suitable type of computing device, such as a desktop computer, a laptop computer, a handheld computer, a tablet computer, a personal digital assistant (PDA), a mobile phone, a network appliance, a camera, a smartphone, an enhanced general packet radio service (EGPRS) mobile phone, a media player, a navigation device, an email device, a gaming console, or any suitable combination of any two or more of these devices or other data processing devices.

[0022] In the illustrated example, the backend system 108 includes at least one server system 112 and a data store 114 (e.g., a database and a knowledge graph structure). In some examples, the backend system 108 hosts one or more computer-implemented services with which users can interact using computing devices. As an example, the backend system 108 can host an early pattern detection platform according to implementations of the present disclosure. In this example, the users 120 can include entities (e.g., employees) at an enterprise that provide one or more goals as input to the early pattern detection system, and in response, the early pattern detection system provides one or more events and, for each event, one or more actions that can be taken to achieve the goals. Furthermore, in this example, the users 122 can include entities (e.g., employees) at an enterprise that can provide input for creating and / or updating a knowledge graph (KG) used by the early detection platform to provide events and actions.

[0023] As described in further detail herein, the disclosed early pattern detection platform builds a KG of enterprise dependencies, a database of associated symptoms, and applies reasoning and pattern recognition models (ML models) for event generation, enrichment, and prioritization. In the illustrative context, the early pattern detection platform enables users to detect subtle patterns of emerging events more efficiently, effectively, and accurately than can be achieved using traditional tools and / or performed manually.

[0024] 2 illustrates an example conceptual architecture 200 including an early pattern detection platform according to an implementation of the present disclosure. In the example of FIG. 2, the conceptual architecture 200 includes an event generation workbench 202, an event generation engine 204, a goal parsing engine 206, a web sensing module 208, an event scoring module 210, an event action prediction module 212, an event store 220, an action store 222, a KG modeling workbench 214, a graph modeling engine 216, a graph enrichment engine 218, a problem-specific KG store 224, and a domain-specific KG store 226.

[0025] In the context of the non-limiting examples presented above, an event may include a sales lead, also referred to herein as a lead. By way of example, the early pattern detection framework compiles web-based data (e.g., provided by the web sensing module 208), constructs problem-specific and domain-specific KGs (e.g., provided by the graph modeling engine 216 and the graph enrichment engine 218) to recognize the existence of one or more events that one or more businesses can act upon, and predicts one or more actions for the execution of the one or more events (e.g., using the event scoring module 210 and the event prediction module 212).

[0026] More specifically, goals can be provided to the early pattern detection platform through the event generation workbench 202. By way of example, a user 120 can interact with the event generation workbench through a computing device 102 and input a goal. In an example commercial context, an exemplary goal may include, but is not limited to, increasing the number of advertisements that small-to-medium sized businesses (SMBs) place in online video games through a particular advertisement network (ad network). In some examples, goals are provided to the event generation engine 204, which triggers the identification of events and the prediction of actions for each identified event.

[0027] In some implementations, the event generation engine 204 provides the goals to the goal parsing engine 206, which processes the goals and parses them into a set of entities and relationships between the entities. With non-limiting reference to the example goals above, example entities can include an advertisement, an SMB, an advertising network, and an online video game.

[0028] As presented above, the early pattern detection platform detects events based on a KG. In general, a KG can be described as a collection of data, related based on a schema that represents entities and the relationships between them. The data (whether provided in a tabular format) can be logically described as a graph, where each distinct entity is represented by a separate node, and each relationship between a pair of entities is represented by an edge between the nodes. Each edge is associated with a relationship, and the existence of an edge represents the existence of the associated relationship between the nodes connected by that edge. As an example, if node A represents a company called Alpha, node B represents a product called Beta, and edge E is associated with the relationship "is manufactured by," then connecting the nodes in the graph with edge E in the direction from node A to node B represents that Alpha is the company that manufactures Beta. In some cases, the knowledge graph can be extended with knowledge related to the schema (e.g., Alpha is a concept company, Charlie is a concept company, and "supplies to" is a property or relationship between the two entities / instances of the company concept). The addition of information related to the schema supports the evaluation of inference results. Knowledge graphs can be represented by any of a variety of physical data structures. For example, a knowledge graph can be represented by triples, each representing two entities in turn and the relationship from the first to the second. For example, [Alpha, Beta, Manufacturer] or [Alpha, Manufacturer, Beta] are alternative ways of expressing the same fact. Each entity and each relationship can be, and typically will be, contained in multiple triples.

[0029] In some examples, once each entity is stored as a node, e.g., as a record or object, it can be linked to all relationships it has and all other entities it is related to through a linked list data structure. More specifically, a knowledge graph can be stored as an adjacency list in which adjacency information contains relationship information. In some examples, each distinct entity and each distinct relationship is represented using a separate unique identifier. The entities represented by a knowledge graph need not be tangible objects or concrete people. Entities can include specific people, places, things, works of art, concepts, events, or other types of entities. Thus, a knowledge graph can include data defining relationships between companies (e.g., suppliers in a supply chain), data defining relationships between companies and objects (e.g., that a particular product is produced by a particular company), data defining relationships between places and objects (e.g., that a particular product comes from a particular geographic location), data defining relationships between companies and places (e.g., that a company is headquartered in a particular city), and other types of relationships between entities.

[0030] In some implementations, each node has a type based on the kind of entity it represents, and each type can have a schema that specifies the kind of data that can be held for entities represented by nodes of that type and how the data should be stored. As an example, a node of a type representing a company could have a schema that defines fields for information such as location, industry, etc. Such information can be represented by fields in a type-specific data structure, or by triples such as a node-relationship-node triple (e.g., [company identifier, is located, in industry]), or any other convenient pre-defined form. In some examples, some or all of the information specified by a type's schema can be represented by links to nodes in a knowledge graph. For example, in [one company identifier, its subsidiary, another company identifier], the identifier of another company is a node in the graph.

[0031] According to implementations of the present disclosure, exemplary KGs include domain-specific KGs and problem-specific KGs. In some examples, the domain-specific KGs represent entities and relationships between entities in a particular domain (e.g., telecommunications, mobile games, digital advertising). In some examples, the problem-specific KGs represent entities and relationships between entities in a particular domain considering a particular problem to be addressed. In the context of implementations of the present disclosure, a problem is provided as a goal to be achieved in early pattern detection.

[0032] Continuing with reference to Figure 2, one or more domain-specific KGs are provided through the graph modeling engine 216 and stored in the domain-specific KG store 226. In some examples, the domain-specific KGs are created entirely. In some examples, the domain-specific KGs are updated (i.e., an existing domain-specific KG is updated to add / remove nodes and / or edges).

[0033] In some implementations, the domain-specific KG is provided through a knowledge extraction process and a knowledge fusion process. In some examples, the knowledge extraction process is based on multiple data sources, each containing data representing entities and relationships between entities in a particular domain. Exemplary data sources may include, but are not limited to, structured data sources (e.g., relational databases), semi-structured data sources (e.g., extensible markup language (XML) documents, Javascript object notation (JSON) documents), and unstructured data sources (e.g., reports, websites). In some examples, data from each of the multiple data sources is received (e.g., by a knowledge extraction engine), and the knowledge extraction engine processes the data in entity extraction, relationship extraction, and attribute extraction. Entity extraction is performed to identify a set of entities within the data. Exemplary entities may include, but are not limited to, companies (e.g., SMBs, advertising networks, publishers, developers, advertisers) and products (e.g., games, mobile applications (APPs), videos). A relationship extraction process is performed to identify a set of relationships that represent relationships between entities in a set of entities. Example relationships can include, but are not limited to, relationships between companies (e.g., buy from / sell to, developer for / publisher of), and relationships between companies and products (e.g., developer of / producer of). Attribute extraction is performed to assign one or more attributes to each entity and / or relationship.

[0034] In some examples, a set of entities and a set of relationships are processed through a knowledge fusion process to provide (or update) a domain-specific KG. In some examples, a set of entities and a set of relationships are received (e.g., by a knowledge extraction engine from a knowledge extraction engine), which processes the set of entities and / or the set of relationships through entity alignment, data integration, and conflict resolution. In some examples, entity alignment is performed to disambiguate entities and / or eliminate entity duplication. In some examples, data integration is performed to merge the aligned entities and disparate relationships into an existing domain-specific KG to provide an updated version of the domain-specific KG. In some examples, conflict resolution is performed to resolve conflicts between entities and / or relationships and / or filter entities and / or relationships from the domain-specific KG.

[0035] Continuing with reference to Figure 2, one or more problem-specific KGs are provided through the graph enrichment engine 216 and stored in the problem-specific KG store 226. In some examples, the problem-specific KGs are created by enriching the domain-specific KG with problem-specific nodes and relationships and, for each problem-specific node, one or more instances related to the entities represented by the nodes. Examples are described in further detail herein with reference to Figures 3A-3D.

[0036] In some implementations, the creation and / or update of the domain-specific KG and / or the creation and / or update of the problem-specific KG may include, in part, manual input provided by the user 122. By way of example, the user 122 may interact with the KG modeling workbench 214 to provide input for the creation and / or update of the domain-specific KG and / or the creation and / or update of the problem-specific KG. Example inputs may include, but are not limited to, conflict resolution to resolve conflicts between entities and / or relationships.

[0037] 3A-3D show exemplary representations of building a domain-specific KG and updating portions thereof, and providing a problem-specific KG. The examples in FIGS. 3A-3D represent the non-limiting context of an ad network in the digital advertising ecosystem. In some examples, an ad network is a company that connects advertisers with digital media (e.g., websites, apps, digital games, videos) that wish to distribute their ads. The primary role of an ad network is to aggregate ad supply from publishers and match that supply with advertiser demand. The phenomenon of aggregating publisher ad space and selling it to advertisers is most prevalent in the online space, leading to the increasing use of the term "ad network" alone to refer to an "online ad network."

[0038] Referring in detail to Figure 3A, a domain-specific KG 300 (or at least a portion thereof) is shown, which includes an ad network node 302, an advertiser node 304, a publisher node 306, and a developer node 308. The domain-specific KG 300 further includes relationships between the nodes. By way of example, the ad network represented by the ad network node 302 has an "advertising" relationship with the advertiser represented by the advertiser node 304, and the advertiser has a "paying" relationship with the ad network. Thus, the domain-specific KG represents a relationship in which the advertiser pays the ad network to have its promoted advertisements served.

[0039] Figure 3B shows an example of an update to the domain-specific KG 300 to provide a domain-specific KG 300'. In the example of Figure 3B, an ad agency node 310 is added to represent entities (i.e., advertising agencies) that have been added to the online advertising network since the domain-specific KG 300 was created or last updated, and relationships between the advertising agencies and existing entities in the online advertising network. In the example of Figure 3B, the domain-specific KG 300' represents the relationship of the advertising agency paying the advertising network to place advertising advertisements, and the relationship of the advertiser paying the advertising agency to create advertising advertisements.

[0040] 3A to 3B represents an example of updating the domain-specific KG. In some examples, the domain may be dynamic and change relatively frequently. With this in mind, the domain-specific KG may be updated at predefined intervals (e.g., hourly, daily, weekly, monthly) and / or in response to one or more triggers (e.g., user commands).

[0041] 3C and 3D, an example of providing a problem-specific KG will be described. As presented above, a problem-specific KG is provided from a domain-specific KG by enriching the domain-specific KG with problem-specific nodes and, for each problem-specific node, one or more relevant instances of the entities represented by the nodes. In the example of FIG. 3C, a partially enriched domain-specific KG 320 is shown, which is based on the domain-specific KG 300′ of FIG. 3B. In the example of FIG. 3C, the domain-specific KG 300′ is enriched with nodes 312a, 312b representing distinct categories of advertisers and nodes 314a, 314b, 314c representing distinct categories of developers to provide the partially enriched domain-specific KG 320. The partially enriched domain-specific KG 320 is further enriched to include one or more instances of each of nodes 312a, 312b, 314a, 314b, and 314c to provide the problem-specific KG 320′ of FIG. 3D . That is, nodes 312a, 312b, 314a, 314b, and 314c are enriched with instances to provide enriched nodes 312a′, 312b′, 314a′, 314b′, and 314c′. Each instance is a real-world entity related to the respective node's entity category. By way of example, in the example of FIG. 3D , instances XXX, XXY, and XXZ are advertising companies considered major brands, and instances YYX, YYY, and YYZ are advertising companies considered SMBs.

[0042] 3A-3D are based on a relatively simple KG with a relatively small number of nodes. However, it is contemplated that implementations of the present disclosure may be realized with more complex KGs that may include a much larger number of nodes (e.g., hundreds, thousands, or millions) and the relationships between them.

[0043] Referring again to Figure 2, in response to the goal, the event generation engine 204 retrieves the problem-specific KG from the problem-specific KG store 224. In some examples, the event generation engine 204 queries the problem-specific KG store 224 based on one or more entities parsed from the goal, and the problem-specific KG store 224 returns the problem-specific KG in response to the query. For example, referring to the non-limiting examples herein, the problem-specific KG store 224 returns the problem-specific KG 320' of Figure 3D in response to a query including one or more of a promotional ad, an SMB, and an advertising network.

[0044] In some implementations, the event generation engine 204 processes the problem-specific KG to identify a set of events (e.g., one or more events). In some examples, the set of events is determined based on the goal as a pathfinding problem in the problem-specific KG. As an example, entities determined from the goal are mapped to nodes in the problem-specific KG (e.g., finding corresponding nodes by cosine similarity scores using word embeddings such as word2vec). The node values ​​of corresponding nodes are assigned to be the cosine similarity scores, while the node values ​​of any other nodes are zero. Using a non-limiting example herein, the example goals SMB, advertising network, and game can be mapped to nodes 312b′, 302, and 314b′. With this in mind, one or more paths in the problem-specific KG that meet the goal and cover key terms (i.e., include the identified nodes) are determined. To determine the specific path to be used, the problem is formulated and solved as a minimum-finding path on the graph. In some examples, the minimum path is determined by calculating the minimum distance between every particular pair of corresponding nodes by finding an all-pairs shortest path, similar to the so-called traveling salesman problem, and then identifying the shortest distance path connecting all corresponding nodes. In some examples, the average node value of the minimum path is determined and compared to a threshold. If the average node value exceeds the threshold, the path is returned. Otherwise, a failure is indicated.

[0045] In some examples, events that may be relevant to the intent are determined from the target nodes of the input goals provided from the returned paths. In the example of Figure 3D, instances YYX, YYY, and YYZ of node 312b' may be returned as events (e.g., leads in the example context). As an example, for an example goal of increasing the number of promotional ads that SMBs place in online video games through a particular ad network, instances YYX, YYY, and YYZ of node 312b' represent SMBs that may be able to increase the promotional ads.

[0046] In some examples, the output of the event generation engine 204 is a set of events (e.g., one or more events). The set of events is provided to the event scoring module 210, which provides an event score for each event. More specifically, the event scoring module scores each event based on data related to the event from multiple data sources. In some implementations, the data is provided by a web sensing module 208, which performs web crawling functionality and leverages AI to obtain data representing the events. In the example context in which the events include leads (e.g., companies that could be sold to), the data includes data representing the leads (companies).

[0047] FIG. 4 illustrates an example architecture 400 for web sensing using artificial intelligence (AI) in accordance with implementations of the present disclosure. In the example of FIG. 4, the architecture 400 includes a parallel fetch module 402, a parsing module 404, a validation module 406, a link deduplication module 408, a job queue store 410, a page store 412, and a data store 414. In some examples, the architecture processes URLs 416 in a set of uniform resource locators (URLs), extracts data from the underlying resources (e.g., web pages), and stores the data in the data store 414. In some examples, the URLs may originate from one or more sources, including URLs extracted from an internal enterprise capture system or a system provided by an enterprise's partner or third-party vendor, URLs obtained from search engine results by querying the names of known SMBs, or URLs obtained from links from any other public website.

[0048] More specifically, for each URL, a job queue is generated and stored in the job queue store 410. When a job is processed, the parallel fetch module 402 fetches URLs (e.g., web pages) and extracts data therefrom. In the example of FIG. 4 , the parallel fetch module 402 includes an IP rotation sub-module 420, a login handler module 422, and a retry strategy module 424. The IP rotation sub-module 420 can perform rotation of IP addresses from which URL requests are made (e.g., to mitigate anti-scraping measures). In some examples, the login handler 422 manages logins for URLs that require login (e.g., entering a username / password). In some examples, the retry strategy sub-module 424 implements one or more retry strategies for fetching URLs if the fetch initially fails. The web pages retrieved from the URLs are stored in the page store 412.

[0049] Each web page is provided to a parsing module 404, which processes the web page and extracts data therefrom. In the example of FIG. 4 , the parsing module 404 includes a category understanding submodule 426, a link filter submodule 428, a link parser submodule 430, a content understanding module 432, a product extraction module 434, and a shipping extraction module 436. In some examples, each of the category understanding submodule 426, the content understanding module 432, the product extraction module 434, and the shipping extraction module 436 is automated using AI (e.g., processing data using ML models) to process the web page to identify categories and content and further extract product and shipping data. Additionally, links provided within the web page (e.g., URLs to other web pages) are filtered, parsed, and provided to a link de-duplication module 408. In this manner, URLs extracted from the web page can be processed as individual jobs, with some being excluded from processing if they duplicate other URLs already processed (i.e., only unique URLs are processed).

[0050] In some implementations, the product data and shipping data are provided to the validation module 406 for further processing and eventual storage in the data store 414. In some examples, the product data includes, but is not limited to, product name, category, description, product number (e.g., a unique identifier assigned to the product), and the like. In some examples, the product may be a physical product or a service. In some examples, the shipping data includes, but is not limited to, data describing the form in which the product is provided (e.g., physical shipment to an address, online download). In the example of FIG. 4, the validation module 406 includes a data deduplication sub-module 438 and a data validation sub-module 440, which process the product data and shipping data to remove any duplicate data (e.g., data already considered and stored in the data store 214) and ensure the data is valid (e.g., in a format suitable for further processing).

[0051] In some implementations, an event score is determined for each event, and an action is determined for each event having an event score above a threshold event score. The event score for an event is determined based on data representing the event obtained through web sensing, as described herein with reference to FIG. 4. In some implementations, the event score is determined by processing the data through an ML model, which provides an event score for each individual event.

[0052] 5 illustrates an example ML model 500 for scoring an event, according to implementations of the present disclosure. The example ML model 500 receives a sparse feature set 502 and a dense feature set 504 representing an event, and processes each to provide an event score 506. In the example of FIG. 5, the ML model 500 includes an embedding layer 510, a factorization machine layer 512, a hidden layer 514, and a sigmoid function 516.

[0053] In some implementations, the sparse feature set 502 and the dense feature set 504 are each provided from data extracted for individual events from web sensing (described herein with reference to FIG. 4 ). Thus, for a particular event, the sparse feature set 502 and the dense feature set 504 are provided, and an individual event score 506 is generated based thereon. In the example context of a lead, where the event is a sales lead for a company to which the event is pitched, example sparse features in the sparse feature set 502 are categorical attributes with a fixed set of values, which may include, but are not limited to, address information (e.g., state, city, country), revenue data, number of employees, industry type, product, and the like. Example dense features in the dense feature set 504 are typically numerical attributes, which may include, but are not limited to, website statistics (e.g., website traffic), review scores, and the like.

[0054] In some implementations, the sparse feature subset is provided to an embedding layer 510, which provides an embedding for each sparse feature in the sparse feature subset. In some examples, the embedding is provided as a multidimensional representation (e.g., a multidimensional vector) of the individual sparse features. The sparse features in the sparse feature subset and their individual embeddings are provided to a factorizer layer 512, which processes each and provides a set of scalar values ​​as output to a sigmoid function 516. In some implementations, each embedding and each portion of each dense feature in the dense feature set 504 are provided as input to a hidden layer 514, which processes each through a series of nonlinear transformations (e.g., N nonlinear transformations) and provides an output to a sigmoid function 516. The sigmoid function 516 processes the inputs received from the factorizer layer 512 and the hidden layer 514 to provide a score (e.g., a scalar value) for the event. This process is performed for each event identified from the goal (eg, as described herein with reference to FIG. 3D) to provide a set of scored events (eg, event-score pairs).

[0055] In some implementations, events are ranked based on their individual scores. In some examples, events with higher scores are ranked higher than events with lower scores. The ranked events allow targeted actions to be determined for more promising outcomes. For example, a higher-ranked event indicates a higher likelihood of achieving a goal than a lower-ranked event, based on data obtained from web sensing. In some examples, a threshold score can be used to determine which events should be considered for further processing (e.g., determining possible actions). For example, one or more actions are determined only for those events whose scores meet or exceed the threshold score.

[0056] FIG. 6 illustrates an example ML model 600 for determining an action for an event, according to implementations of the present disclosure. In the example of FIG. 6, the ML model 600 includes an embedding layer 602, a transformer layer 604, a linear layer 606, a softmax layer 608, and a position encoder 610. The ML model 600 receives an action sequence 620 and processes the action sequence 620 to predict a next action 622 in the action sequence. In the context of the present disclosure, the next action 622 represents an action that can be taken with respect to the event. By way of example, in the case of a lead, the next action 622 is a product or service that can be communicated to the lead as part of an effort to encourage sales of the product or service to the lead (business).

[0057] In some implementations, sequence action prediction uses U={u1,u2,u n} denotes the set of events, and I={i1,i2,i m} denotes a set of actions, and S u =[i1 u ,…i t u ,…i T u ] denotes the chronological interaction sequence for event u∈U, i(u)∈I is the action that u interacted with at time step t, and T is the length of the interaction sequence for event u. The set of actions is input to ML model 600, which determines the set of actions i for a given input sequence. t That is, for event u, predict action i t is the next action to be taken. In the context of the example where the event is a lead, the action can include a product or service that can be offered to the company (i.e., the lead), and the action i t is the particular product / service that should be suggested given the given input sequence.

[0058] More specifically, the action to be predicted i tAn action sequence 620 including a mask for σ is provided as input to the embedding layer 602. In some examples, a position encoder 610 encodes the individual positions of actions in the action sequence 620 before input to the embedding layer 602. In some examples, the embedding layer provides an embedding (e.g., a multidimensional representation of the action, such as a vector representation) for each action in the action sequence 620. Each embedding is provided as input to a transformer layer 604 including multiple layers of transformers 630. By way of example, the transformer layer 630 may include n layers of transformers 630, with each layer having m transformers 630. Thus, the set of transformers may range from 1,1 to n,m.

[0059] Details of an example transformer 630 are shown in Figure 6. The transformer 630 can be described as a deep learning (DL) model that applies an attention mechanism to differentially weight the importance of each part of the output data from the embedding layer 602. The output of the final transform 630' (e.g., transformer n, m) is a vector that is then passed through a fully connected linear layer 606, followed by a softmax layer 608, to produce a single-valued output action prediction, i t Generates 622.

[0060] According to implementations of the present disclosure, at least one action is determined for each event identified for action. In some implementations, the action is performed as part of an effort to provoke a response from the event. In an example context, the event is a lead (business) and the action is a product / service offered to the lead. As an example, the event can be contacted directly to propose the action to determine whether to act on the action (e.g., purchase the product / service). As another example, the action can be proposed to the event through one or more electronic communication channels. As an example, as part of an effort to provoke a response from the event, a promotion for the action can be displayed in a user interface (UI) used by an entity associated with the event.

[0061] 7 illustrates an example process 700 that can be performed in implementations of the present disclosure. In some examples, the example process 700 is provided using one or more computer-executable programs executed by one or more computing devices.

[0062] A domain-specific KG is provided (702). By way of example, one or more domain-specific KGs may be provided through the graph modeling engine 216 of FIG. 2 and stored in the domain-specific KG store 226, as described herein. In some examples, the domain-specific KG is created entirely. In some examples, the domain-specific KG is updated (i.e., an existing domain-specific KG is updated to add / remove nodes and / or edges). In some examples, the domain-specific KG is provided through a knowledge extraction process and a knowledge fusion process.

[0063] A goal is received (704). By way of example, the goal can be provided to the early pattern detection platform through the event generation workbench 202, as described herein. By way of example, a user 120 can interact with the event generation workbench through a computing device 102 and input a goal. In an example commercial context, an example goal can include, but is not limited to, increasing the number of promotional advertisements that an SMB places in an online video game through a particular advertising network. In some examples, the goal is provided to the event generation engine 204, which triggers the identification of events and the prediction of actions for each identified event.

[0064] A problem-specific KG is provided 706. By way of example, as described herein, one or more problem-specific KGs are provided through the graph enrichment engine 216 and stored in the problem-specific KG store 226. In some examples, the problem-specific KG is created by enriching a domain-specific KG with problem-specific nodes and relationships and, for each problem-specific node, one or more instances related to the entities represented by the nodes.

[0065] A set of events is determined from the problem-specific KG (708). By way of example, as described herein, the event generation engine 204 processes the problem-specific KG to identify a set of events (e.g., one or more events). In some examples, the set of events is determined based on a goal as a pathfinding problem in the problem-specific KG. Data relevant to each event in the set of events is extracted (710) using web sensing. By way of example, as described herein, the data is provided by the web sensing module 208, which performs web crawling functionality and leverages AI to obtain data representing each event in the set of events.

[0066] The events in the set of events are scored (712). By way of example, as described herein, data representing the set of events and individual events is provided to the event scoring module 210, which provides an event score for each event. More specifically, the event scoring module scores each event based on data related to the event from multiple data sources. In some examples, the ML model 500 of FIG. 5 receives, for each event, a sparse feature set 502 and a dense feature set 504 representing the individual event, and processes each to provide an event score 506 for the individual event. A subset of events is identified based on the scores (714). By way of example, as described herein, the events may be ranked based on their individual scores, and a threshold score may be used to determine the subset of events (i.e., the events to be considered for further processing).

[0067] One or more actions are predicted for each event in the subset of events (716). By way of example, as described herein, an ML model 600 is used to determine actions for events, where the ML model 600 receives an action sequence 620 associated with an individual event and processes the action sequence 620 to predict a next action 622 in the action sequence for the individual event. In the context of this disclosure, the next action 622 represents an action that can be taken with respect to the event. The action is performed (718). By way of example, as described herein, the event can be contacted directly to propose an action to determine whether to act on the action, and / or the action can be proposed to the event through one or more electronic communication channels.

[0068] The implementations and all functional operations described herein may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. The implementations may be realized as one or more computer program products (i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or control the operation of a data processing apparatus). The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, an article of matter that provides a machine-readable propagated signal, or a combination of one or more of these. The term "computing system" includes all apparatuses, devices, and machines that process data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code that creates an execution environment for a subject computer program (e.g., code comprising processor firmware, a protocol stack, a database management system, an operating system, or any suitable combination of one or more of these). A propagated signal is an artificially generated signal (e.g., a machine-generated electrical, optical, or electromagnetic signal) that encodes information to be transmitted to an appropriate receiver apparatus.

[0069] A computer program (also known as a program, software, software application, script, or code) may be written in any suitable form of programming language, including compiled or interpreted languages, and may be deployed in any suitable form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored as part of a file that contains other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple associated files (e.g., multiple files that store one or more modules, subprograms, or portions of code). A computer program may be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0070] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs that perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry (e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit)).

[0071] Processors suitable for executing a computer program include, by way of example, both general-purpose and special-purpose microprocessors, and any one or more processors of any suitable type of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random-access memory, or both. Components of a computer may include a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data, or be operatively coupled to receive data from or transfer data to, or both. Note that a computer need not have such devices. Furthermore, a computer may be incorporated into another device (e.g., a mobile phone, a personal digital assistant (PDA), a mobile audio player, a Global Positioning System (GPS) receiver). Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0072] To provide for user interaction, each implementation may be realized on a computer having a display device (e.g., a cathode ray tube (CRT), liquid crystal display (LCD) monitor) that displays information to the user, as well as a keyboard and pointing device (e.g., a mouse, trackball, touchpad) that allows the user to provide input to the computer. Other types of devices may also be used to provide for user interaction. By way of example, feedback provided to the user may be any suitable form of sensory feedback (e.g., visual feedback, auditory feedback, haptic feedback), and input from the user may be received in any suitable form, including acoustic, speech, or tactile input.

[0073] An implementation may be realized in a computing system including back-end components (e.g., as a data server), a computing system including middleware components (e.g., an application server), and / or a computing system including front-end components (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with the implementation), or any suitable combination of one or more such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any suitable form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), such as the Internet.

[0074] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on respective computers and having a client-server relationship to each other.

[0075] While this specification contains numerous details, these should not be construed as limitations on the scope of the disclosure or the claims, but rather as descriptions of features specific to particular implementations. Certain features described herein in the context of separate implementations may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented separately in multiple implementations or in any suitable subcombination. Furthermore, while features may be described above as operating in a particular combination and may even be initially claimed as such, in some cases one or more features of a claimed combination may be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of a subcombination.

[0076] Similarly, although operations are shown in a particular order in the figures, this should not be understood as requiring that the operations be performed in the particular order or sequence shown, or that all of the operations shown be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the above-described implementations should not be understood as requiring such separation in all implementations, and it should be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0077] Several implementations have been described. However, it should be understood that various modifications may be made without departing from the spirit and scope of the present disclosure. For example, various forms of the flows shown above may be used, with steps rearranged, added, or removed. Accordingly, other implementations are within the scope of the following claims.

Claims

1. 1. A computer-implemented method for generating events and actions based on pattern recognition in data of a connected network, the method comprising: receiving a target; providing a problem-specific knowledge graph responsive to at least a portion of said goal; determining a set of events from the problem-specific knowledge graph; processing data representing events in the set of events through a first machine learning (ML) model to provide a set of event scores, each event score in the set of event scores associated with a respective event in the set of events; determining a subset of events based on the set of event scores; determining at least one action for each event in the subset of events by processing a sequence of actions through a second ML model; outputting the subset of events and the set of actions for execution of at least one action in the set of actions; Including, determining a set of events from the problem-specific knowledge graph, mapping at least a portion of the target entities to nodes in the problem-specific knowledge graph; identifying a path in the problem-specific knowledge graph that includes the node; Including, Computer-implemented methods.

2. The computer-implemented method of claim 1 , wherein an event is determined as a distinct instance of at least one node in the problem-specific knowledge graph.

3. 10. The computer-implemented method of claim 1, wherein the first ML model processes at least a portion of a sparse feature set through an embedding layer and a dense feature set through a hidden layer to provide an event score for an individual event.

4. 10. The computer-implemented method of claim 1, wherein the second ML model receives a sequence of actions associated with a discrete event and predicts a next action in the sequence of actions for the discrete event.

5. The computer-implemented method of claim 1 , wherein the second ML model includes a set of transformers that process the sequence of actions.

6. The computer-implemented method of claim 1 , further comprising, for each event in the set of events, extracting data representative of the event through web sensing.

7. 1. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon, the instructions, when executed by the one or more processors, causing the one or more processors to perform operations for generating events and actions based on pattern recognition in data of a connected network, the operations comprising: Receiving the target; providing a problem-specific knowledge graph responsive to at least a portion of said goal; determining a set of events from the problem-specific knowledge graph; processing data representing events in the set of events through a first machine learning (ML) model to provide a set of event scores, each event score in the set of event scores associated with a respective event in the set of events; determining a subset of events based on the set of event scores; determining at least one action for each event in the subset of events by processing a sequence of actions through a second ML model; outputting the subset of events and the set of actions for execution of at least one action in the set of actions; Including, Determining a set of events from the problem-specific knowledge graph includes: mapping at least a portion of the target entities to nodes in the problem-specific knowledge graph; identifying a path in the problem-specific knowledge graph that includes the node; and Including, A non-transitory computer-readable storage medium.

8. The non-transitory computer-readable storage medium of claim 7 , wherein an event is determined as a distinct instance of at least one node in the problem-specific knowledge graph.

9. 8. The non-transitory computer-readable storage medium of claim 7, wherein the first ML model processes at least a portion of a sparse feature set through an embedding layer and a dense feature set through a hidden layer to provide an event score for an individual event.

10. 8. The non-transitory computer-readable storage medium of claim 7, wherein the second ML model receives a sequence of actions associated with a respective event and predicts a next action in the sequence of actions for the respective event.

11. The non-transitory computer-readable storage medium of claim 7 , wherein the second ML model includes a set of transformers that process the sequence of actions.

12. The non-transitory computer-readable storage medium of claim 7 , wherein operations further include, for each event in the set of events, extracting data representative of the event through web sensing.

13. a computing device; a computer-readable storage device coupled to the computing device and having instructions stored thereon; the instructions, when executed by the computing device, cause the computing device to perform operations for generating events and actions based on pattern recognition in data of a connected network, the operations comprising: Receiving the target; providing a problem-specific knowledge graph responsive to at least a portion of said goal; determining a set of events from the problem-specific knowledge graph; processing data representing events in the set of events through a first machine learning (ML) model to provide a set of event scores, each event score in the set of event scores associated with a respective event in the set of events; determining a subset of events based on the set of event scores; determining at least one action for each event in the subset of events by processing a sequence of actions through a second ML model; outputting the subset of events and the set of actions for execution of at least one action in the set of actions; Including, Determining a set of events from the problem-specific knowledge graph includes: mapping at least a portion of the target entities to nodes in the problem-specific knowledge graph; identifying a path in the problem-specific knowledge graph that includes the node; and Including, system.

14. The system of claim 13 , wherein an event is determined as a distinct instance of at least one node in the problem-specific knowledge graph.

15. 14. The system of claim 13, wherein the first ML model processes at least a portion of a sparse feature set through an embedding layer and a dense feature set through a hidden layer to provide an event score for an individual event.

16. The system of claim 13 , wherein the second ML model receives a sequence of actions associated with a discrete event and predicts a next action in the sequence of actions for the discrete event.

17. The system of claim 13 , wherein the second ML model includes a set of transformers that process the sequence of actions.

Images