A dynamic heterogeneous graph attention neural architecture search method and product applied to a recommendation system
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2022-11-18
- Publication Date
- 2026-07-24
Smart Images
Figure CN115907007B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and more specifically, to a method and product for searching dynamic heterogeneous graph attention neural architectures in recommendation systems. Background Technology
[0002] Dynamic heterogeneous graphs are ubiquitous in real-world applications, including social networks, e-commerce networks, and academic citation networks. Compared to static homogeneous graphs, dynamic heterogeneous graphs contain richer heterogeneous information, manifested in the types of nodes and edges, as well as dynamic information such as the graph structure that evolves over time. Modeling heterogeneity and temporal evolution patterns is crucial for the application of dynamic heterogeneous graphs, including the prediction of future links, node labels, and attributes.
[0003] However, while dynamic heterogeneous graph neural networks have recently made significant progress in mining graph dynamics and heterogeneous information, their architectures are still manually designed. This results in a significant need for human effort and expert knowledge in architecture design, and because manually designed architectures are fixed, they cannot automatically adapt to different datasets and application scenarios. Therefore, how to search for suitable dynamic heterogeneous graph models based on different application scenarios to achieve better results is a pressing problem that needs to be solved. Summary of the Invention
[0004] This application provides a method and product for searching dynamic heterogeneous graph attention neural architectures in recommendation systems, aiming to solve the problem of searching for suitable dynamic heterogeneous graph models according to different application scenarios to achieve better results.
[0005] The first aspect of this application provides a method for searching dynamic heterogeneous graph attention neural architectures in a recommender system, including: Obtain heterogeneous information of the input dynamic heterogeneous graph, and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneous information. The search space includes a localization space and a parameterization space. A multi-stage differentiable search is performed on the search space to output multiple candidate neural architectures and a performance score for each candidate neural architecture. Based on the performance score of each candidate neural architecture, a dynamic heterogeneous graph attention neural architecture is determined. The dynamic heterogeneous graph attention neural architecture is used to process the dynamic heterogeneous graph and output the node representation of the dynamic heterogeneous graph.
[0006] Optionally, the heterogeneity information of the input dynamic heterogeneous graph is obtained, and a search space based on the attention of the dynamic heterogeneous graph is constructed according to the heterogeneity information, including: The input dynamic heterogeneous graph is decomposed according to the timestamp to obtain multiple static heterogeneous graphs corresponding to different timestamps; Obtain the heterogeneity information for each static heterogeneous graph, including node type, edge type, and timestamp; The heterogeneous information is combined according to attention to obtain a search space based on dynamic heterogeneous graph attention.
[0007] Optionally, a multi-stage differentiable search is performed on the search space to output multiple candidate neural architectures and a performance score for each candidate neural architecture, including: By combining the connection localization and mapping functions of the dynamic heterogeneous neighborhoods corresponding to heterogeneous information in the localization space and parameterization space, a candidate neural architecture is obtained. The candidate neural architectures are trained in a differentiable manner in multiple stages and validated on a validation set to obtain a performance score for each candidate neural architecture.
[0008] Optionally, after determining the neural architecture for attention of dynamic heterogeneous graphs, the following is included: Input the dynamic heterogeneous graph into the dynamic heterogeneous graph attention neural architecture and output the representation of all nodes on the dynamic heterogeneous graph; The representations of all nodes on the dynamic heterogeneous graph are input into the downstream task of the dynamic heterogeneous graph attention neural architecture, and the accuracy of the output result is given. Based on the accuracy of the results, a search is performed in the dynamic heterogeneous graph attention architecture search space to obtain the optimal dynamic heterogeneous graph attention neural architecture.
[0009] Optionally, the heterogeneous information is combined according to attention to obtain a search space based on dynamic heterogeneous graph attention, including: The heterogeneous information is connected according to attention to obtain the connection and positioning of multiple dynamic heterogeneous neighborhoods, which serve as the positioning space corresponding to the input dynamic heterogeneous graph. The heterogeneous information is mapped to a preset attention calculation function to obtain a mapping function for a variety of dynamic heterogeneous neighborhoods, which serves as the parameterized space corresponding to the input dynamic heterogeneous graph. The preset attention calculation function includes a node mapping function and a relation mapping function. The localization space and the parameterization space are used as the search space based on dynamic heterogeneous graph attention.
[0010] Optionally, the candidate neural architectures are trained in a differentiable manner through multiple stages and validated on a validation set to obtain a performance score for each candidate neural architecture, including: For each candidate neural architecture, the mapping function of the dynamic heterogeneous neighborhood is selected sequentially with respect to node mapping and relation mapping, and trained in a differentiable manner. The node mapping weight and relation mapping weight are output, and the node mapping weight and relation mapping weight are validated on the validation set to obtain the performance score of the mapping function of each candidate neural architecture. The connection localization of the dynamic heterogeneous neighborhood in each candidate neural architecture is trained in a differentiable manner, and the localization weights are output. The localization weights are then validated on a validation set to obtain the localization performance score of each candidate neural architecture. The performance score of each candidate neural architecture is obtained by combining the mapping function performance score and the localization performance score.
[0011] A second aspect of this application provides a dynamic heterogeneous graph attention neural architecture search device applied to a recommender system, comprising: The search space construction module is used to obtain heterogeneous information of the input dynamic heterogeneous graph and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneous information. The search space includes a localization space and a parameterization space. The search module is used to perform a multi-stage differentiable search of the search space and output multiple candidate neural architectures and a performance score for each candidate neural architecture. A determination module is used to determine a dynamic heterogeneous graph attention neural architecture based on the performance score of each candidate neural architecture. The dynamic heterogeneous graph attention neural architecture is used to process the dynamic heterogeneous graph and output the node representation of the dynamic heterogeneous graph.
[0012] The search space construction module includes: The decomposition submodule is used to decompose the input dynamic heterogeneous graph according to the timestamp to obtain multiple static heterogeneous graphs corresponding to different timestamps. The heterogeneous information acquisition submodule is used to acquire the heterogeneous information of each static heterogeneous graph, which includes node type, edge type and timestamp; The search space construction submodule is used to combine the heterogeneous information according to attention to obtain a search space based on dynamic heterogeneous graph attention.
[0013] The search module includes: The candidate architecture submodule is used to combine the connection localization and mapping functions of the dynamic heterogeneous neighborhood corresponding to the heterogeneous information in the localization space and the parameterization space to obtain the candidate neural architecture. The multi-stage differentiable training submodule is used to train the candidate neural architecture in a differentiable manner in multiple stages and to validate it on a validation set to obtain a performance score for each candidate neural architecture.
[0014] After determining the dynamic heterogeneous graph attention neural architecture, the device further includes: The node representation module is used to input the dynamic heterogeneous graph into the dynamic heterogeneous graph attention neural architecture and output the representation of all nodes on the dynamic heterogeneous graph. The task processing module is used to input the representations of all nodes on the dynamic heterogeneous graph into the downstream task of the dynamic heterogeneous graph attention neural architecture and output the accuracy of the result. An optimization module is used to search the dynamic heterogeneous graph attention architecture search space based on the accuracy of the results to obtain the optimal dynamic heterogeneous graph attention neural architecture.
[0015] The search space construction submodule includes: The positioning space subunit is used to connect the heterogeneous information according to attention to obtain the connection positioning of multiple dynamic heterogeneous neighborhoods, which serves as the positioning space corresponding to the input dynamic heterogeneous graph. A parameterized space subunit is used to map the heterogeneous information to a preset attention calculation function to obtain a mapping function for a variety of dynamic heterogeneous neighborhoods, which serves as the parameterized space corresponding to the input dynamic heterogeneous graph. The preset attention calculation function includes a node mapping function and a relation mapping function. The search space subunit is used to use the localization space and the parameterization space as the search space based on dynamic heterogeneous graph attention.
[0016] The multi-stage differentiable training submodule includes: The mapping function subunit is used to sequentially select the mapping function of the dynamic heterogeneous neighborhood in each candidate neural architecture with respect to node mapping and relation mapping, and train it in a differentiable manner. It outputs node mapping weights and relation mapping weights, and performs validation on the validation set based on the node mapping weights and relation mapping weights to obtain the mapping function performance score of each candidate neural architecture. The localization subunit is used to train the connection localization of the dynamic heterogeneous neighborhood in each candidate neural architecture in a differentiable manner, output localization weights, and verify on the validation set based on the localization weights to obtain the localization performance score of each candidate neural architecture. The performance estimation subunit is used to combine the mapping function performance score and the localization performance score of each candidate neural architecture to obtain the performance score of each candidate neural architecture.
[0017] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the dynamic heterogeneous graph attention neural architecture search method applied to a recommendation system as described in any of the first aspects.
[0018] A fourth aspect of this application provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps in the dynamic heterogeneous graph attention neural architecture search method applied to a recommendation system as described in any of the first aspects.
[0019] A fifth aspect of this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps in the dynamic heterogeneous graph attention neural architecture search method applied to a recommendation system as described in any of the first aspects.
[0020] Beneficial effects: This application provides a method and product for searching dynamic heterogeneous graph attention neural architectures in recommendation systems, comprising: acquiring heterogeneous information of an input dynamic heterogeneous graph; constructing a search space based on dynamic heterogeneous graph attention according to the heterogeneous information; performing a multi-stage differentiable search on the search space to output multiple candidate neural architectures and a performance score for each candidate neural architecture; and determining the dynamic heterogeneous graph attention neural architecture based on the performance score of each candidate neural architecture. This application can automatically design dynamic heterogeneous graph attention neural architectures based on an input dynamic heterogeneous graph. Furthermore, the search space constructed by incorporating timestamp information into the heterogeneous neighborhood has universality and flexibility, automatically adapting to different datasets and application scenarios. It searches for suitable dynamic heterogeneous graph attention neural architectures for different application scenarios, enabling the searched dynamic heterogeneous graph attention neural architectures to achieve better results when processing downstream tasks. Attached Figure Description
[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a flowchart of a dynamic heterogeneous graph attention neural architecture search method for recommendation systems proposed in an embodiment of this application; Figure 2 This is a schematic diagram of a dynamic heterogeneous graph attention neural architecture search method for recommendation systems proposed in an embodiment of this application; Figure 3 This is a flowchart of the search space construction based on dynamic heterogeneous graph attention proposed in one embodiment of this application; Figure 4This is a schematic diagram of a dynamic heterogeneous graph attention neural architecture search device for a recommendation system proposed in an embodiment of this application; Figure 5 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0024] In related technologies, dynamic heterogeneous graph neural architectures are all designed manually, which means that the architecture design requires a lot of human effort and expert knowledge. Moreover, since the manually designed architecture is fixed, it cannot automatically adapt to different datasets and application scenarios.
[0025] In view of this, embodiments of this application propose a method for searching dynamic heterogeneous graph attention neural architectures in recommender systems. Figure 1 The flowchart of a dynamic heterogeneous graph attention neural architecture search method applied to recommender systems is shown, such as... Figure 1 As shown, it includes the following steps: S101. Obtain the heterogeneous information of the input dynamic heterogeneous graph, and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneous information.
[0026] Obtain heterogeneous information of the input dynamic heterogeneous graph, and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneous information. The search space includes a localization space and a parameterization space.
[0027] S102. Perform a multi-stage differentiable search on the search space and output multiple candidate neural architectures and the performance score of each candidate neural architecture.
[0028] S103. Determine the dynamic heterogeneous graph attention neural architecture based on the performance score of each candidate neural architecture. Based on the performance score of each candidate neural architecture, a dynamic heterogeneous graph attention neural architecture is determined. The dynamic heterogeneous graph attention neural architecture is used to process the dynamic heterogeneous graph and output the node representation of the dynamic heterogeneous graph.
[0029] In specific implementation step S101, the dynamic heterogeneous graph of the input dynamic heterogeneous graph attention search (DHGAS) architecture is first obtained. This dynamic heterogeneous graph is composed of static heterogeneous graphs with different timestamps, and each static heterogeneous graph with a different timestamp contains heterogeneous information on node type and edge type.
[0030] Compared to static heterogeneous graphs, dynamic heterogeneous graphs also exhibit dynamic graph structure information that evolves over time, as the timestamp information of each static heterogeneous graph represents its own evolution. The edges and nodes of static heterogeneous graphs corresponding to different timestamps are different. For example, in a recommendation system, items purchased this year and items purchased next year, as well as buyers this year and buyers next year, can all be considered as different nodes. The dynamic heterogeneous graph attention neural architecture searched by the method in this application extends the classic neighborhood (static heterogeneous graph) to a dynamic heterogeneous neighborhood by introducing a time constraint. It also focuses on the neighborhood through attention parameterization of node and edge types, enabling the architecture to simultaneously consider heterogeneous neighborhoods spanning different timestamps, thus achieving better performance in downstream task processing.
[0031] Therefore, after obtaining the input dynamic heterogeneous graph, it is decomposed according to timestamps to obtain multiple static heterogeneous graphs corresponding to different timestamps. For example, if the dynamic heterogeneous graph represents the products purchased by users over 12 months, then each month can be used as a timestamp, and the static heterogeneous graphs for each month can be divided according to the order of the timestamps. Each static heterogeneous graph contains different node types and edge types. Figure 2 A schematic diagram of a dynamic heterogeneous graph attention neural architecture search method applied to recommender systems is shown, such as... Figure 2 As shown, the left side displays the static heterogeneous graph information at three timestamps t=1, t=2, and t=3 from the input dynamic heterogeneous graph decomposition. The graph contains different node types and edge types between nodes.
[0032] Subsequently, from each of the static heterogeneous graphs decomposed above, the heterogeneous information of each static heterogeneous graph is obtained. The heterogeneous information includes different node types, different edge types, and timestamps corresponding to each static heterogeneous graph. The heterogeneous information of all static heterogeneous graphs is combined to form the heterogeneous information of the dynamic heterogeneous graph.
[0033] After obtaining the heterogeneous information of the dynamic heterogeneous graph, the search space based on dynamic heterogeneous graph attention (DHGA) can be constructed by processing this heterogeneous information. The search space defines all connection localizations and all mapping function relationships that may be used to construct the dynamic heterogeneous graph attention neural architecture. By searching this search space, the dynamic heterogeneous graph attention neural architecture (DHGNN) can be output.
[0034] Specifically, the heterogeneous information needs to be combined according to attention to obtain a search space based on dynamic heterogeneous graph attention. Figure 3 A flowchart illustrating the search space construction process based on dynamic heterogeneous graph attention is shown, such as... Figure 3 As shown, it includes the following steps: S201. Connect heterogeneous information according to attention to obtain the connection location of multiple dynamic heterogeneous neighborhoods, which serve as the localization space corresponding to the dynamic heterogeneous graph as input.
[0035] In practical implementation, the heterogeneous information of the dynamic heterogeneous graph is acquired, noisy edges in the original heterogeneous information are removed, and each node type is connected to other node types at a certain timestamp according to attention. Each different connection (such as different node types or different timestamps) is used as a connection location for a dynamic heterogeneous neighborhood, and all the connection locations of the dynamic heterogeneous neighborhoods are combined to form the localization space corresponding to the input dynamic heterogeneous graph. For example, ... Figure 2 As shown, in the heterogeneous information of the input dynamic heterogeneous graph, there is no connection between node type 1 at t=1 and node type 2 at t=3. When constructing the positioning space, the connection that did not exist will be connected to obtain the connection positioning of node type 1 at t=1 and node type 2 at t=3, and this connection positioning will be used as a kind of connection positioning of dynamic heterogeneous neighborhood in the positioning space.
[0036] Therefore, by determining all possible connection locations through heterogeneous information according to attention, a location space composed of connection locations in multiple dynamic heterogeneous neighborhoods is obtained. This location space determines the types of relationships and timestamps that the dynamic heterogeneous graph attention neural architecture needs to focus on, and the connection locations in dynamic heterogeneous neighborhoods can be customized through this location space.
[0037] S202. Map the heterogeneous information to the preset attention calculation function to obtain the mapping function of various dynamic heterogeneous neighborhoods, which serves as the parameterization space corresponding to the input dynamic heterogeneous graph.
[0038] In practice, for nodes in heterogeneous information, mapping is performed sequentially based on a preset attention calculation function, which includes a node mapping function. and relational mapping functions This yields mapping functions for various dynamic heterogeneous neighborhoods. For example, such as... Figure 2 As shown, two types of node mapping functions are preset in the attention parameterization space. and 3 types of relation mapping functions For node type 1 at t=1 and node type 2 at t=3, the mapping process can have multiple possible paths after passing through the node mapping function and the relation mapping function. Each mapping process through the node mapping function and the relation mapping function is taken as a mapping function for the dynamic heterogeneous neighborhood of these two nodes. The mapping functions for the multiple dynamic heterogeneous neighborhoods corresponding to all node types in the heterogeneous relation are obtained in the above manner, and used as the parameterization space corresponding to the input dynamic heterogeneous graph.
[0039] This parameterization space determines the mapping function for computing attention, as well as which types of nodes, edges, and timestamps should share the same parameterization. This parameterization space allows for customization of mapping functions (including node mapping and relation mapping) in dynamic heterogeneous neighborhoods.
[0040] It should be noted that the number of preset attention calculation functions, including node mapping functions and relation mapping functions, depends on the computing resources (such as computing power and capacity) of the parameter space, and this application does not limit the specific number.
[0041] S203. The positioning space and the parameterized space are used as the search space based on dynamic heterogeneous graph attention.
[0042] By combining the obtained localization space and parameterization space into a search supernetwork, a search space based on dynamic heterogeneous graph attention is established. This search space contains all possible connection localizations and all possible node and relation mappings for constructing a dynamic heterogeneous graph attention neural architecture. The search space based on dynamic heterogeneous graph attention proposed in this application has versatility and flexibility, can cover many classic DHGNN architectures as special cases, can adapt to different dynamic heterogeneous graph scenarios, and allows subsequent search algorithms to make trade-offs between performance estimation and computational resources.
[0043] This completes the search space required for searching dynamic heterogeneous graph attention neural architecture. Next, step S103 is executed to search within this search space and determine the optimal connection localization and mapping functions, thereby forming the dynamic heterogeneous graph attention neural architecture.
[0044] In specific implementation step S103, this embodiment of the application employs a multi-stage differentiable search algorithm to search the search space based on dynamic heterogeneous graph attention constructed in S102. First, the connection localization and mapping functions of the dynamic heterogeneous neighborhoods corresponding to heterogeneous information in the localization space and parameterization space are combined to obtain multiple candidate neural architectures. Each candidate neural architecture contains different combinations of connection localization and mapping functions (including node mapping and relation mapping) of the dynamic heterogeneous neighborhoods. Next, the obtained candidate neural architectures are trained in a differentiable manner through multiple stages and validated on a validation set to obtain a performance score for each candidate neural architecture. Specifically, as... Figure 2The multi-stage differentiable search section is shown, which is divided into three stages: node mapping, relation mapping, and localization. During the search, single-variable searches are performed separately by controlling variables. For example, the hypernetwork sharing connecting relation mapping and localization in the search space is fixed, and the node mapping search is performed; then, the hypernetwork sharing connecting node mapping and localization in the search space is fixed, and the relation mapping search is performed; finally, the hypernetwork sharing connecting node mapping and relation mapping in the search space is fixed, and the connection localization search is performed. The specific process is as follows: The training process for both node mapping and relation mapping stages is similar. For each candidate neural architecture, the mapping function of the dynamic heterogeneous neighborhood is sequentially selected with respect to node mapping and relation mapping, and trained in a differentiable manner. The node mapping weights and relation mapping weights are output. Based on the node mapping weights and relation mapping weights, the results are validated on the validation set. The obtained node mapping weights and relation mapping weights are then applied to the corresponding loss functions in the validation set to obtain the loss function value corresponding to the convergence of each candidate neural architecture on the validation set. This loss function value is used as the performance score of the mapping function of each candidate neural architecture.
[0045] In traditional hypernetwork search space training of mapping functions, discrete selection is used. That is, among multiple preset attention calculation functions in the parameterized space, only one node mapping function can be selected for each node type or relation type in each dynamic heterogeneous neighborhood mapping function. or relational mapping function Therefore, only discrete selection can be used during training. However, in this embodiment, multiple preset attention calculation functions in the space are transformed into a continuous distribution, thereby relaxing the discrete selection during training to a continuous selection, making the training process differentiable.
[0046] Similarly, the connection localization of the dynamic heterogeneous neighborhood in each candidate neural architecture is trained in a differentiable manner, and the localization weights are output. The localization weights are then validated on the validation set, and the obtained localization weights are applied to the corresponding connection localization loss function in the validation set to obtain the loss function value corresponding to the convergence of each candidate neural architecture on the validation set, which is used as the localization performance score of each candidate neural architecture.
[0047] The performance score of each candidate neural architecture is obtained by combining the mapping function performance score and the localization performance score. This embodiment relaxes discrete selection to continuous selection in the search space. The multi-stage differentiable search algorithm of this embodiment can jointly optimize the selection of the architecture and the supernet weights in a differentiable manner, and applies these weights to the loss function validated on the validation set. The loss function value obtained based on the weights when each candidate neural architecture converges on the validation set is used as its performance score, which can serve as a performance estimate of the candidate neural architecture.
[0048] like Figure 2 As shown on the right, all candidate neural architectures and their performance scores have been obtained. Step S103 is then executed, and the candidate neural architecture with the highest performance score is selected as the Dynamic Heterogeneous Graph Attention Neural Architecture (DHGNN). This DHGNN is the output of the Dynamic Heterogeneous Graph Attention Search (DHGAS) architecture.
[0049] This dynamic heterogeneous graph attention neural architecture (DHGNN) is used to process dynamic heterogeneous graphs and output node representations of the graphs. After obtaining the DHGNN, the dynamic heterogeneous graph is input into it, and after processing, representations of all nodes in the graph are output. These representations are then input into downstream tasks of the DHGNN (such as node classification or connection prediction), and the accuracy of the results is output. Finally, based on the accuracy, a multi-stage differentiable search algorithm in the DHGNN space is optimized. Based on the optimized DHGNN space, a search is performed within the DHGNN space to obtain the optimal DHGNN.
[0050] This application provides a method for searching dynamic heterogeneous graph attention neural architectures in a recommender system, comprising: acquiring heterogeneous information of an input dynamic heterogeneous graph; constructing a search space based on dynamic heterogeneous graph attention according to the heterogeneous information; performing a multi-stage differentiable search on the search space to output multiple candidate neural architectures and a performance score for each candidate neural architecture; and determining the dynamic heterogeneous graph attention neural architecture based on the performance score of each candidate neural architecture. This application can automatically design dynamic heterogeneous graph attention neural architectures based on an input dynamic heterogeneous graph. Furthermore, the search space constructed by incorporating timestamp information into the heterogeneous neighborhood has universality and flexibility, automatically adapting to different datasets and application scenarios. It searches for suitable dynamic heterogeneous graph attention neural architectures for different application scenarios, enabling the searched dynamic heterogeneous graph attention neural architectures to achieve better results when processing downstream tasks.
[0051] Based on the same inventive concept, embodiments of this application disclose a dynamic heterogeneous graph attention neural architecture search device applied to a recommendation system. Figure 4 A schematic diagram of a dynamic heterogeneous graph attention neural architecture search device applied to a recommender system is shown, such as... Figure 4 As shown, the device includes: The search space construction module is used to obtain heterogeneous information of the input dynamic heterogeneous graph and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneous information. The search space includes a localization space and a parameterization space. The search module is used to perform a multi-stage differentiable search of the search space and output multiple candidate neural architectures and a performance score for each candidate neural architecture. A determination module is used to determine a dynamic heterogeneous graph attention neural architecture based on the performance score of each candidate neural architecture. The dynamic heterogeneous graph attention neural architecture is used to process the dynamic heterogeneous graph and output the node representation of the dynamic heterogeneous graph.
[0052] The search space construction module includes: The decomposition submodule is used to decompose the input dynamic heterogeneous graph according to the timestamp to obtain multiple static heterogeneous graphs corresponding to different timestamps. The heterogeneous information acquisition submodule is used to acquire the heterogeneous information of each static heterogeneous graph, which includes node type, edge type and timestamp; The search space construction submodule is used to combine the heterogeneous information according to attention to obtain a search space based on dynamic heterogeneous graph attention.
[0053] The search module includes: The candidate architecture submodule is used to combine the connection localization and mapping functions of the dynamic heterogeneous neighborhood corresponding to the heterogeneous information in the localization space and the parameterization space to obtain the candidate neural architecture. The multi-stage differentiable training submodule is used to train the candidate neural architecture in a differentiable manner in multiple stages and to validate it on a validation set to obtain a performance score for each candidate neural architecture.
[0054] After determining the dynamic heterogeneous graph attention neural architecture, the device further includes: The node representation module is used to input the dynamic heterogeneous graph into the dynamic heterogeneous graph attention neural architecture and output the representation of all nodes on the dynamic heterogeneous graph. The task processing module is used to input the representations of all nodes on the dynamic heterogeneous graph into the downstream task of the dynamic heterogeneous graph attention neural architecture and output the accuracy of the result. An optimization module is used to search the dynamic heterogeneous graph attention architecture search space based on the accuracy of the output results to obtain the optimal dynamic heterogeneous graph attention neural architecture.
[0055] The search space construction submodule includes: The positioning space subunit is used to connect the heterogeneous information according to attention to obtain the connection positioning of multiple dynamic heterogeneous neighborhoods, which serves as the positioning space corresponding to the input dynamic heterogeneous graph. A parameterized space subunit is used to map the heterogeneous information to a preset attention calculation function to obtain a mapping function for a variety of dynamic heterogeneous neighborhoods, which serves as the parameterized space corresponding to the input dynamic heterogeneous graph. The preset attention calculation function includes a node mapping function and a relation mapping function. The search space subunit is used to use the localization space and the parameterization space as the search space based on dynamic heterogeneous graph attention.
[0056] The multi-stage differentiable training submodule includes: The mapping function subunit is used to sequentially select the mapping function of the dynamic heterogeneous neighborhood in each candidate neural architecture with respect to node mapping and relation mapping, and train it in a differentiable manner. It outputs node mapping weights and relation mapping weights, and performs validation on the validation set based on the node mapping weights and relation mapping weights to obtain the mapping function performance score of each candidate neural architecture. The localization subunit is used to train the connection localization of the dynamic heterogeneous neighborhood in each candidate neural architecture in a differentiable manner, output localization weights, and verify on the validation set based on the localization weights to obtain the localization performance score of each candidate neural architecture. The performance estimation subunit is used to combine the mapping function performance score and the localization performance score of each candidate neural architecture to obtain the performance score of each candidate neural architecture.
[0057] Based on the same inventive concept, embodiments of this application disclose an electronic device. Figure 5 A schematic diagram of an electronic device disclosed in an embodiment of this application is shown, such as... Figure 5 As shown, the electronic device 100 includes a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus for communication. The memory 110 stores a computer program that can run on the processor 120 to implement the steps in the dynamic heterogeneous graph attention neural architecture search method for recommendation systems disclosed in this application embodiment.
[0058] Based on the same inventive concept, embodiments of this application disclose a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps in the dynamic heterogeneous graph attention neural architecture search method for recommendation systems disclosed in embodiments of this application.
[0059] Based on the same inventive concept, this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in the dynamic heterogeneous graph attention neural architecture search method for recommendation systems disclosed in the embodiments of this application.
[0060] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0061] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0064] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.
[0065] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0066] The above provides a detailed description of the dynamic heterogeneous graph attention neural architecture search method and product applied to recommendation systems provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.
Claims
1. A method for searching dynamic heterogeneous graph attention neural architectures in recommender systems, characterized in that, include: Obtain heterogeneous information of the input dynamic heterogeneous graph, and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneous information. The search space includes a localization space and a parameterization space. A multi-stage differentiable search is performed on the search space to output multiple candidate neural architectures and a performance score for each candidate neural architecture. Based on the performance score of each candidate neural architecture, a dynamic heterogeneous graph attention neural architecture is determined. The dynamic heterogeneous graph attention neural architecture is used to process the dynamic heterogeneous graph and output the node representation of the dynamic heterogeneous graph. Obtain the heterogeneity information of the input dynamic heterogeneous graph, and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneity information, including: The input dynamic heterogeneous graph is decomposed according to the timestamp to obtain multiple static heterogeneous graphs corresponding to different timestamps; the dynamic heterogeneous graph represents the products purchased by users over 12 months, and the multiple static heterogeneous graphs corresponding to different timestamps represent the static heterogeneous graphs for each month of the 12 months. Obtain the heterogeneity information for each static heterogeneous graph, including node type, edge type, and timestamp, where the node type includes product type and buyer type; The heterogeneous information is combined according to attention to obtain a search space based on dynamic heterogeneous graph attention, including the following steps: The heterogeneous information is connected according to attention to obtain the connection location of multiple dynamic heterogeneous neighborhoods, which serves as the location space corresponding to the input dynamic heterogeneous graph. This includes: connecting each node type with other node types according to attention at a certain time stamp, and treating each different connection as a connection location of a dynamic heterogeneous neighborhood. The connection locations of all dynamic heterogeneous neighborhoods are combined to form the location space corresponding to the input dynamic heterogeneous graph. The location space determines the relationship type and time stamp that the dynamic heterogeneous graph attention neural architecture needs to focus on. Specifically, this includes: connecting the previously non-existent connection relationship between node type 1 at t=1 and node type 2 at t=3, and treating it as a connection location of a dynamic heterogeneous neighborhood in the location space. The heterogeneous information is mapped to a preset attention calculation function to obtain multiple dynamic heterogeneous neighborhood mapping functions, which serve as the parameterized space corresponding to the input dynamic heterogeneous graph. The preset attention calculation function includes node mapping functions and relation mapping functions. The specific preset number of preset attention calculation functions depends on the computing resources of the parameterized space, including computing power and capacity. Specifically, the traditional discrete selection is relaxed to continuous selection. For node type 1 at t=1 and node type 2 at t=3, they are mapped sequentially through node mapping functions and relation mapping functions. Each process of mapping through node mapping functions and relation mapping functions is used as a mapping function for the dynamic heterogeneous neighborhood of these two nodes. The mapping functions for multiple dynamic heterogeneous neighborhoods corresponding to all node types in the heterogeneous relation are obtained in the above manner, which serve as the parameterized space corresponding to the input dynamic heterogeneous graph. The traditional discrete selection means that each node type or relation type can only choose one node mapping function or relation mapping function. The localization space and the parameterization space are used as the search space based on dynamic heterogeneous graph attention; wherein, a multi-stage differentiable search is performed on the search space to output multiple candidate neural architectures and a performance score for each candidate neural architecture, including: By combining the connection localization and mapping functions of the dynamic heterogeneous neighborhoods corresponding to heterogeneous information in the localization space and parameterization space, a candidate neural architecture is obtained. The candidate neural architectures are trained in a differentiable manner through multiple stages and validated on a validation set to obtain a performance score for each candidate neural architecture, specifically including: For each candidate neural architecture, the mapping function of the dynamic heterogeneous neighborhood is selected sequentially with respect to node mapping and relation mapping, and trained in a differentiable manner. The node mapping weight and relation mapping weight are output, and the node mapping weight and relation mapping weight are validated on the validation set to obtain the performance score of the mapping function of each candidate neural architecture. The connection localization of the dynamic heterogeneous neighborhood in each candidate neural architecture is trained in a differentiable manner, and the localization weights are output. The localization weights are then validated on a validation set to obtain the localization performance score of each candidate neural architecture. The performance score of each candidate neural architecture is obtained by combining the mapping function performance score and the localization performance score.
2. The method for searching dynamic heterogeneous graph attention neural architectures in recommendation systems according to claim 1, characterized in that, After determining the neural architecture for attention in dynamic heterogeneous graphs, the following is included: Input the dynamic heterogeneous graph into the dynamic heterogeneous graph attention neural architecture and output the representation of all nodes on the dynamic heterogeneous graph; The representations of all nodes on the dynamic heterogeneous graph are input into the downstream task of the dynamic heterogeneous graph attention neural architecture, and the accuracy of the output result is given. Based on the accuracy of the results, a search is performed in the dynamic heterogeneous graph attention architecture search space to obtain the optimal dynamic heterogeneous graph attention neural architecture.
3. A dynamic heterogeneous graph attention neural architecture search device applied to a recommender system, characterized in that, include: The search space construction module is used to obtain heterogeneous information of the input dynamic heterogeneous graph and construct a search space based on the attention of the dynamic heterogeneous graph according to the heterogeneous information. The search space includes a localization space and a parameterization space. The search module is used to perform a multi-stage differentiable search of the search space and output multiple candidate neural architectures and a performance score for each candidate neural architecture. The determination module is used to determine the dynamic heterogeneous graph attention neural architecture based on the performance score of each candidate neural architecture. The dynamic heterogeneous graph attention neural architecture is used to process the dynamic heterogeneous graph and output the node representation of the dynamic heterogeneous graph. The search space construction module is further configured to: decompose the input dynamic heterogeneous graph according to the timestamp to obtain multiple static heterogeneous graphs corresponding to different timestamps; The dynamic heterogeneous graph represents the products purchased by users over 12 months, while multiple static heterogeneous graphs corresponding to different timestamps represent the static heterogeneous graphs for each month within those 12 months. Obtain the heterogeneity information for each static heterogeneous graph, including node type, edge type, and timestamp, where the node type includes product type and buyer type; The heterogeneous information is combined according to attention to obtain a search space based on dynamic heterogeneous graph attention, including the following steps: The heterogeneous information is connected according to attention to obtain the connection location of multiple dynamic heterogeneous neighborhoods, which serves as the location space corresponding to the input dynamic heterogeneous graph. This includes: connecting each node type with other node types according to attention at a certain time stamp, and treating each different connection as a connection location of a dynamic heterogeneous neighborhood. The connection locations of all dynamic heterogeneous neighborhoods are combined to form the location space corresponding to the input dynamic heterogeneous graph. The location space determines the relationship type and time stamp that the dynamic heterogeneous graph attention neural architecture needs to focus on. Specifically, this includes: connecting the previously non-existent connection relationship between node type 1 at t=1 and node type 2 at t=3, and treating it as a connection location of a dynamic heterogeneous neighborhood in the location space. The heterogeneous information is mapped to a preset attention calculation function to obtain multiple dynamic heterogeneous neighborhood mapping functions, which serve as the parameterized space corresponding to the input dynamic heterogeneous graph. The preset attention calculation function includes node mapping functions and relation mapping functions. The specific preset number of preset attention calculation functions depends on the computing resources of the parameterized space, including computing power and capacity. Specifically, the traditional discrete selection is relaxed to continuous selection. For node type 1 at t=1 and node type 2 at t=3, they are mapped sequentially through node mapping functions and relation mapping functions. Each process of mapping through node mapping functions and relation mapping functions is used as a mapping function for the dynamic heterogeneous neighborhood of these two nodes. The mapping functions for multiple dynamic heterogeneous neighborhoods corresponding to all node types in the heterogeneous relation are obtained in the above manner, which serve as the parameterized space corresponding to the input dynamic heterogeneous graph. The traditional discrete selection means that each node type or relation type can only choose one node mapping function or relation mapping function. The localization space and the parameterization space are used as the search space based on dynamic heterogeneous graph attention; The search space is subjected to a multi-stage differentiable search, outputting multiple candidate neural architectures and a performance score for each candidate neural architecture, including: By combining the connection localization and mapping functions of the dynamic heterogeneous neighborhoods corresponding to heterogeneous information in the localization space and parameterization space, a candidate neural architecture is obtained. The candidate neural architectures are trained in a differentiable manner through multiple stages and validated on a validation set to obtain a performance score for each candidate neural architecture, specifically including: For each candidate neural architecture, the mapping function of the dynamic heterogeneous neighborhood is selected sequentially with respect to node mapping and relation mapping, and trained in a differentiable manner. The node mapping weight and relation mapping weight are output, and the node mapping weight and relation mapping weight are validated on the validation set to obtain the performance score of the mapping function of each candidate neural architecture. The connection localization of the dynamic heterogeneous neighborhood in each candidate neural architecture is trained in a differentiable manner, and the localization weights are output. The localization weights are then validated on a validation set to obtain the localization performance score of each candidate neural architecture. The performance score of each candidate neural architecture is obtained by combining the mapping function performance score and the localization performance score.
4. An electronic device, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the dynamic heterogeneous graph attention neural architecture search method for recommendation systems as described in any one of claims 1-2.
5. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When executed by a processor, the computer program / instruction implements the steps of the dynamic heterogeneous graph attention neural architecture search method for recommendation systems as described in any of claims 1-2.
6. A computer program product comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program / instruction implements the steps of the dynamic heterogeneous graph attention neural architecture search method for recommendation systems as described in any of claims 1-2.