A method for constructing a cross-domain multi-task unified graph language model

Abstract

This application discloses a method for constructing a cross-domain, multi-task unified graph language model, belonging to the field of computer technology. This method constructs a multi-scale graph neural network encoder and pre-trains it to learn multi-scale representations from graph data across multiple domains and tasks. During the representation learning process of the multi-scale graph neural network encoder, a contrastive learning loss function is constructed based on the similarity between graph and text representations, and the different importance of negative samples from different domains in graph-text contrastive learning. This promotes spatial alignment between graph representations and text semantics, enhances the discriminative ability between different domains, and enables the model to implement adaptive representation learning and alignment strategies for different graph data. This improves the model's alignment effect and practical application capability, thereby effectively enhancing its prediction performance in cross-domain, multi-task scenarios.

Description

Technical Field

[0001] This application belongs to the field of computer technology, specifically relating to the construction of a cross-domain, multi-task unified graph language model. Background Technology

[0002] In recent years, with the rapid development of language models such as Large Language Models (LLMs) in natural language understanding and reasoning, extending their capabilities to graph structure data has become a research hotspot, giving rise to Graph Language Models (GLMs). These models aim to learn unified representations from graph data across multiple domains and tasks, thereby applying them to a wider range of graph tasks.

[0003] However, current graph language model construction methods typically rely on a single graph neural network (GNN) encoder and employ a uniform representation learning and alignment approach. They fail to consider the varying requirements for representation granularity across different tasks (such as node classification, edge prediction, and graph classification) and the significant modal gap between graph representations and the semantic space of language models. Consequently, the alignment performance and practical application capabilities of these models are poor, greatly impacting their prediction performance in cross-domain, multi-task scenarios. Summary of the Invention

[0004] The purpose of this application is to provide a method for constructing a cross-domain, multi-task unified graph language model, which can solve the problem that the alignment effect and practical application capability of graph language models in related technologies are not good, which greatly affects their prediction performance in cross-domain, multi-task scenarios.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a method for constructing a cross-domain, multi-task unified graph language model, the method comprising: A multi-scale graph neural network encoder is constructed, comprising: a graph neural network for extracting node-level representations from graph instances; a pooling module for generating graph-level representations based on node-level representations; and multilayer perceptrons corresponding to different task requirements for generating graph representations at the levels associated with the corresponding task requirements based on node-level and graph-level representations. The multi-scale graph neural network encoder is pre-trained using a contrastive learning loss function, which is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples from different domains in graph-text contrastive learning. A pre-built projection layer is trained to map the output of a multi-scale graph neural network encoder into graph structure semantic information and input it into a language model. Based on a pre-trained multi-scale graph neural network encoder and language model, as well as a trained projection layer, a target graph language model is constructed. This target graph language model is used to process graph and text data in cross-domain multi-task scenarios.

[0006] Secondly, embodiments of this application provide an apparatus for constructing a cross-domain, multi-task unified graph language model, the apparatus comprising: The first construction module is used to construct a multi-scale graph neural network encoder, which includes: a graph neural network for extracting node-level representations from graph instances; a pooling module for generating graph-level representations based on node-level representations; and multilayer perceptrons corresponding to different task requirements for generating graph representations at the levels associated with the corresponding task requirements based on node-level representations and graph-level representations. The pre-training module is used to pre-train the multi-scale graph neural network encoder using a contrastive learning loss function, which is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples from different domains in graph-text contrastive learning. The training module is used to train a pre-built projection layer, which maps the output of the multi-scale graph neural network encoder into graph structure semantic information and inputs it into the language model. The second building module is used to construct a target graph language model based on a pre-trained multi-scale graph neural network encoder and language model, as well as a trained projection layer. The target graph language model is used to process graph and text data in cross-domain multi-task scenarios.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method for constructing a cross-domain multi-task unified graph language model as described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method for constructing a cross-domain multi-task unified graph language model as described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method for constructing a cross-domain multi-task unified graph language model as described in the first aspect.

[0010] In this embodiment, a multi-scale graph neural network encoder is constructed and pre-trained to learn multi-scale representations from graph data across multiple domains and tasks, thereby meeting the different requirements of different tasks for representation granularity. Furthermore, during the representation learning process of the multi-scale graph neural network encoder, a contrastive learning loss function is constructed based on the similarity between graph and text representations, and the different importance of negative samples from different domains in graph-text contrastive learning. This promotes spatial alignment between graph representations and text semantics, enhances the discriminative ability between different domains, and enables the model to implement adaptive representation learning and alignment strategies for different graph data. This improves the model's alignment effect and practical application capabilities, effectively enhancing its prediction performance in cross-domain, multi-task scenarios. Attached Figure Description

[0011] Figure 1 A flowchart illustrating the implementation of a method for constructing a cross-domain, multi-task unified graph language model, as provided in this application embodiment; Figure 2 A diagram illustrating a cross-domain, multi-task unified graph language model training framework provided in this application embodiment; Figure 3 A schematic diagram of a device for constructing a cross-domain, multi-task unified graph language model provided in an embodiment of this application; Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] Current graph language modeling methods typically rely on a single GNN encoder and employ a uniform representation learning and alignment approach, making it difficult to effectively handle differences between different domains and tasks. For example, different datasets (such as Cora, PubMed, knowledge graphs, and molecular graphs) exhibit significant differences in structural characteristics, feature distributions, and task objectives, while current methods still use a uniform representation learning and alignment approach, failing to simultaneously meet the needs of diverse data types. Furthermore, different tasks have different requirements for representation granularity, and current graph language models based on a single GNN encoder lack unified multi-scale modeling capabilities, making it difficult to support multiple task scenarios simultaneously.

[0015] Specifically, current graph language model construction methods mainly include two categories: graph-to-text methods and graph-to-to-token methods.

[0016] Graph-to-Text methods transform graph structures into natural language text for input into large language models. These methods leverage the reasoning capabilities of language models to directly process graph tasks. However, due to the complexity and large scale of graph structures, converting complete graph information into text presents challenges in expression and incurs high token overhead, thus impacting computational efficiency and scalability.

[0017] Graph-to-Token methods encode graph data using graph neural networks and map the graph representation to the token space of a large language model. While these methods improve computational efficiency to some extent through structured representation, they still rely on fixed graph encoders and uniform alignment strategies, failing to fully consider the differences between different domains and tasks.

[0018] Based on the above analysis, current graph language model construction methods typically rely on a single GNN encoder and adopt a uniform representation learning and alignment approach. They do not consider the different requirements of different tasks for representation granularity, lack targeted modeling capabilities when processing multi-domain and multi-task graph data, and fail to consider the large modal gap between graph representation and the semantic space of language models. They also lack effective pre-training mechanisms to alleviate this problem, which limits the alignment effect and practical application capability of the model, making it difficult to directly apply to unified graph-based model scenarios.

[0019] To address the aforementioned issues, this application provides a method for constructing a cross-domain, multi-task unified graph language model. A unified graph language model framework is designed and trained. Through graph-text pair pre-training, it resolves the inconsistency issues in representation learning and semantic alignment processes for graph data from different domains and tasks. This enables the model to implement adaptive representation learning and alignment strategies for different graph data, thereby achieving effective fusion of graph representation and the semantic space of the language model. This method significantly improves the model's generalization ability across graph data from different domains and tasks, and can be widely applied to graph intelligent question answering systems, knowledge graph reasoning, molecular property prediction, and recommender systems, achieving unified modeling and cross-task transfer capabilities.

[0020] The following description, in conjunction with the accompanying drawings, details a method for constructing a cross-domain, multi-task unified graph language model provided by this application, through specific embodiments and application scenarios.

[0021] See Figure 1 The diagram shown is an implementation flowchart of a method for constructing a cross-domain, multi-task unified graph language model provided in this application embodiment. The method may include the following steps: Step S101: Construct a multi-scale graph neural network encoder, which includes: a graph neural network for extracting node-level representations of graph instances; a pooling module for generating graph-level representations based on node-level representations; and multilayer perceptrons corresponding to different task requirements for generating graph representations at the levels associated with the corresponding task requirements based on node-level representations and graph-level representations.

[0022] In practical implementation, for graph data (hereinafter referred to as graph instances) from different domains and tasks, the graph neural network and pooling module (which is used to perform pooling operations such as max pooling or average pooling) in the multi-scale graph neural network encoder can be used to extract node-level and graph-level representations. This process can be specifically represented as follows:

[0023] in, Indicates a graph instance Extracted node-level representation, Indicates to Extracted graph-level representation, This represents a graph neural network. This indicates a pooling module.

[0024] Based on this, this application introduces a multilayer perceptron (MLP) configured for different task requirements, which is used to further process the extracted node-level and graph-level representations into graph representations at different levels suitable for different task requirements. This gives the multi-scale graph neural network encoder the ability to aggregate multi-scale representations in a structural way, so as to overcome the shortcomings of traditional methods that rely on a single GNN encoder and thus adapt to different task requirements.

[0025] Step S102: The multi-scale graph neural network encoder is pre-trained using a contrastive learning loss function, which is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples from different domains in graph-text contrastive learning.

[0026] For specific implementation, see Figure 2 The diagram illustrates a cross-domain, multi-task unified graph language model training framework. For graph data from different domains and tasks, the framework first constructs corresponding graph-text pairs (e.g., graph instances and their corresponding text representations). For example, a text encoder can be used to encode the descriptive text of a graph instance to obtain its corresponding text representation, i.e.:

[0027] in, and Graph Examples The description text and the corresponding text representation, This indicates a text encoder.

[0028] Then, to address the significant modal gap between graph representations and the semantic space of language models, this application employs a contrastive learning method based on graph-text pairs. The multi-scale graph neural network encoder is pre-trained to learn a general graph representation aligned with the text semantic space from diverse graph data. Considering that negative samples from different domains (e.g., datasets or domains) should have different importance in graph-text contrastive learning (for example, samples within the same domain are usually semantically closer, while samples from different domains have greater semantic differences), they should not be treated uniformly. Therefore, a contrastive learning loss function is constructed based on the similarity between graph representations and text representations (e.g., cosine similarity) and the different importance of negative samples from different domains in graph-text contrastive learning. This promotes spatial alignment between graph representations and text semantics and enhances the discriminative ability between different domains. The model can then implement adaptive representation learning and alignment strategies for different graph data, thereby improving the cross-domain generalization ability of graph representations.

[0029] Step S103: Train the pre-built projection layer, which is used to map the output of the multi-scale graph neural network encoder into graph structure semantic information and input it into the language model.

[0030] In specific implementation, such as Figure 2 As shown, a pre-trained multi-scale graph neural network encoder and language model (such as LLM), along with a pre-built projection layer, can be used to construct a graph language model to be trained. The graph language model is then trained, and the parameters of the multi-scale graph neural network encoder and language model are kept frozen during the training process, thereby achieving the training of the projection layer.

[0031] Step S104: Based on the pre-trained multi-scale graph neural network encoder and language model, and the trained projection layer, construct a target graph language model, which is used to process graph and text data in cross-domain multi-task scenarios.

[0032] In this embodiment of the application, during the application phase, the input to the language model (such as LLM) in the target graph language model mainly includes the natural language task instruction q. i The graph tokens z are encoded by a multi-scale graph neural network encoder and mapped by a projection layer. i The output is the prediction result or text answer for a specific task (such as node category, relation type, graph category, etc.). Understandably, compared to traditional language models that only process plain text input, this application additionally introduces graph structure semantic information (such as z...). i This allows for joint modeling of graph modalities and linguistic modalities, thereby improving model prediction performance.

[0033] As can be seen from the above technical solution, this application constructs a multi-scale graph neural network encoder and pre-trains it to learn multi-scale representations from graph data of multiple domains and multiple tasks, thereby meeting the different requirements of different tasks for representation granularity. In the representation learning process of the multi-scale graph neural network encoder, a contrastive learning loss function is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples in graph-text contrastive learning in different domains. This promotes spatial alignment between graph representation and text semantics and enhances the distinguishing ability between different domains, enabling the model to implement adaptive representation learning and alignment strategies for different graph data. This can improve the alignment effect and practical application capability of the model, thereby effectively improving its prediction performance in cross-domain multi-task scenarios.

[0034] In some embodiments, the method further includes: During the training of the projection layer, the alignment difficulty of each domain is calculated based on the loss of the training samples associated with each domain in the current iteration. Based on the alignment difficulty of different domains in the current iteration round, determine the optimization weights of the training loss associated with each domain in the training objective function in the current iteration round.

[0035] In this embodiment, considering that traditional alignment methods do not take into account the differences in alignment difficulty between different data, the uniform optimization strategy adopted during training can easily lead to the model biasing towards simpler data, thus affecting overall performance. To address the aforementioned problem, this application proposes an alignment optimization method based on curriculum learning. This method dynamically adjusts the training strategy according to the alignment difficulty of different data during the alignment process between graph representation and large language model, thereby adapting to diverse data distributions and encouraging the model to learn in a balanced manner between complex and simple data, thus mitigating the training instability caused by differences in alignment difficulty between different data.

[0036] Specifically, during the training of the projection layer, the loss of the training samples associated with each domain in the current iteration can be used as the alignment difficulty of each domain. Then, based on the alignment difficulty of each domain in the current iteration, the optimization weight of the training loss associated with each domain in the current iteration in the training objective function is determined. By dynamically adjusting the optimization weight of the training loss associated with each domain in the training process, the model can prioritize training samples with higher alignment difficulty, thereby achieving more stable and efficient cross-domain alignment and improving the alignment effect of the model on multi-domain data.

[0037] For example, the training objective function used during the training of the projection layer It is expressed as follows:

[0038] in, This represents the training loss associated with neighborhood D in the k-th iteration. Optimize weights in the training objective function.

[0039] Optionally, the optimization weights of the training losses associated with different domains in the current iteration round in the training objective function are determined by the following formula:

[0040]

[0041] in, This represents the optimization weight of the training loss associated with neighborhood D in the training objective function during the k-th iteration. , and Indicates smoothing , and , and This represents the alignment difficulty of neighborhood D in the k-th and (k-1)-th iteration rounds. In the k-th iteration, the domain is represented as... Alignment difficulty Indicates the length of the warm-up phase (warm-up horizon). The momentum coefficient of an exponential moving average is used to determine the weighting ratio between historical and current difficulty levels. Indicates moving average, Represents the normalization factor. As the denominator for softmax normalization of the difficulty scores of all current active domains, it is used to standardize the optimization weights of each domain so that the sum of the optimization weights of all domains is 1.

[0042] In this embodiment, after calculating the alignment difficulty of each domain based on the loss of different training samples during training, the above-mentioned smoothing process is first applied, and then the training loss associated with different domains is assigned optimization weights based on the smoothed alignment difficulty; thus, by combining the warm-up stage (i.e., the pre-training stage) Moving average of the first step and the later stage (i.e., the later step) The exponential moving average (EMA) smoothing strategy (step 1) provides a stable estimate of the domain gradient difficulty, which can reduce the impact of gradient noise in the early stage of training and achieve more stable and efficient cross-domain course alignment optimization.

[0043] Optionally, the alignment difficulty of each of the different domains is determined by the following formula:

[0044]

[0045] in, This represents the alignment difficulty of the domain D (e.g., a dataset) in the k-th iteration round. This represents the average training loss (dataset-level loss) of domain D in the k-th iteration (e.g., during the k-th optimization (training step)). It is used to measure the alignment difficulty of the corresponding domain in the corresponding training stage. This represents the subset of training samples associated with domain D in the k-th iteration (e.g., the subset of samples from domain D in the k-th mini-batch). This represents the loss of the i-th training sample (e.g., single-sample loss).

[0046] In some embodiments, the contrastive learning loss function is represented as follows:

[0047] in, This represents the contrastive learning loss function. This represents the graph-to-text loss. Indicates the text-to-graph loss. This represents the total number of samples participating in the current contrastive learning (e.g., the number of graph-text pairs in a mini-batch). Indicates the domain to which sample i belongs With the domain of sample j The domain difference weights between them This represents the similarity (e.g., cosine similarity) between the i-th graph representation and the i-th text representation, which is also known as the "positive pair" similarity. This represents the similarity (e.g., cosine similarity) between the i-th graph representation and the j-th text representation, which is also known as the "negative pair" similarity.

[0048] In this embodiment of the application, in order to enhance the cross-domain alignment capability, a domain-difference-based reweighting mechanism is introduced into the Contrastive Language–Image Pretraining (CLIP) loss function. That is, the importance of negative samples in different domains is reweighted by domain difference weights, and the above-mentioned contrastive learning loss function is constructed accordingly to encourage graph representations to achieve cross-domain alignment in a shared semantic space, while enhancing the distinguishability between different domains.

[0049] Optionally, the similarity between the graph representation and the text representation is determined by the following formula:

[0050] in, express The i-th graph representation used With the j-th text representation The similarity between them (i.e., the similarity used to calculate) middle , ), express The i-th graph representation used With the j-th text representation The similarity between them (i.e., the similarity used to calculate) middle , ).

[0051] In some embodiments, the graph representation of different task requirements at their respective associated levels includes:

[0052]

[0053]

[0054] in, , and This represents the graph representation of the node-level task, edge-level task, and graph-level task associated with the i-th graph instance. and Represents the node corresponding to the i-th graph instance. and The node-level representation, This represents the graph-level representation corresponding to the i-th graph instance. This represents the multilayer perceptron corresponding to the task requirements.

[0055] In this embodiment, different levels of graph representation determination methods are designed, suitable for node-level tasks (such as node classification), edge-level tasks (such as edge prediction), and graph-level tasks (such as graph classification), respectively. This supports multi-scale graph neural network encoders in providing a unified multi-scale graph representation for each instance based on task requirements. To achieve the acquisition.

[0056] Based on the above embodiments, this application can perform unified representation learning and semantic alignment for graph data of different domains and tasks, and has at least the following advantages compared with related technologies: 1. Current methods employ a uniform representation learning and alignment strategy, lacking adaptive modeling capabilities for data differences across different domains and tasks. This makes it difficult to fully capture the structural and semantic differences in graph data across different domains, resulting in models struggling to learn graph representations with good generalization ability and thus failing to adapt to data differences across domains and tasks. In contrast, this application utilizes a graph-text pair pre-training method to learn a general graph representation aligned with the text semantic space, thereby improving cross-domain generalization ability.

[0057] 2. Current methods employ static optimization strategies during the alignment process, lacking modeling for the alignment difficulty of different data. This makes it difficult to adapt to the varying alignment difficulty across different domains and tasks, easily leading to overfitting on simple data and underfitting on complex data. This application, however, introduces a course-based alignment optimization method that dynamically adjusts optimization weights according to the alignment difficulty of different data, thereby achieving more stable and effective alignment results and improving the model's predictive performance in multi-task scenarios.

[0058] It should be noted that the method for constructing a cross-domain multi-task unified graph language model provided in this application embodiment can be executed by a device for constructing a cross-domain multi-task unified graph language model, or by a control module within that device for executing the method for loading the construction method of the cross-domain multi-task unified graph language model. This application embodiment uses the execution of the method for loading the construction method of the cross-domain multi-task unified graph language model by a device for constructing a cross-domain multi-task unified graph language model as an example to illustrate the method for constructing a cross-domain multi-task unified graph language model provided in this application embodiment.

[0059] This application provides an apparatus for constructing a cross-domain, multi-task unified graph language model, such as... Figure 3 As shown, the device includes: The first construction module is used to construct a multi-scale graph neural network encoder, which includes: a graph neural network for extracting node-level representations from graph instances; a pooling module for generating graph-level representations based on node-level representations; and multilayer perceptrons corresponding to different task requirements for generating graph representations at the levels associated with the corresponding task requirements based on node-level representations and graph-level representations. The pre-training module is used to pre-train the multi-scale graph neural network encoder using a contrastive learning loss function, which is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples from different domains in graph-text contrastive learning. The training module is used to train a pre-built projection layer, which maps the output of the multi-scale graph neural network encoder into graph structure semantic information and inputs it into the language model. The second building module is used to construct a target graph language model based on a pre-trained multi-scale graph neural network encoder and language model, as well as a trained projection layer. The target graph language model is used to process graph and text data in cross-domain multi-task scenarios.

[0060] Optionally, the training module is further configured to perform the following steps: During the training of the projection layer, the alignment difficulty of each domain is calculated based on the loss of the training samples associated with each domain in the current iteration. Based on the alignment difficulty of different domains in the current iteration round, determine the optimization weights of the training loss associated with each domain in the training objective function in the current iteration round.

[0061] Optionally, the optimization weights of the training losses associated with different domains in the current iteration round within the training objective function are determined by the following formula:

[0062]

[0063] in, This represents the optimization weight of the training loss associated with neighborhood D in the training objective function during the k-th iteration. , and Indicates smoothing , and , and This represents the alignment difficulty of neighborhood D in the k-th and (k-1)-th iteration rounds. In the k-th iteration, the domain is represented as... Alignment difficulty Indicates the length of the preheating stage. The momentum coefficient represents the exponential moving average. Indicates moving average, This represents the normalization factor.

[0064] Optionally, the alignment difficulty of each of the different domains is determined by the following formula:

[0065]

[0066] in, This represents the alignment difficulty of neighborhood D in the k-th iteration round. This represents the average training loss of neighborhood D in the k-th iteration. This represents the subset of training samples associated with domain D in the k-th iteration round. This represents the loss for the i-th training sample.

[0067] Optionally, the contrastive learning loss function is expressed as follows:

[0068] in, This represents the contrastive learning loss function. This represents the graph-to-text loss. Indicates the text-to-graph loss. This represents the total number of samples participating in the current contrastive learning. Indicates the domain to which sample i belongs With the domain of sample j The domain difference weights between them Let represent the similarity between the i-th graph representation and the i-th text representation. Let represent the similarity between the i-th graph representation and the j-th text representation.

[0069] Optionally, the similarity between the graphical representation and the textual representation is determined by the following formula:

[0070] in, express The i-th graph representation used With the j-th text representation Similarity between them express The i-th graph representation used With the j-th text representation The similarity between them.

[0071] Optionally, the diagram representations at the levels associated with different task requirements include:

[0072]

[0073]

[0074] in, , and This represents the graph representation of the node-level task, edge-level task, and graph-level task associated with the i-th graph instance. and Represents the node corresponding to the i-th graph instance. and The node-level representation, This represents the graph-level representation corresponding to the i-th graph instance. This represents the multilayer perceptron corresponding to the task requirements.

[0075] As can be seen from the above technical solution, this application constructs a multi-scale graph neural network encoder and pre-trains it to learn multi-scale representations from graph data of multiple domains and multiple tasks, thereby meeting the different requirements of different tasks for representation granularity. In the representation learning process of the multi-scale graph neural network encoder, a contrastive learning loss function is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples in graph-text contrastive learning in different domains. This promotes spatial alignment between graph representation and text semantics and enhances the distinguishing ability between different domains, enabling the model to implement adaptive representation learning and alignment strategies for different graph data. This can improve the alignment effect and practical application capability of the model, thereby effectively improving its prediction performance in cross-domain multi-task scenarios.

[0076] The device for constructing the cross-domain multi-task unified graph language model in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), PCs, televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0077] The apparatus for constructing the cross-domain, multi-task unified graph language model in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0078] The cross-domain, multi-task unified graph language model construction apparatus provided in this application embodiment can achieve Figure 1 The various processes implemented in the embodiment of the cross-domain multi-task unified graph language model construction method shown will not be repeated here to avoid repetition.

[0079] Optionally, this application embodiment also provides an electronic device, including a processor 110, a memory 109, and a program or instructions stored in the memory 109 and executable on the processor 110. When the program or instructions are executed by the processor 110, they implement the various processes of the above-described cross-domain multi-task unified graph language model construction method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0080] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0081] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0082] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0083] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0084] The processor 110 is used to perform the following steps: A multi-scale graph neural network encoder is constructed, comprising: a graph neural network for extracting node-level representations from graph instances; a pooling module for generating graph-level representations based on node-level representations; and multilayer perceptrons corresponding to different task requirements for generating graph representations at the levels associated with the corresponding task requirements based on node-level and graph-level representations. The multi-scale graph neural network encoder is pre-trained using a contrastive learning loss function, which is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples from different domains in graph-text contrastive learning. A pre-built projection layer is trained to map the output of a multi-scale graph neural network encoder into graph structure semantic information and input it into a language model. Based on a pre-trained multi-scale graph neural network encoder and language model, as well as a trained projection layer, a target graph language model is constructed. This target graph language model is used to process graph and text data in cross-domain multi-task scenarios.

[0085] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described cross-domain multi-task unified graph language model construction method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0086] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0087] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described cross-domain multi-task unified graph language model construction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0088] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0089] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0090] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0091] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for constructing a cross-domain, multi-task unified graph language model, characterized in that, The method includes: A multi-scale graph neural network encoder is constructed, comprising: a graph neural network for extracting node-level representations from graph instances; a pooling module for generating graph-level representations based on node-level representations; and multilayer perceptrons corresponding to different task requirements for generating graph representations at the levels associated with the corresponding task requirements based on node-level and graph-level representations. The multi-scale graph neural network encoder is pre-trained using a contrastive learning loss function, which is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples from different domains in graph-text contrastive learning. A pre-built projection layer is trained to map the output of a multi-scale graph neural network encoder into graph structure semantic information and input it into a language model. Based on a pre-trained multi-scale graph neural network encoder and language model, as well as a trained projection layer, a target graph language model is constructed. This target graph language model is used to process graph and text data in cross-domain multi-task scenarios.

2. The method according to claim 1, characterized in that, The method further includes: During the training of the projection layer, the alignment difficulty of each domain is calculated based on the loss of the training samples associated with each domain in the current iteration. Based on the alignment difficulty of different domains in the current iteration round, determine the optimization weights of the training loss associated with each domain in the training objective function in the current iteration round.

3. The method according to claim 2, characterized in that, The optimization weights of the training losses associated with different domains in the current iteration round in the training objective function are determined by the following formula: in, This represents the optimization weight of the training loss associated with neighborhood D in the training objective function during the k-th iteration. , and Indicates smoothing , and , and This represents the alignment difficulty of neighborhood D in the k-th and (k-1)-th iteration rounds. In the k-th iteration, the domain is represented as... Alignment difficulty Indicates the length of the preheating stage. The momentum coefficient represents the exponential moving average. Indicates moving average, This represents the normalization factor.

4. The method according to claim 2, characterized in that, The alignment difficulty of the different fields is determined by the following formula: in, This represents the alignment difficulty of neighborhood D in the k-th iteration round. This represents the average training loss of neighborhood D in the k-th iteration. This represents the subset of training samples associated with domain D in the k-th iteration round. This represents the loss for the i-th training sample.

5. The method according to claim 1, characterized in that, The contrastive learning loss function is expressed as follows: in, This represents the contrastive learning loss function. This represents the graph-to-text loss. Indicates the text-to-graph loss. This represents the total number of samples participating in the current contrastive learning. Indicates the domain to which sample i belongs With the domain of sample j The domain difference weights between them Let represent the similarity between the i-th graph representation and the i-th text representation. Let represent the similarity between the i-th graph representation and the j-th text representation.

6. The method according to claim 5, characterized in that, The similarity between the graphical representation and the textual representation is determined by the following formula: in, express The i-th graph representation used With the j-th text representation Similarity between them express The i-th graph representation used With the j-th text representation The similarity between them.

7. The method according to claim 1, characterized in that, The diagram representations of different task requirements at their respective levels include: in, , and This represents the graph representation of the node-level task, edge-level task, and graph-level task associated with the i-th graph instance. and Represents the node corresponding to the i-th graph instance. and The node-level representation, This represents the graph-level representation of the i-th graph instance. This represents the multilayer perceptron corresponding to the task requirements.

8. A device for constructing a cross-domain, multi-task unified graph language model, characterized in that, The device includes: The first construction module is used to construct a multi-scale graph neural network encoder, which includes: a graph neural network for extracting node-level representations from graph instances; a pooling module for generating graph-level representations based on node-level representations; and multilayer perceptrons corresponding to different task requirements for generating graph representations at the levels associated with the corresponding task requirements based on node-level representations and graph-level representations. The pre-training module is used to pre-train the multi-scale graph neural network encoder using a contrastive learning loss function, which is constructed based on the similarity between graph representation and text representation, as well as the different importance of negative samples from different domains in graph-text contrastive learning. The training module is used to train a pre-built projection layer, which maps the output of the multi-scale graph neural network encoder into graph structure semantic information and inputs it into the language model. The second building module is used to construct a target graph language model based on a pre-trained multi-scale graph neural network encoder and language model, as well as a trained projection layer. The target graph language model is used to process graph and text data in cross-domain multi-task scenarios.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of the method for constructing a cross-domain multi-task unified graph language model as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by the processor, they implement the steps of the method for constructing a cross-domain multi-task unified graph language model as described in any one of claims 1-7.

Images