Network model training method, event extraction method, device, and storage medium

Abstract

This application discloses a network model training method, an event extraction method, a device, and a storage medium. The model training method includes: obtaining first sample data from a training set corresponding to a target task, and obtaining second sample data from training samples corresponding to an auxiliary task; training a first network model using the first sample data, and training a second network model using the second sample data; sharing gradient updates for several shared network layers between the first and second network models, updating gradients for non-shared network layers of the first network model according to the target task, and updating gradients for non-shared network layers of the second network model according to the auxiliary task, wherein the shared and non-shared network layers are pre-determined using gradient similarity; and using the first network model as the final network model. Through the above methods, the first network model can learn knowledge from the second network model, and a final network model can be obtained.

Description

Technical Field

[0001] This application relates to the field of event extraction technology, and in particular to network model training methods, event extraction methods, devices and storage media. Background Technology

[0002] Currently, knowledge transfer is typically achieved by using auxiliary tasks to train an auxiliary network model, which is then used to transfer knowledge to the target network model.

[0003] However, the auxiliary task and the target task of the target network model are different, so the target network model trained in this way still has the problem of low accuracy. Summary of the Invention

[0004] This application provides a network model training method, an event extraction method, an apparatus, and a storage medium. By allowing a first network model to learn the knowledge of a second network model, the trained first network model is used as the final model. This final network model can be used for event extraction, and the model performance is not affected when data resources are scarce.

[0005] To address the aforementioned technical problems, this application adopts the following technical solution: A network model training method based on gradient similarity is provided. The network model includes a first network model and a second network model. Specifically, the method includes: obtaining first sample data from the training set corresponding to the target task, and obtaining second sample data from the training samples corresponding to the auxiliary task; training the first network model using the first sample data, and training the second network model using the second sample data; sharing gradient updates for several shared network layers between the first and second network models; updating the gradients of the non-shared network layers of the first network model according to the target task; updating the gradients of the non-shared network layers of the second network model according to the auxiliary task, wherein the shared and non-shared network layers are pre-determined using gradient similarity; and using the first network model as the final network model.

[0006] The process of training a first network model using first sample data and a second network model using second sample data includes: obtaining a first loss value corresponding to the first network model; obtaining a second loss value corresponding to the second network model; determining a first gradient corresponding to each network layer in the first network model based on the first loss value; determining a second gradient corresponding to each network layer in the second network model based on the second loss value; determining the gradient similarity between network layers of the first and second network models based on the first and second gradients; and determining shared and non-shared network layers between the first and second network models based on the gradient similarity.

[0007] The determination of shared and non-shared network layers based on gradient similarity includes: if the gradient similarity is greater than or equal to a threshold, the network layer corresponding to the gradient similarity is determined to be a shared network layer; if the gradient similarity is less than the threshold, the network layer corresponding to the gradient similarity is determined to be a non-shared network layer.

[0008] There are multiple first sample data and second sample data.

[0009] Specifically, determining the first gradient corresponding to each network layer in the first network model based on the first loss value includes: determining the first gradient corresponding to each network layer in the first network model based on each first sample data; and accumulating all the first gradients to obtain the first cumulative gradient.

[0010] Specifically, determining the second gradient corresponding to each network layer in the second network model based on the second loss value includes: determining the second gradient corresponding to each network layer in the second network model based on each second sample data; and accumulating all the second gradients to obtain the second cumulative gradient.

[0011] The process of determining the gradient similarity between the network layers of the first network model and the second network model based on the first gradient and the second gradient includes: determining the gradient similarity between the network layers of the first network model and the second network model based on the first cumulative gradient and the second cumulative gradient.

[0012] The first network model and the second network model share the same embedding layer.

[0013] The target task includes event extraction, and the auxiliary tasks include at least relation extraction and named entity recognition.

[0014] To address the aforementioned technical problems, another technical solution adopted in this application is to provide an event extraction method, which includes: acquiring the text corresponding to the event extraction task; inputting the text into a network model to obtain the event corresponding to the text.

[0015] The network model is the final network model trained using the gradient similarity-based network model training method described above.

[0016] To address the aforementioned technical problems, another technical solution adopted in this application is to provide an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the aforementioned gradient similarity-based network model training method or event extraction method.

[0017] To address the aforementioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing a computer program, which, when executed by a processor, is used to implement the aforementioned gradient similarity-based network model training method or event extraction method.

[0018] The beneficial effects of this application are as follows: Unlike existing technologies, the gradient similarity-based network model training method provided in this application can solve the problem of insufficient accuracy of the first network model when the first sample data in the training set corresponding to the target task is sparse by using the knowledge transfer from the second network model trained on the auxiliary task to the first network model. Furthermore, by using gradient similarity to pre-determine the shared and non-shared network layers between the first and second network models, when training the first network model using the first sample data in the training set corresponding to the target task, and training the second network model using the second sample data from the training samples corresponding to the auxiliary task, gradient updates can be shared for several shared network layers between the first and second network models. Gradient updates are performed on the non-shared network layers of the first network model according to the target task, and gradient updates are performed on the non-shared network layers of the second network model according to the auxiliary task. This alleviates the problem of transferring irrelevant knowledge and noise learned in the second network model from the auxiliary task to the first network model, reduces the transfer of irrelevant knowledge and noise, and thus improves the accuracy of the first network model in completing the target task. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0020] Figure 1 This is a flowchart illustrating the first embodiment of the gradient similarity-based network model training method provided in this application;

[0021] Figure 2 This is a flowchart illustrating an embodiment of step 12 provided in this application;

[0022] Figure 3 This is a schematic diagram of the structure of an embodiment of the training network model provided in this application;

[0023] Figure 4 This is a schematic diagram of the structure of an embodiment of the final network model provided in this application;

[0024] Figure 5This is a flowchart illustrating an embodiment of the event extraction method provided in this application;

[0025] Figure 6 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application;

[0026] Figure 7 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0028] See Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the gradient similarity-based network model training method provided in this application. The network model includes a first network model and a second network model. The model training method includes:

[0029] Step 11: Obtain the first sample data from the training set corresponding to the target task, and obtain the second sample data from the training samples corresponding to the auxiliary task.

[0030] In this application, the auxiliary task is used to assist the network model in making decisions on the target task.

[0031] In some embodiments, the target task includes an event extraction task, and the auxiliary tasks include at least a relation extraction task and a named entity task, and there is an association between the auxiliary tasks and the target task.

[0032] Event extraction tasks refer to document-level event extraction tasks, which aim to extract structured event information from unstructured document text. In other words, document-level event extraction can be the extraction of event arguments from events. For example, the event "consumption" could contain event arguments such as consumer, merchant, and product.

[0033] Among them, event arguments are a subtask of the event extraction task.

[0034] Of course, event extraction can also extract trigger words from events, where trigger words can be used to define the type of event.

[0035] The Named Entity Task aims to extract named entities from document text, where named entities can be people's names, place names, organizations, etc.

[0036] The relation extraction task aims to extract relationships between named entities from document text.

[0037] In addition, both the target task and the auxiliary task are subtasks of information extraction. Information extraction is a type of subtask in natural language processing. All information extraction tasks aim to extract structured data knowledge from unstructured natural language and apply the extracted data knowledge to downstream tasks, such as question answering systems, information retrieval systems, and knowledge graphs.

[0038] In some embodiments, first sample data can be obtained from the training set corresponding to the event extraction task, and second sample data can be obtained from the training samples corresponding to the named entity task and the relation extraction task.

[0039] In some embodiments, event extraction is chosen as the target task because document-level information extraction is similar to document-level event extraction in terms of input text length, and event extraction is a subtask of information extraction, exhibiting a certain similarity between the two. That is, some knowledge contained in other tasks within information extraction can be used by event extraction. After screening, relation extraction and named entity tasks were found to be widely studied in the field of information extraction. Data for these two tasks is relatively easy to obtain; both tasks have document-level datasets and corpora, similar to event extraction. Furthermore, relation extraction and named entity tasks have a strong correlation with event extraction; the entities identified by named entity tasks may be event arguments in event extraction tasks, and event arguments in event extraction tasks are often defined as having some kind of relation in relation extraction tasks. Therefore, relation extraction and named entity tasks are used as auxiliary tasks for event extraction.

[0040] It is worth noting that after obtaining the second sample data for the auxiliary task, the second sample data needs to be processed to obtain data in a similar form to the first sample data, so as to facilitate knowledge transfer and model generalization between the first network model and the second network model.

[0041] Specifically, the document is X = {x1, x2, ..., xn}. We need to obtain the sequence of the target task based on the document. If the target task is an event extraction task, the sequence corresponding to the event extraction task can be Y1 = {y1, y2, ..., ym}, where m is the number of events, yi (i∈[1, m]) represents the i-th event, and yi = {event-type, role1, role2, ..., role...} k}, role d (d∈[1,k]) represents the d-th role, role d ={entity1,entity2,…entityj The sequence corresponding to the relation extraction task can be Y2 = {y1, y2, ..., ym}, where m represents the number of relation tuples, yi (i ∈ [1, m]) represents the i-th relation tuple, yi = {relation-type, head_entity, tail_entity}, where head_entity and tail_entity are the head entity and tail entity in the relation tuple, respectively; the sequence corresponding to the named entity task can be Y3 = {y1, y2, ..., ym}, where m represents the number of named entities, yi (i ∈ [1, m]) represents the i-th named entity, yi = {entity}.

[0042] The first sample data is labeled with the corresponding real information. For example, if the target task is event extraction, the real information labeled in the first sample data is the corresponding real event. If the auxiliary task is relation extraction, the real information labeled in the second sample data is the corresponding real relation.

[0043] Step 12: Train the first network model using the first sample data, and train the second network model using the second sample data.

[0044] The network model includes a first network model and a second network model. The network model can be a BERT (Bidirectional Encoder Representations from Transformer) model, a T5 (Text-to-Text Transfer Transformer) model, or other natural language processing models; there are no restrictions here. However, the first and second network models need to belong to the same type of model, such as both being BERT models or both being T5 models.

[0045] Every natural language processing model has well-initialized parameters. However, if the model is trained using second sample data corresponding to the auxiliary task, and then trained again using first sample data corresponding to the target task, catastrophic forgetting can easily occur. That is, the second sample data from the auxiliary task can disrupt the model's well-initialized parameters, leading to a decrease in the performance of the target task during training. To avoid this problem, a multi-task learning strategy can be used to train the network model.

[0046] Specifically, the first sample data and the second sample data are separated, and the first network model is trained using the first sample data, while the second network model is trained using the second sample data.

[0047] By training simultaneously, compared to training first using sample data from the auxiliary task and then using sample data from the target task, or training first using sample data from the target task and then using sample data from the auxiliary task, problems such as catastrophic forgetting in the network model can be avoided. Furthermore, it is more convenient to observe the performance changes of the target task and thus dynamically adjust the model training strategy.

[0048] In some embodiments, see Figure 2 Step 12 may include the following process:

[0049] Step 21: Obtain the first loss value corresponding to the first network model; and obtain the second loss value corresponding to the second network model.

[0050] Understandably, training a model will result in a loss value. Training algorithms used to train models include Teacher Forcing, Scheduled Sampling, MIXER (Mixed Incremental Cross-Entropy Reinforcement), SCST (Self-critical Sequence Training), etc., and there are no restrictions here.

[0051] In some embodiments, a teacher-mandated training method can be used to train the model and calculate the corresponding loss value. A first loss value can be obtained when training a first network model with first sample data, and a second loss value can be obtained when training a second network model with second sample data.

[0052] Step 22: Determine the first gradient corresponding to each network layer in the first network model based on the first loss value, and determine the second gradient corresponding to each network layer in the second network model based on the second loss value.

[0053] In some embodiments, when training a network model, in order to reduce the introduction of noise and irrelevant knowledge during multi-task learning, the gradient similarity between the network layers of the two network models during training can be used to locate the noise and irrelevant knowledge of the auxiliary task on the network layers of the network model, thereby distinguishing the shared network layers and non-shared network layers between the first network model and the second network model.

[0054] In some embodiments, there are multiple first sample data and second sample data. The first gradient corresponding to each network layer in the first network model can be determined based on each first sample data, and all first gradients are accumulated to obtain the first cumulative gradient. The second gradient corresponding to each network layer in the second network model can be determined based on each second sample data, and all second gradients are accumulated to obtain the second cumulative gradient.

[0055] In some embodiments, a gradient accumulator is initialized for each task, and the data of all tasks are mixed together using a multi-task learning strategy before being learned together. During the training phase, individual task samples are sampled in batches from the mixed data, and the gradient accumulator is used to accumulate the gradients obtained from each backpropagation during each backpropagation.

[0056] Step 23: Determine the gradient similarity between the network layers of the first network model and the second network model based on the first gradient and the second gradient.

[0057] In some embodiments, the gradient similarity between network layers of the first network model and the second network model can be determined based on the first cumulative gradient of each network layer in the first network model and the second cumulative gradient of each network layer in the second network model.

[0058] First, the first network model and the second network model are associated with the network layers according to the hierarchical relationship. Then, the gradient similarity between the associated network layers is determined by using the first gradient and the second gradient of the associated network layers.

[0059] Step 24: Determine the shared and non-shared network layers between the first network model and the second network model based on gradient similarity.

[0060] In some embodiments, the relationship between gradient similarity and a threshold is used to determine which layers in the network model are shared network layers and which are non-shared network layers.

[0061] Specifically, if the gradient similarity is greater than or equal to a threshold, the network layer corresponding to the gradient similarity is determined to be a shared network layer; if the gradient similarity is less than the threshold, the network layer corresponding to the gradient similarity is determined to be a non-shared network layer. For example, the threshold can be 80%, 85%, 90%, or 95%. The specific threshold is set according to the actual accuracy requirements of the network model.

[0062] In some embodiments, the network model can be decoupled at the network layer level, separating network layers with high-relevance gradients from those with low-relevance gradients. Gradient updates are shared in network layers with high-relevance gradients, while they are not shared in network layers with low-relevance gradients. This reduces noise and irrelevant knowledge translation introduced by network layers with low-relevance gradients. For network layers with high-relevance gradients, gradients can be shared between different tasks, while for network layers with low-relevance gradients, gradients can be kept separate between tasks.

[0063] Step 13: Share gradient updates for several shared network layers between the first network model and the second network model; perform gradient updates for the non-shared network layers of the first network model according to the target task; and perform gradient updates for the non-shared network layers of the second network model according to the auxiliary task. The shared network layers and non-shared network layers are determined in advance using gradient similarity.

[0064] In some embodiments, training the first network model and the second network model may include multiple training iterations. After the initial training iterations, shared and non-shared network layers in the first and second network models can be determined using gradient similarity. Then, based on the determined shared and non-shared network layers, gradient updates are shared for several shared network layers between the first and second network models, and gradient updates are performed on the non-shared network layers of the first network model according to the target task.

[0065] Specifically, in shared network layers, the first and second network models share gradients; in non-shared network layers, the first and second network models do not share gradients. For network layers with shared gradients, the first and second network models can perform parameter updates and other operations according to the shared gradients. In non-shared network layers, the first network model updates gradients according to the target task, and the second network model updates gradients according to the auxiliary task.

[0066] In this way, the knowledge of the auxiliary task learned by the second network model can be implicitly transferred to the first network model using a multi-task learning strategy. This method can achieve better training results when the data is sparse.

[0067] The network model contains several network layers. The first network model and the second network model can share the embedding layer. See [link / reference]. Figure 3 The first sample data corresponding to the target task is input into the first network model, and the second sample data corresponding to the auxiliary task is input into the second network model.

[0068] Since network model training typically requires batch training, each batch acquires a corresponding number of first and second sample data. This can be understood as the fusion of sample data from the target task and auxiliary task to form the training sample data for the corresponding batch.

[0069] Specifically, after constructing the first network model and the second network model, the first network model and the second network model are trained using a small batch of training sample data to determine several shared network layers and non-shared network layers between the first network model and the second network model.

[0070] Optionally, after training the first network model and the second network model using each batch of training sample data, a first loss value corresponding to the first network model can be obtained; and a second loss value corresponding to the second network model can be obtained.

[0071] Then, the first gradient corresponding to each network layer in the first network model is determined based on the first loss value. Specifically, the gradient corresponding to each network layer is determined through backpropagation.

[0072] The second gradient corresponding to each network layer in the second network model is determined based on the second loss value; specifically, the gradient corresponding to each network layer is determined through backpropagation.

[0073] Therefore, each network layer in both the first and second network models has a gradient. Then, gradient similarity is calculated between the corresponding network layers in the first and second network models, and shared and non-shared network layers are determined based on the gradient similarity.

[0074] like Figure 3 As shown, the first and second network models share the same embedding layer, but the number of network layers and the connection structure between them are identical. That is, the network layers in the first and second network models correspond to each other according to the feature propagation order. Specifically, the first network layer in the first model corresponds to the first network layer in the second network model, the second network layer in the first model corresponds to the second network layer in the second network model, and so on, with the seventeenth network layer in the first model corresponding to the seventeenth network layer in the second network model, and the eighteenth network layer in the first model corresponding to the eighteenth network layer in the second network model.

[0075] Furthermore, to avoid large errors in a single batch, multiple batches can be used to accumulate the gradients of the network layers in each batch, and the gradient similarity can be determined by accumulating the gradients.

[0076] Based on the gradient similarity described above, the fifth network layer in the first network model and the fifth network layer in the second network model, the sixth network layer in the first network model and the sixth network layer in the second network model, and the eighteenth network layer in the first network model and the eighteenth network layer in the second network model are identified as shared network layers. The fourth network layer in the first network model and the fourth network layer in the second network model, and the seventeenth network layer in the first network model and the seventeenth network layer in the second network model are identified as non-shared network layers.

[0077] Understandable. Figure 3 The illustration is for illustrative purposes only; the specific determination needs to be based on gradient similarity.

[0078] After determining the shared and non-shared network layers, the first and second network models can be trained again using a new batch of training sample data. Gradient updates are shared between the shared network layers of the first and second network models, gradient updates are performed on the non-shared network layers of the first network model according to the target task, and gradient updates are performed on the non-shared network layers of the second network model according to the auxiliary task.

[0079] After meeting the training requirements, refer to Figure 4 The first network model is tested using the test sample data of the test set corresponding to the target task. When the output result of the first network model meets the preset requirements, step 14 is executed.

[0080] Step 14: Use the first network model as the final network model.

[0081] Understandably, the first network model is trained using first sample data corresponding to the target task, while the second network model is trained using second sample data corresponding to the auxiliary task. The model trained in this application is ultimately used to process the target task. After sharing gradient updates for several shared network layers between the first and second network models, and updating the non-shared network layers of the first network model according to the target task, the first network model is used as the final network model.

[0082] Unlike existing technologies, the gradient similarity-based network model training method provided in this application solves the problem of insufficient accuracy of the first network model when the first sample data in the training set corresponding to the target task is sparse by using a second network model trained on an auxiliary task to transfer knowledge to the first network model. Furthermore, by using gradient similarity to pre-determine the shared and non-shared network layers between the first and second network models, when training the first network model using the first sample data in the training set corresponding to the target task, and when training the second network model using the second sample data from the training samples corresponding to the auxiliary task, gradient updates can be shared for several shared network layers between the first and second network models. Gradient updates are performed on the non-shared network layers of the first network model according to the target task, and on the non-shared network layers of the second network model according to the auxiliary task. This alleviates the problem of transferring irrelevant knowledge and noise learned in the second network model from the auxiliary task to the first network model, reducing the transfer of irrelevant knowledge and noise, and thus improving the accuracy of the first network model in completing the target task.

[0083] See Figure 5 , Figure 5 This is a flowchart illustrating an embodiment of the event extraction method provided in this application. The method includes:

[0084] Step 51: Obtain the text corresponding to the event extraction task.

[0085] The text used for the event extraction task can be unstructured text, but it contains structured events.

[0086] Step 52: Input the text into the network model to obtain the event corresponding to the text.

[0087] The network model is the final network model trained using the method described in any of the above embodiments. This final network model has the ability to learn knowledge from other models, which can solve the problem of insufficient accuracy of the trained network model when the first sample data in the training set corresponding to the target task is sparse. Furthermore, the shared and non-shared network layers of the final network model are known in advance. Several shared network layers can share gradient updates, while non-shared network layers are updated according to the target task. This final network model can alleviate the problem of transferring irrelevant knowledge and noise learned from other network models in the auxiliary task to this network model, reducing the transfer of irrelevant knowledge and noise, and thus improving the accuracy of the final network model in completing the target task.

[0088] In some embodiments, text is input into a first network model, and corresponding events can be extracted from the text.

[0089] Unlike existing technologies, the event extraction method provided in this application can use a trained network model to extract events from text in order to obtain the corresponding events.

[0090] See Figure 6 , Figure 6 This is a schematic diagram of an embodiment of the electronic device provided in this application. The electronic device 60 includes a memory 601 and a processor 602. The memory 601 is used to store computer programs, and the processor 602 is used to execute the computer programs to implement the methods described in any of the above embodiments, which will not be described in detail here.

[0091] See Figure 7 , Figure 7 This is a schematic diagram of an embodiment of a computer-readable storage medium provided in this application. The computer-readable storage medium 70 is used to store a computer program 701. When the computer program 701 is executed by a processor, it is used to implement the methods described in any of the above embodiments, which will not be described again here.

[0092] In summary, the gradient similarity-based network model training method provided in this application can solve the problem of insufficient accuracy of the first network model when the first sample data in the training set corresponding to the target task is sparse by transferring knowledge from the second network model trained using the auxiliary task to the first network model. Furthermore, by using gradient similarity to pre-determine the shared and non-shared network layers between the first and second network models, when training the first network model using the first sample data in the training set corresponding to the target task, and training the second network model using the second sample data from the training samples corresponding to the auxiliary task, gradient updates can be shared for several shared network layers between the first and second network models. Gradient updates are performed on the non-shared network layers of the first network model according to the target task, and on the non-shared network layers of the second network model according to the auxiliary task. This alleviates the problem of transferring irrelevant knowledge and noise learned in the second network model from the auxiliary task to the first network model, reducing the transfer of irrelevant knowledge and noise, and thus improving the accuracy of the first network model in completing the target task.

[0093] The processor involved in this application may be referred to as a CPU (Central Processing Unit), which may be an integrated circuit chip, or a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0094] The storage media used in this application include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), or optical discs.

[0095] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An event extraction method, characterized in that, The method includes: Retrieve the text corresponding to the event extraction task; The text is input into a network model to obtain the event corresponding to the text. The network model includes a first network model and a second network model, and the final network model is obtained by training the network model using the following method: Obtain the first sample data from the training set corresponding to the target task, and obtain the second sample data from the training samples corresponding to the auxiliary task; Obtain the first loss value corresponding to the first network model; and obtain the second loss value corresponding to the second network model; The first gradient corresponding to each network layer in the first network model is determined based on the first loss value, and the second gradient corresponding to each network layer in the second network model is determined based on the second loss value. The gradient similarity between the network layers of the first network model and the second network model is determined based on the first gradient and the second gradient. If the gradient similarity is greater than or equal to the threshold, then the network layer corresponding to the gradient similarity is determined to be a shared network layer. If the gradient similarity is less than the threshold, then the network layer corresponding to the gradient similarity is determined to be a non-shared network layer. Gradients are shared between several shared network layers of the first network model and the second network model to update parameters according to the shared gradients. Gradients are updated for the non-shared network layers of the first network model according to the target task, and gradients are updated for the non-shared network layers of the second network model according to the auxiliary task. The shared network layers and the non-shared network layers are pre-determined using gradient similarity. The shared network layers and the non-shared network layers do not share gradients. The target task includes an event extraction task, and the auxiliary tasks include at least a relation extraction task and a named entity recognition task. The first network model is used as the final network model.

2. The method according to claim 1, characterized in that, There are multiple sets of the first sample data and the second sample data; Determining the first gradient corresponding to each network layer in the first network model based on the first loss value includes: Determine the first gradient corresponding to each network layer in the first network model based on each of the first sample data; Accumulate all of the first gradients to obtain the first cumulative gradient; The step of determining the second gradient corresponding to each network layer in the second network model based on the second loss value includes: Determine the second gradient corresponding to each network layer in the second network model based on each of the second sample data; Accumulate all the second gradients to obtain the second cumulative gradient.

3. The method according to claim 2, characterized in that, Determining the gradient similarity between network layers of the first network model and the second network model based on the first gradient and the second gradient includes: The gradient similarity between the network layers of the first network model and the second network model is determined based on the first cumulative gradient and the second cumulative gradient.

4. The method according to claim 1, characterized in that, The first network model and the second network model share the same embedding layer.

5. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory being used to store a computer program and the processor being used to execute the computer program to implement the method as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed by a processor, is used to implement the method as described in any one of claims 1-4.

Images