Training method, model, device and migration method of corpus annotation migration model

Abstract

The present application discloses a training method, model, device and migration method of a corpus annotation migration model, wherein the training method comprises: initializing the corpus annotation migration model, wherein the corpus annotation migration model comprises a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer; based on the source corpus with annotations, performing entity extraction task training on the initialized corpus annotation migration model to obtain an updated corpus annotation migration model; performing corpus annotation migration task training on the updated corpus annotation migration model based on the source language and the unannotated target language to obtain the final corpus annotation migration model. The present application not only uses the knowledge of the two languages ​​themselves, but also effectively connects the relationship between the two different languages, and uses the premise knowledge that has been annotated in the source language to migrate the annotated named entities to the target language, so that the target language also has annotated data.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and specifically to a training method, model, device and migration method of a corpus annotation migration model. Background Art

[0002] In Natural Language Processing (NLP), machine learning models are usually used to identify entities that are not very regular. The machine learning models used to identify entities need to rely on a large number of labeled predictions, but entity recognition and labeling are time-consuming and labor-intensive.

[0003] When training machine learning models, it is common to encounter situations where the corpus is insufficient or even completely absent. Summary of the invention

[0004] In response to the above problems, the embodiments of the present application provide a training method, model, device and migration method of a corpus annotation migration model to overcome or partially overcome the shortcomings of the prior art.

[0005] In a first aspect, an embodiment of the present application provides a method for training a corpus annotation transfer model, comprising:

[0006] Initializing a corpus annotation migration model, the corpus annotation migration model includes a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer; wherein the source language semantic recognition layer and the target language semantic recognition layer are connected through the matcher Mapper, and the source language semantic recognition layer and the target language semantic recognition layer are also connected in sequence to the fully connected layer and the CRF layer;

[0007] Based on the annotated source corpus, the initialized corpus annotation transfer model is trained on the entity extraction task to obtain an updated corpus annotation transfer model.

[0008] The updated corpus annotation transfer model is trained on a corpus annotation transfer task based on the source language and the unannotated target language to obtain a final corpus annotation transfer model.

[0009] In a second aspect, the embodiment of the present application further provides a corpus annotation migration model, the corpus annotation migration model includes a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer;

[0010] The source language semantic recognition layer and the target language semantic recognition layer are connected through the matcher Mapper, and the source language semantic recognition layer and the target language semantic recognition layer are also connected in sequence to a fully connected layer and the CRF layer.

[0011] In a third aspect, an embodiment of the present application further provides a training device for a corpus annotation transfer model, wherein the device is used to implement any of the aforementioned methods.

[0012] In a fourth aspect, an embodiment of the present application further provides a corpus annotation migration method, wherein the method is implemented by using any of the corpus annotation migration models described above;

[0013] The method comprises:

[0014] Determining a first semantics of a source corpus based on the source language semantic recognition layer, and determining a second semantics of a target corpus based on the target language semantic recognition layer, wherein the source corpus has annotations and the target corpus does not have annotations;

[0015] Based on the matcher Mapper, determining an association relationship between the first semantics and the second semantics;

[0016] Based on the fully connected layer and the CRF layer, the annotations of the source language are transferred to the target corpus according to the association relationship.

[0017] In a fifth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to perform any of the above methods.

[0018] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes any of the above methods.

[0019] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0020] The application designs a corpus annotation migration model. During the training process, the source corpus with annotations is first used to train the corpus annotation migration model for entity recognition tasks to achieve model optimization, especially the optimization of the matcher Mapper; then the source corpus and the target corpus are input into the corpus annotation migration model together. During the corpus prediction process, the correspondence between the characters or words in the source corpus and the characters or words in the target corpus is established. Based on this correspondence, the knowledge of the source corpus is transferred to the target corpus, so that the target corpus also has annotations. This application not only uses the pre-trained knowledge of the two languages ​​itself, but also effectively connects the relationship between the two different languages. Using the premise knowledge that the source language has been annotated, the annotated named entities are transferred to the target language, so that the target language also has annotated data. The model provided by this application can quickly obtain a large amount of annotated corpus, providing a solid foundation for model training; especially in the early stage of lack of data, the recognition model method can be quickly put into use to identify entities that are difficult to identify by rules, and quickly accumulate data for subsequent training and other work. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A schematic diagram of a process of training a corpus annotation transfer model according to an embodiment of the present application is shown;

[0023] Figure 2 A schematic diagram of the structure of a corpus annotation migration model according to an embodiment of the present application is shown;

[0024] Figure 3-a A schematic diagram of the structure of the source language semantic recognition layer 210 according to an embodiment of the present application is shown;

[0025] Figure 3-b A schematic diagram of the structure of the target language semantic recognition layer 220 according to an embodiment of the present application is shown;

[0026] Figure 4 A schematic diagram showing the flow of the source language semantic recognition layer 210 processing corpus according to an embodiment of the present application is shown;

[0027] Figure 5 A schematic diagram showing a flow chart of a corpus annotation migration method according to an embodiment of the present application is shown;

[0028] Figure 6 A schematic diagram of the structure of a training device for a corpus annotation transfer model according to an embodiment of the present application is shown;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0031] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0032] Named Entity Recognition (NER) refers to identifying the boundaries and categories of entities in text. Named entity recognition is a basic technology in text processing and is widely used in natural language processing, recommendation systems, knowledge graphs and other fields, such as entity-based user profiling and entity-based recall in recommendation systems.

[0033] With the development of technology, named entity recognition can be roughly divided into rule-based methods, statistical methods, and machine learning model-based methods, such as deep learning methods and attention methods. Among them, the training of machine learning models for entities is highly dependent on labeled data, and the labeling requirements are different in different fields and languages.

[0034] In the prior art, for language texts that lack annotations or are not annotated enough to train the model, we can only rely on cold start, usually relying on rules to extract named entities. For entities such as names and addresses that have no obvious rules, it is difficult to extract them effectively. The annotation rules for entities are relatively complicated and require high standards. When training the model, we often encounter situations where the corpus is insufficient or even scarce.

[0035] In response to this situation, the present application proposes a training method for a corpus annotation transfer model. The present application uses the knowledge of the two languages ​​themselves and effectively connects the two different languages. By using the premise knowledge that has been annotated in the source language, the annotated named entities are migrated to the target language, so that the target language also has annotated data. Furthermore, a conventional named entity extraction model can be used for training, and finally the named entity recognition function of the target language is achieved. In this way, in the early stage when there is a lack of data, the model can be quickly put into use to identify entities that are difficult to identify by rules, laying the foundation for accumulating data in the future and further optimizing the model.

[0036] Figure 1 A flow chart of a training method for a corpus annotation transfer model according to an embodiment of the present application is shown. Figure 1 The method shown in the present application at least includes steps S110 to S130:

[0037] Step S110: Initialize the corpus annotation migration model, which includes a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer; wherein the source language semantic recognition layer and the target language semantic recognition layer are connected through the matcher Mapper, and the source language semantic recognition layer and the target language semantic recognition layer are also connected in sequence to the fully connected layer and the CRF layer.

[0038] In view of the shortcomings of the existing technology, this application designs a network model, which can be referred to in detail. Figure 2 , Figure 2 A structural schematic diagram of a corpus annotation migration model according to an embodiment of the present application is shown. In the corpus annotation migration model 200, a source language semantic recognition layer 210, a target language semantic recognition layer 220, a matcher Mapper230, a fully connected layer 240 and a CRF layer 250 are included; wherein the source language semantic recognition layer 210 and the target language semantic recognition layer 220 are connected via the matcher Mapper230, and the source language semantic recognition layer 210 and the target language semantic recognition layer 220 are also connected in sequence to the fully connected layer 240 and the CRF layer 250.

[0039] More specifically, the input ends of the source language semantic recognition layer 210 and the target language semantic recognition layer 220 are respectively used to receive training corpus or prediction corpus. After the semantics are predicted by the source language semantic recognition layer 210 and the target language semantic recognition layer 220, in the semantic prediction process, the matcher is used to determine the association relationship between the source language and the target language. The output results of the source language semantic recognition layer 210 and the target language semantic recognition layer 220 are input into the fully connected layer through certain processing, and then enter the CRF layer, and the whole process ends.

[0040] First, the corpus annotation transfer model can be initialized by assigning initial parameters to the corpus annotation transfer model. In the subsequent training process, the initial parameters are continuously adjusted as the iterations proceed. When the convergence standard is reached, the training of the model is completed.

[0041] Step S120: Based on the annotated source corpus, the initialized corpus annotation transfer model is trained on the entity extraction task to obtain an updated corpus annotation transfer model.

[0042] In this application, the source corpus refers to the corpus with annotations; the target corpus refers to the corpus without annotations. The goal of this application is to transfer the existing "knowledge" of the source language to the target corpus. It should be noted that in this application, it is mainly applicable to different languages. The source corpus corresponds to the source language, and the target corpus corresponds to the target language. The source language and the target language can be different languages, such as the source language is Chinese and the target language is English.

[0043] Based on the source corpus, the process of training the initialized corpus annotation transfer model for the entity extraction task can be specifically described as follows: the source corpus is input into the source language semantic recognition layer and the target language semantic recognition layer respectively to obtain a first semantic vector and a second semantic vector respectively, wherein the first semantic vector and the second semantic vector have an association relationship, and the association relationship is determined by the matcher Mapper according to the first semantic vector and the second semantic vector; the first semantic vector and the second semantic vector are concatenated to obtain a first fusion vector; the first fusion vector is passed through the fully connected layer and the CRF layer in sequence to realize entity extraction; according to the entity extraction result, the initial parameters of the corpus annotation transfer model are updated to obtain an updated corpus annotation transfer model.

[0044] During training, the target language is first shielded, and the result of the entity extraction task is trained separately based on the corpus of the source language. Specifically, the source corpus with annotations is input into the source language semantic recognition layer 210 and the target language semantic recognition layer 220 of the corpus annotation transfer model. The function of the source language semantic recognition layer is to recognize the semantics of the corpus. When the source language is used for training, the source corpus is input into the source language semantic recognition layer 210 and the target language semantic recognition layer 220 respectively, and the semantics of the source corpus are predicted respectively, so as to obtain the semantic knowledge of the language that is fully retained. The output of the source language semantic recognition layer 210 is recorded as the first semantic vector, and the output of the target language semantic recognition layer 220 is recorded as the second semantic vector. The function of the matcher Mapper is to match and associate the semantic vectors of the two languages, that is, after obtaining the first semantic vector and the second semantic vector, the matcher Mapper matches and associates them at the word level. Before the first semantic vector and the second semantic vector enter the fully connected layer, this matching association has been established.

[0045] Then the first semantic vector is concatenated and fused with the second semantic vector to obtain a fused vector, which is input into the task-related fully connected layer and CRF layer. After the fused vector passes through the fully connected layer and CRF layer, the entity extraction task is completed. The entity extraction task can be understood as follows: if the source corpus is "I am Zhang San", the name entity "Zhang San" can be extracted. The entity extraction results include but are not limited to the entity name, entity start, end, entity label, etc.

[0046] In the above model, the fully connected layer plays the role of a classifier in the model, mapping the learned distributed feature representation to the sample label space, and the CRF layer adds some constraints to the final predicted label to ensure the legitimacy of the predicted label.

[0047] During the training process, as the iterations proceed, the parameters of each model are continuously adjusted according to the results of the previous round of entity extraction to obtain an updated corpus annotation migration model, especially the parameters of the matcher Mapper. During the training process, the parameters of the matcher Mapper are continuously adjusted to obtain an optimized matcher. Subsequently, the optimized matcher Mapper is mainly used to fit the source language and the target language, that is, to determine the relationship between the source language and the target language, thereby realizing knowledge transfer.

[0048] Step S130: performing corpus annotation transfer task training on the updated corpus annotation transfer model based on the source language and the unannotated target language to obtain a final corpus annotation transfer model.

[0049] Then, the annotated source language and the unannotated target language are output to the updated corpus annotation transfer model obtained through the training, and the corpus annotation transfer task training is performed. During the training process, iterations are continuously performed until the iteration standard is met.

[0050] Depend on Figure 1 It can be seen from the method shown that the present application designs a corpus annotation migration model. During the training process, the source corpus with annotations is first used to train the corpus annotation migration model for the entity recognition task to achieve model optimization, especially the optimization of the matcher Mapper; then the source corpus and the target corpus are input into the corpus annotation migration model together. During the corpus prediction process, the correspondence between the characters or words in the source corpus and the characters or words in the target corpus is established. Based on this correspondence, the knowledge of the source corpus is transferred to the target corpus, so that the target corpus also has annotations. The present application not only uses the pre-trained knowledge of the two languages ​​itself, but also effectively connects the relationship between the two different languages. Using the premise knowledge that the source language has been annotated, the annotated named entities are transferred to the target language, so that the target language also has annotated data. The model provided by the present application can quickly obtain a large amount of annotated corpus, which provides a solid foundation for the training of the model; especially in the early stage of lack of data, the method of the recognition model can be quickly put into use to identify entities that are difficult to identify by rules, and quickly accumulate data for subsequent training and other work.

[0051] In some embodiments of the present application, in the above method, the updated corpus annotation transfer model is trained for a corpus annotation transfer task based on the source language and the unannotated target language to obtain a final corpus annotation transfer model, including: inputting the source corpus into the source language semantic recognition layer to determine a third semantic vector of the source corpus; inputting the target corpus into the target language semantic recognition layer to determine a fourth semantic vector of the target corpus, wherein the third semantic vector and the fourth semantic vector have an association relationship, and the association relationship is determined by the matcher Mapper based on the third semantic vector and the fourth semantic vector; concatenating the third semantic vector and the fourth semantic vector to obtain a second fusion vector; allowing the second fusion vector to pass through the fully connected layer and the CRF layer in sequence to migrate the annotations of the source corpus to the target corpus; and updating the parameters of the updated corpus annotation transfer model according to the annotation migration results to obtain the final corpus annotation migration model.

[0052] That is to say, the source corpus is input into the source language semantic recognition layer, the target corpus is input into the target language semantic recognition layer, and the third semantic vector is obtained from the source language semantic recognition layer, and the fourth semantic vector is obtained from the target language semantic recognition layer. The third semantic vector and the fourth semantic vector are concatenated to obtain a first fusion vector; the first fusion vector is passed through the fully connected layer and the CRF layer in sequence, so that the "knowledge" of the source corpus can be transferred to the target corpus.

[0053] Specifically, we need to determine the association between the source corpus and the target corpus first. This relationship is determined based on the matcher Mapper. In the process of semantic prediction of the source corpus and the target corpus at the same time, the matcher Mapper will establish an association between characters or words with the same semantics. This association can be understood as a mapping or correspondence. For example, if the source corpus is "I am Zhangsan" and the target corpus is "I am Zhangsan", then it can be considered that "I" is associated with "I", "I am" is associated with "am", and "Zhangsan" is associated with "Zhangsan".

[0054] The matcher Mapper can match the semantically preserved vector output by the semantic recognition layer with the characters or words in the "dictionary" to determine the association between the source corpus and the target corpus. The dictionary can be derived from the prior art, and characters and vectors that are not available in the prior art can also be manually established. For example, if the source language is Chinese and the target language is English, and the "I" in the source language corresponds to the "I" in the target language in the dictionary, then the association between the source corpus and the target corpus can be established based on this correspondence in the dictionary.

[0055] From this association relationship, we can find out which character or word in the target corpus corresponds to a character or word in the source corpus, and the position of this character or word in the target language.

[0056] Finally, according to the relationship between the source corpus and the target corpus, the source corpus entity tags are assigned to the corresponding characters or words in the target corpus, thus obtaining the target corpus with annotations, thus completing the corpus annotation migration task. This is a round of training process, and after multiple rounds of iterations and updates, the final corpus annotation migration model is obtained.

[0057] In some embodiments of the present application, in the above method, the source language semantic recognition layer 210 includes two BERT layers connected in sequence, Figure 3-a A structural schematic diagram of a source language semantic recognition layer 210 according to an embodiment of the present application is shown, wherein the source language semantic recognition layer 210 includes a first BERT layer 211 and a second BERT layer 212, wherein the input of the first BERT layer 211 serves as the input of the source language semantic recognition layer 210, the output of the first BERT layer 211 is connected to the input of the second BERT layer 212, and the output of the second BERT layer 212 serves as the output of the source language semantic recognition layer 210.

[0058] In some embodiments of the present application, Figure 4 As shown, Figure 4 The flowchart of the source language semantic recognition layer 210 processing the corpus according to an embodiment of the present application is shown. The input of the first BERT layer 211 is the character feature input of the corpus. For example, if it is expected to be "I am Zhang XX", the corpus is divided into multiple characters according to the characters, that is, the corpus is divided into "I, is, Zhang, XX, XX", as the input of the first BERT layer 211. After the corpus passes through the first BERT layer 211, the character vector feature is obtained. The character vector feature is combined with the word feature input of the corpus as the input of the second BERT layer 212. The word feature input is to divide the corpus into multiple characters according to the words. For example, "I am Zhang XX" is divided into "I, is, Zhang XX". After the output from the second BERT layer 212, the final feature vector (final embedding) of the input text of the fully connected layer is obtained. The final feature vector combines the dual features of the character level and word level of the corpus.

[0059] The final embedding layer is fused with the output of the target language semantic recognition layer 220 and enters the subsequent fully connected layer. The purpose of setting two BERT layers here is to combine the dual features of the expected character level and word level.

[0060] It should be noted that if the target language and the source language have the same character set, they share a character vector set, such as Spanish and English. Otherwise, a separate character vector set for the language is used.

[0061] In some embodiments of the present application, in the above method, the structure of the target language semantic recognition layer 220 is consistent with that of the source language semantic recognition layer 210. Specifically, Figure 3-b A structural schematic diagram of a target language semantic recognition layer 220 according to an embodiment of the present application is shown, wherein the target language semantic recognition layer 220 includes a third BERT layer 221 and a fourth BERT layer 222, wherein the input of the third BERT layer 221 is used as the input of the target language semantic recognition layer 220, the output of the third BERT layer 221 is connected to the input of the fourth BERT layer 222, and the output of the fourth BERT layer 222 is used as the output of the target language semantic recognition layer 220.

[0062] The process of the corpus undergoing the target language semantic recognition layer 220 is similar to the process of the corpus undergoing the source language semantic recognition layer 210. Figure 4 , no more details.

[0063] The functions of the source language semantic recognition layer 210 and the target language semantic recognition layer 220 are the same, and are generally used to recognize the semantics of the corpus. Specifically, the functions of each layer in the target language semantic recognition layer 220 are consistent with those of the source language semantic recognition layer 210, and will not be repeated here. The difference is that the input of the source language semantic recognition layer 210 is annotated corpus, while the input of the target language semantic recognition layer 220 is unannotated corpus.

[0064] The process of implementing the corpus annotation migration method using the corpus annotation migration model of the present application can be briefly described as follows:

[0065] A1: Determine a first semantics of a source corpus based on the source language semantic recognition layer, and determine a second semantics of a target corpus based on the target language semantic recognition layer, wherein the source corpus has annotations and the target corpus does not have annotations.

[0066] A2: Based on the matcher Mapper, determine the association relationship between the first semantics and the second semantics.

[0067] A3: Based on the fully connected layer and the CRF layer, the annotations of the source language are transferred to the target corpus according to the association relationship.

[0068] The process of the corpus annotation migration method of the present application is consistent with the training process of the corpus annotation migration model and will not be described in detail.

[0069] Please also refer to Figure 2 , Figure 3-a and Figure 3-b , this embodiment adopts Figure 2 , Figure 3-a and Figure 3-b The corpus annotation migration model jointly constituted is implemented. Specifically, Figure 5 A schematic diagram showing a flow chart of a corpus annotation migration method according to an embodiment of the present application is shown, from Figure 5 It can be seen that this embodiment includes:

[0070] The source language corpus is obtained with annotations, namely the source corpus; the target language corpus is obtained without annotations, namely the target corpus.

[0071] The source corpus is used to train the corpus annotation transfer model to obtain a trained corpus annotation transfer model.

[0072] The source corpus is input into the Embedding layer of the first BERT layer 211 , and the target corpus is input into the Embedding layer of the third BERT 221 . The first output result is obtained from the second BERT layer 212 , and the second output result is obtained from the fourth BERT layer 222 .

[0073] The matcher Mapper 230 performs word-level matching association on the first output result and the second output result.

[0074] The first output result and the second output result after matching and association are concatenated together and input into the fully connected layer 240 .

[0075] The output of the fully connected layer 240 is used as the input of the CRF layer 250, and the output of the CRF layer 250 has the labeled target corpus.

[0076] In some embodiments of the present application, the target corpus obtained above can be used for training various models, such as training a pre-trained model based on the obtained target corpus to optimize the parameters of the training model and obtain a target model; wherein the training model includes but is not limited to a model obtained by combining a BiLSTM model with a CRF layer, and a model obtained by combining a BERT model with a CRF layer. The obtained target corpus is particularly suitable for a named entity recognition model. By using the present application, a large amount of annotated target corpus can be quickly obtained. These corpora can be used to train conventional named entity models, and finally the named entity recognition function of the target language is achieved. In this way, in the early stage of lack of data, the model can be quickly put into use to identify entities that are difficult to identify by rules, laying a foundation for accumulating data and further optimizing the model in the future.

[0077] Figure 6 The structural diagram of the training device of the corpus annotation transfer model according to one embodiment of the present application is shown. Figure 6It can be seen that the training device 600 of the corpus annotation transfer model includes:

[0078] The initialization unit 610 is used to initialize the corpus annotation migration model, which includes a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer; wherein the source language semantic recognition layer and the target language semantic recognition layer are connected through the matcher Mapper, and the source language semantic recognition layer and the target language semantic recognition layer are also connected in sequence to the fully connected layer and the CRF layer.

[0079] The first training unit 620 is used to perform entity extraction task training on the initialized corpus annotation transfer model based on the source corpus with annotations to obtain an updated corpus annotation transfer model.

[0080] The second training unit 630 is used to perform corpus annotation transfer task training on the updated corpus annotation transfer model based on the source language and the unannotated target language to obtain a final corpus annotation transfer model.

[0081] In some embodiments of the present application, in the above-mentioned device, the first training unit 620 is used to input the source corpus into the source language semantic recognition layer and the target language semantic recognition layer, respectively, to obtain a first semantic vector and a second semantic vector, respectively, wherein the first semantic vector and the second semantic vector have an association relationship, and the association relationship is determined by the matcher Mapper based on the first semantic vector and the second semantic vector; the first semantic vector and the second semantic vector are concatenated to obtain a first fusion vector; the first fusion vector is passed through the fully connected layer and the CRF layer in sequence to realize entity extraction; according to the entity extraction result, the initial parameters of the corpus annotation migration model are updated to obtain an updated corpus annotation migration model.

[0082] In some embodiments of the present application, in the above-mentioned device, the second training unit 630 is used to input the source corpus into the source language semantic recognition layer to determine the third semantic vector of the source corpus; input the target corpus into the target language semantic recognition layer to determine the fourth semantic vector of the target corpus, wherein the third semantic vector and the fourth semantic vector have an association relationship, and the association relationship is determined by the matcher Mapper based on the third semantic vector and the fourth semantic vector; concatenate the third semantic vector and the fourth semantic vector to obtain a second fusion vector; make the second fusion vector pass through the fully connected layer and the CRF layer in sequence to migrate the annotations of the source corpus to the target corpus; according to the annotation migration results, update the parameters of the updated corpus annotation migration model to obtain the final corpus annotation migration model.

[0083] In some embodiments of the present application, in the above-mentioned device, the source language semantic recognition layer includes a first BERT layer and a second BERT layer; the second training unit 630 is used to perform character segmentation and word segmentation on the source corpus to obtain a first character vector and a first word vector of the source corpus; the first character vector is input into the first BERT layer to obtain a first character feature vector of the source corpus; the first word vector and the first character feature vector are input into the second BERT layer to obtain a first semantic vector of the source corpus.

[0084] In some embodiments of the present application, in the above-mentioned device, the target language semantic recognition layer includes a third BERT layer and a fourth BERT layer; the second training unit 630 is used to perform character segmentation and word segmentation on the target corpus to obtain a second character vector and a second word vector of the target corpus; the second character vector is input into the third BERT layer to obtain a second character feature vector of the target corpus; the second word vector and the second character feature vector are input into the fourth BERT layer to obtain a fourth semantic vector of the target corpus.

[0085] It can be understood that the training device for the above-mentioned corpus annotation transfer model can implement the various steps of the training method for the corpus annotation transfer model provided in the aforementioned embodiment. The relevant explanations on the training method for the corpus annotation transfer model are applicable to the training device for the corpus annotation transfer model and will not be repeated here.

[0086] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 7 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0087] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0088] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0089] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a training device for the corpus annotation migration model at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0090] Initializing a corpus annotation migration model, the corpus annotation migration model includes a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer; wherein the source language semantic recognition layer and the target language semantic recognition layer are connected through the matcher Mapper, and the source language semantic recognition layer and the target language semantic recognition layer are also connected in sequence to the fully connected layer and the CRF layer;

[0091] Based on the annotated source corpus, the initialized corpus annotation transfer model is trained on the entity extraction task to obtain an updated corpus annotation transfer model.

[0092] The updated corpus annotation transfer model is trained on a corpus annotation transfer task based on the source language and the unannotated target language to obtain a final corpus annotation transfer model.

[0093] The above application Figure 6The method performed by the training device of the corpus annotation transfer model disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0094] The electronic device may also perform Figure 6 A method for executing a training device for a corpus annotation transfer model in a method for executing a training device for a corpus annotation transfer model in a Figure 6 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0095] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 6 The method executed by the training device of the corpus annotation transfer model in the illustrated embodiment is specifically used to execute:

[0096] Initializing a corpus annotation migration model, the corpus annotation migration model includes a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer; wherein the source language semantic recognition layer and the target language semantic recognition layer are connected through the matcher Mapper, and the source language semantic recognition layer and the target language semantic recognition layer are also connected in sequence to the fully connected layer and the CRF layer;

[0097] Based on the annotated source corpus, the initialized corpus annotation transfer model is trained on the entity extraction task to obtain an updated corpus annotation transfer model.

[0098] The updated corpus annotation transfer model is trained on a corpus annotation transfer task based on the source language and the unannotated target language to obtain a final corpus annotation transfer model.

[0099] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0104] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0105] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0108] The above is only the embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A training method for a corpus annotation transfer model, characterized in that: include: Initializing a corpus annotation migration model, the corpus annotation migration model includes a source language semantic recognition layer, a target language semantic recognition layer, a matcher Mapper, a fully connected layer and a CRF layer; wherein the source language semantic recognition layer and the target language semantic recognition layer are connected through the matcher Mapper, and the source language semantic recognition layer and the target language semantic recognition layer are also connected in sequence to the fully connected layer and the CRF layer; Based on the annotated source corpus, the initialized corpus annotation transfer model is trained on the entity extraction task to obtain an updated corpus annotation transfer model. Based on the source corpus and the unlabeled target corpus, the updated corpus annotation transfer model is trained on a corpus annotation transfer task to obtain a final corpus annotation transfer model; The method of performing entity extraction task training on the initialized corpus annotation transfer model based on the annotated source corpus to obtain an updated corpus annotation transfer model includes: Input the source corpus into the source language semantic recognition layer and the target language semantic recognition layer respectively, and obtain a first semantic vector and a second semantic vector respectively, wherein the first semantic vector and the second semantic vector have an association relationship, and the association relationship is determined by the matcher Mapper according to the first semantic vector and the second semantic vector; Concatenate the first semantic vector and the second semantic vector to obtain a first fusion vector; Allowing the first fusion vector to pass through the fully connected layer and the CRF layer in sequence to achieve entity extraction; According to the entity extraction result, the initial parameters of the corpus annotation transfer model are updated to obtain an updated corpus annotation transfer model; The step of performing corpus annotation migration task training on the updated corpus annotation migration model based on the source corpus and the unannotated target corpus to obtain a final corpus annotation migration model includes: Inputting the source corpus into the source language semantic recognition layer to determine a third semantic vector of the source corpus; Inputting the target corpus into the target language semantic recognition layer to determine a fourth semantic vector of the target corpus, wherein the third semantic vector and the fourth semantic vector have an association relationship, and the association relationship is determined by the matcher Mapper according to the third semantic vector and the fourth semantic vector; Concatenate the third semantic vector and the fourth semantic vector to obtain a second fusion vector; Allowing the second fused vector to pass through the fully connected layer and the CRF layer in sequence to transfer the annotations of the source corpus to the target corpus; According to the annotation migration result, the parameters of the updated corpus annotation migration model are updated to obtain a final corpus annotation migration model.

2. The method according to claim 1, characterized in that The source language semantic recognition layer includes a first BERT layer and a second BERT layer; The step of inputting the source corpus into the source language semantic recognition layer to determine a third semantic vector of the source corpus includes: Performing character segmentation and word segmentation on the source corpus to obtain a first character vector and a first word vector of the source corpus; Input the first character vector into the first BERT layer to obtain the first character feature vector of the source corpus; The first word vector and the first character feature vector are input into the second BERT layer to obtain a third semantic vector of the source corpus.

3. The method according to claim 1, characterized in that The target language semantic recognition layer includes a third BERT layer and a fourth BERT layer; The step of inputting the target corpus into the target language semantic recognition layer to determine a fourth semantic vector of the target corpus includes: Performing character segmentation and word segmentation on the target corpus to obtain a second character vector and a second word vector of the target corpus; Input the second character vector into the third BERT layer to obtain a second character feature vector of the target corpus; The second word vector and the second character feature vector are input into the fourth BERT layer to obtain a fourth semantic vector of the target corpus.

4. A training device for a corpus annotation transfer model, characterized in that: The device is used to implement the method described in any one of claims 1 to 3.

5. A corpus annotation migration method, characterized in that: The method is implemented by using a corpus annotation transfer model trained by the method of any one of claims 1 to 3; The method comprises: Determining a first semantics of a source corpus based on the source language semantic recognition layer, and determining a second semantics of a target corpus based on the target language semantic recognition layer, wherein the source corpus has annotations and the target corpus does not have annotations; Based on the matcher Mapper, determining an association relationship between the first semantics and the second semantics; Based on the fully connected layer and the CRF layer, the annotations of the source corpus are migrated to the target corpus according to the association relationship.

Images