Term extraction method, apparatus, electronic device, and storage medium

Abstract

The application provides a term extraction method and device, electronic equipment and storage medium, the method comprises: performing word segmentation processing on input corpus of a to-be-extracted term to obtain a plurality of candidate words; for any candidate word, based on a word embedding language model, a word vector corresponding to the target candidate word is obtained; based on a character embedding language model, a character vector corresponding to each character in the target candidate word is obtained, and a sum vector of each character vector is further obtained; according to the similarity comparison result between the word vector and the sum vector, it is determined whether the target candidate word is extracted as a term; wherein the word embedding language model and the character embedding language model are both obtained by training a pre-trained language model based on sample corpus in the vertical industry to which the input corpus belongs. Thus, manual annotation of training data is not required, the term extraction cost is reduced, and accurate extraction of terms in vertical industry corpus can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a term extraction method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the normal translation production process, due to the consistency requirement for the translation of some key words (such as special nouns, new words, ambiguous words, etc.), the translator will read through the translation project document before starting the translation project, extract the intermediate key words as "terms", and uniformly translate all "terms" in the entire project according to strict consistency.

[0003] Since the translation project is generally large, it is time-consuming to read through and extract terms, and the current algorithm model related to named entity recognition (NER) that can assist in term extraction generally requires manual annotation of training data, which is costly, and if a general NER model is used, it cannot solve the problem of vertical industry term extraction. Therefore, how to propose a method that does not require manual annotation and can realize vertical industry term extraction is an important topic that the industry needs to solve. SUMMARY

[0004] In view of the problems in the prior art, the present application provides a term extraction method, device, electronic equipment and storage medium.

[0005] In a first aspect, the present application provides a term extraction method, comprising:

[0006] performing word segmentation processing on input corpus of terms to be extracted to obtain a plurality of candidate words;

[0007] For any of the candidate words, based on a word embedding language model, a word vector corresponding to a target candidate word is obtained; based on a character embedding language model, a character vector corresponding to each character in the target candidate word is obtained, and a sum vector of the character vectors is further obtained;

[0008] determining whether to extract the target candidate word as a term according to a similarity comparison result between the word vector and the sum vector;

[0009] The word embedding language model and the character embedding language model are both obtained by training a pre-trained language model based on sample corpus in a vertical industry to which the input corpus belongs.

[0010] Optionally, the determining whether to extract the target candidate word as a term according to the similarity comparison result between the word vector and the sum vector comprises:

[0011] if the difference between the word vector and the sum vector is greater than or equal to a preset threshold, the target candidate word is extracted as a term; or

[0012] if the difference between the word vector and the sum vector is less than a preset threshold, it is determined that the target candidate word is not extracted as a term.

[0013] Optionally, the pre-trained language model is trained based on sample corpus in a vertical industry to which the input corpus belongs to obtain the word embedding language model and the character embedding language model, including:

[0014] The sample corpus is subjected to word segmentation processing and single-character splitting respectively to obtain a plurality of sample words and a plurality of sample characters respectively;

[0015] The pre-trained language model is trained based on the sample words to obtain the word embedding language model, and the pre-trained language model is trained based on the sample characters to obtain the character embedding language model.

[0016] Optionally, the word segmentation processing includes:

[0017] The word segmentation processing is performed based on a standard word segmentation algorithm.

[0018] In a second aspect, the present application further provides a term extraction device, including:

[0019] A word segmentation module is configured to perform word segmentation processing on input corpus of a term to be extracted to obtain a plurality of candidate words;

[0020] A vector acquisition module is configured to, for any candidate word, acquire a word vector corresponding to the target candidate word based on a word embedding language model, and acquire a character vector corresponding to each character in the target candidate word based on a character embedding language model, and further acquire a sum vector of the character vectors.

[0021] A determination module is configured to determine whether the target candidate word is extracted as a term according to a similarity comparison result between the word vector and the sum vector.

[0022] The word embedding language model and the character embedding language model are both obtained by training a pre-trained language model based on sample corpus in a vertical industry to which the input corpus belongs to.

[0023] Optionally, the determination module is configured to:

[0024] if the difference between the word vector and the sum vector is greater than or equal to a preset threshold, the target candidate word is extracted as a term; or

[0025] If the difference between the word vector and the sum vector is less than a preset threshold, it is determined that the target candidate word is not extracted as a term.

[0026] Optionally, the apparatus further comprises a training module configured to:

[0027] The sample corpus is subjected to word segmentation processing and single-character splitting respectively, to obtain a plurality of sample words and a plurality of sample characters respectively.

[0028] The pre-trained language model is trained based on the sample words to obtain the word embedding language model, and the pre-trained language model is trained based on the sample characters to obtain the character embedding language model.

[0029] Optionally, the word segmentation processing comprises:

[0030] The word segmentation processing is performed based on a standard word segmentation algorithm.

[0031] In a third aspect, the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the term extraction method according to the first aspect when executing the program.

[0032] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the term extraction method according to the first aspect.

[0033] In a fifth aspect, the present application further provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the term extraction method according to the first aspect.

[0034] The term extraction method, apparatus, electronic device, and storage medium provided by the present application can accurately extract terms in a corpus without using a supervised model, thereby reducing the cost of term extraction and achieving accurate extraction of terms in a corpus of a vertical industry. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0036] Figure 1is a flowchart of the term extraction method provided by the present application;

[0037] Figure 2 is a training schematic diagram of the word embedding language model and the character embedding language model provided by the present application;

[0038] Figure 3 is an implementation schematic diagram of the term extraction method provided by the present application;

[0039] Figure 4 is a structural schematic diagram of the term extraction device provided by the present application;

[0040] Figure 5 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0041] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0042] Figure 1 is a flowchart of the term extraction method provided by the present application, as shown in Figure 1 the method comprises the following steps:

[0043] Step 100, performing word segmentation processing on the input corpus of the term to be extracted, to obtain a plurality of candidate words;

[0044] Specifically, in order to accurately extract the term in the input corpus (such as an article, a paragraph, a sentence, etc.), after obtaining the input corpus of the term to be extracted, the input corpus can be first processed by a word segmentation algorithm to divide the sentences in the input corpus into a plurality of candidate words.

[0045] Among them, the candidate words should be words with more than one character, that is, only the words with more than one character divided in the input corpus are obtained as candidate words, and single characters and punctuation marks, etc. are discarded. These candidate words are candidate words to be extracted as terms, and subsequent identification will be performed on these candidate words to determine whether there are terms to be extracted among them.

[0046] Optionally, the input corpus of the term to be extracted can be a monolingual corpus, for example, it can be a Chinese corpus of a certain vertical industry, of course, it can also be other languages, which is not specifically limited here.

[0047] The segmentation algorithm used in the above segmentation processing can be any algorithm capable of segmenting a corpus, and is not specifically limited here. Alternatively, the segmentation algorithm can be a standard segmentation algorithm, such as a dictionary-based segmentation algorithm, etc.

[0048] In step 101, for any candidate word, a word vector corresponding to the target candidate word is obtained based on a word embedding language model, and a character vector corresponding to each character in the target candidate word is obtained based on a character embedding language model, and a sum vector of the character vectors is further obtained. The word embedding language model and the character embedding language model are both obtained by training a pre-trained language model based on sample corpora in the vertical industry to which the input corpus belongs.

[0049] Specifically, after the input corpus is segmented to obtain candidate words, each candidate word is sequentially identified.

[0050] For a certain candidate word, taking "apple" as an example, a word vector (for example, denoted as a) corresponding to the candidate word can be obtained based on a pre-trained word embedding language model, i.e., the candidate word can be input into the word embedding language model to obtain a vector representation of the candidate word.

[0051] At the same time, the candidate word can be split into single characters, and here "apple" can be split into two characters "ping" and "guo". Then, character vectors of "ping" and "guo" are obtained based on a pre-trained character embedding language model, i.e., "ping" and "guo" are respectively input into the character embedding language model to obtain a vector representation of each character, such as a character vector of "ping" denoted as γ1 and a character vector of "guo" denoted as γ2. After obtaining the character vectors of "ping" and "guo", the two character vectors can be added to further obtain a sum vector of the character vectors in "apple", for example, denoted as β, β = γ1 + γ2.

[0052] In order to achieve accurate vector representations of words and characters in the input corpus, the above word embedding language model and character embedding language model are both obtained by training a pre-trained language model based on sample corpora in the vertical industry to which the input corpus belongs. That is, the training samples used to train the above word embedding language model and character embedding language model can be set according to actual needs. If it is necessary to extract terms in a corpus, sample corpora in the vertical industry to which the corpus belongs can be used to train the model to obtain more accurate word embedding language model and character embedding language model.

[0053] The pre-trained language model can be various pre-trained language representation models as long as vector representation of words or characters can be achieved, and is not specifically limited herein. For example, the pre-trained language model can be a Bidirectional Encoder Representation from Transformers (BERT) model.

[0054] In step 102, whether the target candidate word is extracted as a term is determined according to a similarity comparison result between the word vector and the sum vector.

[0055] Specifically, in general, a word that cannot be interpreted literally or that has a greater difference from the literal meaning is more likely to be a term. For example, the literal meaning of "apple" and "fruit" is quite different, and the literal meaning of "fruit" is closer to fruit rather than a mobile phone. However, in the training data, an apple is sometimes a fruit and sometimes a mobile phone, so according to the above idea, an apple is likely to be a term.

[0056] Based on the above idea, for a certain candidate word, taking "apple" as an example, after obtaining vectors α and β, a similarity measurement method between vectors can be used to compare the similarity between α and β. According to the comparison result, for example, if the result shows that the difference between α and β is large (the difference degree exceeds a certain value), it can be determined that the candidate word is extracted as a term.

[0057] As can be seen, the term extraction method provided in the embodiment of the present application compares the similarity between the word vector of a candidate word and the sum vector of the character vectors of each character in the candidate word, performs term extraction, does not need to use a supervised model, and can accurately extract terms in the corpus, thereby reducing the cost of term extraction and accurately extracting terms in the corpus of a vertical industry.

[0058] Optionally, whether the target candidate word is extracted as a term is determined according to a similarity comparison result between the word vector and the sum vector, including:

[0059] If the difference degree between the word vector and the sum vector is greater than or equal to a preset threshold, the target candidate word is extracted as a term; or

[0060] If the difference degree between the word vector and the sum vector is less than the preset threshold, it is determined that the target candidate word is not extracted as a term.

[0061] Specifically, based on the term identification idea described above, for a certain candidate word, still taking "apple" as an example, after obtaining vectors a and β, the difference degree between vectors a and β can be compared, for example, a measurement method such as spatial cosine, Euclidean distance, Manhattan distance, etc. can be used to represent the similarity of vectors. Correspondingly, the difference degree can be a measurement value representing the difference between the two vectors in various measurement methods. The larger the measurement value is, the greater the difference between the two vectors is. Thus, according to the comparison result between the difference degree between a and β and the preset threshold, it can be determined whether the candidate word is extracted as a term. It can be understood that the preset threshold can be set accordingly according to the method used to calculate the difference degree. For example, if the measurement method of spatial cosine is used, the preset threshold should also be set accordingly according to the numerical range representing the difference degree in this method.

[0062] Optionally, based on the sample corpus in the vertical industry to which the input corpus belongs, the pre-trained language model is trained to obtain the word embedding language model and the character embedding language model, comprising:

[0063] The sample corpus is subjected to word segmentation processing and single-character splitting respectively, and a plurality of sample words and a plurality of sample characters are obtained respectively;

[0064] The pre-trained language model is trained based on the sample words to obtain the word embedding language model, and the pre-trained language model is trained based on the sample characters to obtain the character embedding language model.

[0065] Specifically, Figure 2 The training schematic diagram of the word embedding language model and the character embedding language model provided by the present application is shown in Figure 2 For example, taking a Chinese corpus of a certain vertical industry as an example, when the model is trained, the sample corpus used for training can be copied in two copies, and different word segmentation methods are used for word segmentation. For example, one copy is a standard word segmentation method (such as dictionary-based word segmentation), and the other copy is split according to Chinese characters. For example, "I love to eat apples" generates the following word segmentation results respectively:

[0066] (1) I love to eat apples.

[0067] (2) I love to eat apples.

[0068] Then, the sample words and sample characters obtained by the two different word segmentation methods are input into the language model respectively for training (for example, BERT, which refers to training a special model using the algorithm of BERT, rather than using the publicly trained BERT model), to obtain the word embedding language model and the character embedding language model.

[0069] Figure 3 The implementation schematic diagram of the term extraction method provided by the present application is shown in Figure 3As shown, the embodiment introduces a method for extracting terms in a Chinese sentence.

[0070] Firstly, the standard segmentation can be performed on the sentence to be extracted, and the Chinese character greater than 1 is reserved as a candidate word for identifying whether it is a term; then, for each candidate word, the corresponding word vector is obtained through the word embedding language model, the word vector of each word in the candidate word is obtained through the word embedding language model, and then the sum vector of the word is obtained; finally, for each candidate word, the similarity comparison of the corresponding word vector and the sum vector of the candidate word is performed, and according to the comparison result, whether each candidate word is a term is determined, so as to finally realize the extraction of each term in the sentence to be extracted.

[0071] The term extraction device provided by the application is described below, and the term extraction device described below can be correspondingly referred to the term extraction method described above.

[0072] Figure 4 The structure diagram of the term extraction device provided by the application is shown as Figure 4 As shown, the device comprises:

[0073] The segmentation module 400 is used for performing segmentation processing on the input corpus of the term to be extracted, and obtaining a plurality of candidate words;

[0074] The vector acquisition module 410 is used for acquiring the word vector corresponding to the target candidate word based on the word embedding language model for any candidate word; acquiring the word vector corresponding to each word in the target candidate word based on the word embedding language model, and further obtaining the sum vector of the word vector;

[0075] The determination module 420 is used for determining whether the target candidate word is extracted as a term according to the similarity comparison result between the word vector and the sum vector.

[0076] The word embedding language model and the word embedding language model are both obtained by training the pre-trained language model based on the sample corpus in the vertical industry to which the input corpus belongs.

[0077] Optionally, the determination module 420 is used for:

[0078] If the difference degree between the word vector and the sum vector is greater than or equal to the preset threshold, the target candidate word is extracted as a term; or,

[0079] If the difference degree between the word vector and the sum vector is less than the preset threshold, it is determined that the target candidate word is not extracted as a term.

[0080] Optionally, the device further comprises a training module 430, and the training module 430 is used for:

[0081] The sample corpus is subjected to word segmentation processing and single-character splitting respectively, and a plurality of sample words and a plurality of sample characters are obtained.

[0082] The pre-trained language model is trained based on the sample words to obtain a word embedding language model, and the pre-trained language model is trained based on the sample characters to obtain a character embedding language model.

[0083] Optionally, the word segmentation processing comprises:

[0084] The word segmentation processing is performed based on a standard word segmentation algorithm.

[0085] It should be noted that the above device provided by the present application can realize all method steps achieved by the above method embodiments and achieve the same technical effects, and the same parts and beneficial effects in the present embodiment as the method embodiments will not be described in detail.

[0086] Figure 5 A structural schematic diagram of an electronic device provided by the present application is shown in FIG. 1. Figure 5 As shown in FIG. 1, the electronic device can include a processor 510, a communications interface 520, a memory 530 and a communications bus 540, wherein the processor 510, the communications interface 520 and the memory 530 complete mutual communication through the communications bus 540. The processor 510 can call logical instructions in the memory 530 to execute steps of any of the term extraction methods provided by the above embodiments, for example, performing word segmentation processing on input corpus of a to-be-extracted term to obtain a plurality of candidate words; for any candidate word, obtaining a word vector corresponding to the target candidate word based on a word embedding language model; obtaining a character vector corresponding to each character in the target candidate word based on a character embedding language model, and further obtaining a sum vector of the character vectors; determining whether to extract the target candidate word as a term according to a similarity comparison result between the word vector and the sum vector; wherein the word embedding language model and the character embedding language model are both obtained by training a pre-trained language model based on sample corpus in a vertical industry to which the input corpus belongs.

[0087] Further, the logic instructions in the memory 530 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0088] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the steps of any of the term extraction methods provided by the above embodiments, for example: performing word segmentation processing on input corpus of terms to be extracted to obtain a plurality of candidate words; for any candidate word, obtaining a word vector corresponding to the target candidate word based on a word embedding language model; obtaining a character vector corresponding to each character in the target candidate word based on a character embedding language model, and further obtaining a sum vector of the character vectors; determining whether to extract the target candidate word as a term according to a similarity comparison result between the word vector and the sum vector; wherein the word embedding language model and the character embedding language model are both obtained by training a pre-trained language model based on sample corpus in a vertical industry to which the input corpus belongs.

[0089] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any of the term extraction methods provided by the above embodiments, for example: performing word segmentation processing on input corpus of terms to be extracted to obtain a plurality of candidate words; for any candidate word, obtaining a word vector corresponding to the target candidate word based on a word embedding language model; obtaining a character vector corresponding to each character in the target candidate word based on a character embedding language model, and further obtaining a sum vector of the character vectors; determining whether to extract the target candidate word as a term according to a similarity comparison result between the word vector and the sum vector; wherein the word embedding language model and the character embedding language model are both obtained by training a pre-trained language model based on sample corpus in a vertical industry to which the input corpus belongs.

[0090] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0091] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0092] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A term extraction method characterized by, The method comprises the following steps: performing word segmentation on an input corpus of a term to be extracted to obtain a plurality of candidate words; for any candidate word, obtaining a word vector corresponding to a target candidate word based on a word embedding language model; obtaining a character vector corresponding to each character in the target candidate word based on a character embedding language model, and further obtaining a sum vector of the character vectors; determining whether to extract the target candidate word as a term according to a similarity comparison result between the word vector and the sum vector; wherein the word embedding language model and the character embedding language model are obtained by training a pre-trained language model based on sample corpora in a vertical industry to which the input corpus belongs; the determining whether to extract the target candidate word as a term according to the similarity comparison result between the word vector and the sum vector comprises: if a difference degree between the word vector and the sum vector is greater than or equal to a preset threshold, extracting the target candidate word as a term; or if the difference degree between the word vector and the sum vector is less than the preset threshold, determining not to extract the target candidate word as a term.

2. The term extraction method of claim 1, wherein, training the pre-trained language model based on the sample corpora in the vertical industry to obtain the word embedding language model and the character embedding language model comprises: performing word segmentation and single-character splitting on the sample corpora respectively to obtain a plurality of sample words and a plurality of sample characters respectively; training the pre-trained language model based on the sample words to obtain the word embedding language model, and training the pre-trained language model based on the sample characters to obtain the character embedding language model.

3. The term extraction method according to claim 1 or 2, characterized in that, the performing word segmentation comprises: performing word segmentation based on a standard word segmentation algorithm.

4. A term extraction apparatus characterized by comprising: The method comprises the following steps: a word segmentation module for performing word segmentation on an input corpus of a term to be extracted to obtain a plurality of candidate words; a vector obtaining module for obtaining a word vector corresponding to a target candidate word based on a word embedding language model for any candidate word, and obtaining a character vector corresponding to each character in the target candidate word based on a character embedding language model, and further obtaining a sum vector of the character vectors; a determining module for determining whether to extract the target candidate word as a term according to a similarity comparison result between the word vector and the sum vector; wherein the word embedding language model and the character embedding language model are obtained by training a pre-trained language model based on sample corpora in a vertical industry to which the input corpus belongs; the determining module is configured to: if a difference degree between the word vector and the sum vector is greater than or equal to a preset threshold, extract the target candidate word as a term; or if the difference degree between the word vector and the sum vector is less than the preset threshold, determine not to extract the target candidate word as a term.

5. The term extraction apparatus of claim 4, wherein, The device further comprises a training module configured to: perform word segmentation and single-character splitting on the sample corpora respectively to obtain a plurality of sample words and a plurality of sample characters respectively; The pre-trained language model is trained based on the sample words to obtain the word embedding language model, and the pre-trained language model is trained based on the sample characters to obtain the character embedding language model.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the term extraction method according to any one of claims 1 to 3 when executing the program.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the term extraction method according to any one of claims 1 to 3.

8. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the term extraction method according to any one of claims 1 to 3. The computer program, when executed by the processor, implements the steps of the term extraction method according to any one of claims 1 to 3.

Images