Sample data generation method, training method, corpus generation method and device

Abstract

Embodiments of the present disclosure disclose a sample data generation method, a corpus model training method, a corpus generation method, an apparatus, an electronic device and a computer readable medium. A specific implementation of the method comprises: performing word segmentation on sample corpus in a Chinese sample corpus library obtained, and obtaining a candidate sample word library according to a word segmentation result; for each candidate sample word in the candidate sample word library, labeling each character in the candidate sample word to generate a sample labeled word, and obtaining a sample labeled word library, wherein the labeled label comprises a position label representing the position of the character in the word and an attribute label representing the attribute of the word; and generating a sample labeled word group library according to the sample labeled word library. The implementation can make full use of the label information in the sample data to realize data enhancement of small sample data.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, in particular to a sample data generation method, a training method, and a corpus generation method and device. BACKGROUND

[0002] Named entity recognition generally refers to recognizing entities with specific meanings in a text, mainly including names of persons, places, organizations, and proper nouns. Named entity recognition is an important basic tool in application fields such as information extraction, question and answer systems, syntax analysis, and machine translation.

[0003] It is well known that a large amount of training data is generally necessary to ensure the effect of deep learning. The model learns the distribution on a large amount of data, and finally achieves generalization and promotion on the test data set. However, the number and types of existing named entity recognition data sets are often very scarce. This not only affects the training effect of the model, but also takes a lot of time and effort to manually prepare a large amount of labeled data.

[0004] In addition, current Chinese data enhancement mainly comes from back translation, adversarial generation network, and noise adding methods including synonym replacement, random insertion, random exchange, and random deletion. However, these methods are more suitable for translation and text classification tasks that rely heavily on context semantics, and are not suitable for named entity recognition (word-level annotation, more sensitive to noise), which affects the recognition effect (such as poor accuracy). Moreover, these methods often require the provision of an additional knowledge base, and have requirements for the coverage of the knowledge base, which is not suitable for the expansion of small sample Chinese data. SUMMARY

[0005] The summary of the present disclosure is used to introduce the concepts in a brief form, which will be described in detail in the specific embodiments part. The summary of the present disclosure is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions. Some embodiments of the present disclosure provide a sample data generation method, a training method of a corpus model, a corpus generation method, a device, an electronic equipment and a computer readable medium, to solve one or more of the technical problems mentioned in the background section.

[0006] In a first aspect, some embodiments of the present disclosure provide a sample data generation method, wherein the method comprises: performing word segmentation on sample corpora in an acquired Chinese sample corpus, and obtaining a candidate sample word library according to a word segmentation result; for each candidate sample word in the candidate sample word library, labeling each character in the candidate sample word with a label to generate a sample labeled word, and obtaining a sample labeled word library, wherein the labeled label includes a position label representing the position of the character in the word and an attribute label representing the attribute of the word; and generating a sample labeled word group library according to the sample labeled word library.

[0007] In some embodiments, before the label is annotated to each character in the candidate sample word, the method further comprises: determining whether the preset character is contained in the candidate sample word; in response to the determination that the preset character is contained, modifying the preset character into the target character.

[0008] In some embodiments, the sample annotated word group library is generated according to the sample annotated word library, comprising: combining each sample annotated word in the sample annotated word library to obtain at least one word group; for each word group in the at least one word group, adding a head identifier and a tail identifier at the start position and the end position of the word group respectively to generate a sample annotated word group, to obtain the sample annotated word group library.

[0009] In some embodiments, before the head identifier and the tail identifier are added at the start position and the end position of the word group respectively to generate the sample annotated word group, the method further comprises: determining whether the target word group is contained in the at least one word group, wherein the position label of each character in the target word group is the preset position label; in response to the determination that the target word group is contained, removing the target word group in the at least one word group.

[0010] In some embodiments, the candidate sample word library is obtained according to the word segmentation result, comprising: screening out a sample word representing a preset word meaning from the word segmentation result as a candidate sample word to obtain the candidate sample word library.

[0011] In the second aspect, some embodiments of the present disclosure provide a sample data generation apparatus, wherein the apparatus comprises: a word segmentation unit configured to perform word segmentation on a sample corpus in a Chinese sample corpus obtained by the apparatus, and obtain a candidate sample word library according to a word segmentation result; a label unit configured to, for each candidate sample word in the candidate sample word library, annotate a label to each character in the candidate sample word to generate a sample annotated word, and obtain a sample annotated word library, wherein the annotated label comprises a position label for representing the position of the character in the word and an attribute label for representing the attribute of the word; and a sample generation unit configured to generate a sample annotated word group library according to the sample annotated word library.

[0012] In the third aspect, some embodiments of the present disclosure provide a corpus model training method, wherein the method comprises: obtaining a predicted annotated word group based on a sample annotated word group and an initial model, wherein the sample annotated word group is obtained by using the sample data generation method described in any of the implementations of the first aspect; analyzing the predicted annotated word group and the sample annotated word group to obtain a target function value; determining whether the initial model is trained according to a comparison result of the target function value and a preset threshold; and in response to the determination that the initial model is not trained, adjusting a related parameter in the initial model.

[0013] In some embodiments, the initial model comprises a long short-term memory network-based model, and the target function value comprises a relative entropy value and a perplexity value; and based on the sample annotated word group and the initial model, the predicted annotated word group is obtained by: generating a sample annotated word vector according to the sample annotated word group; inputting the sample annotated word vector into the initial model to obtain the predicted annotated word group.

[0014] In some embodiments, the method further comprises: in response to determining that the training is completed, taking the initial model as the corpus model.

[0015] In the fourth aspect, some embodiments of the present disclosure provide a device for training a corpus model, wherein the device comprises: an input unit configured to obtain a predicted annotated word group based on a sample annotated word group and an initial model, wherein the sample annotated word group is obtained by using the sample data generation method in any one of claims 1-5; an analysis unit configured to analyze the predicted annotated word group and the sample annotated word group to obtain a target function value; a determination unit configured to determine whether the initial model is trained based on a comparison result of the target function value and a preset threshold; and an adjustment unit configured to adjust a related parameter in the initial model in response to determining that the training is not completed.

[0016] In the fifth aspect, some embodiments of the present disclosure provide a corpus generation method, wherein the method comprises: setting configuration parameters of a corpus model, wherein the corpus model is obtained by using the training method described in any one of the implementations of the third aspect, and the configuration parameters comprise at least one of the following: a maximum word group length, a random number seed, and a batch size; inputting a first identifier and a sample annotated word into the corpus model to generate a new annotated word group, and obtaining a new annotated word group library, wherein the first identifier represents a starting position of a word group, and the sample annotated word is obtained by using the sample data generation method described in any one of the implementations of the first aspect.

[0017] In the sixth aspect, some embodiments of the present disclosure provide a corpus generation device, wherein the device comprises: a setting unit configured to set configuration parameters of a corpus model, wherein the corpus model is obtained by using the training method described in any one of the implementations of the third aspect, and the configuration parameters comprise at least one of the following: a maximum word group length, a random number seed, and a batch size; and a corpus generation unit configured to input a first identifier and a sample annotated word into the corpus model to generate a new annotated word group, and obtain a new annotated word group library, wherein the first identifier represents a starting position of a word group, and the sample annotated word is obtained by using the sample data generation method described in any one of the implementations of the first aspect.

[0018] In a seventh aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect, the third aspect, or the fifth aspect.

[0019] In an eighth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any implementation manner of the first aspect, the third aspect, or the fifth aspect.

[0020] The above various embodiments of the present disclosure have the following beneficial effects: The sample data generation method of some embodiments of the present disclosure can effectively perform data augmentation on small sample corpus data, and is especially suitable for the augmentation of Chinese named entity recognition data. Specifically, the reason why the augmentation of Chinese named entity recognition data does not have a good effect is that most data augmentation is suitable for languages such as English with word granularity. English has spaces as word boundaries. Chinese needs to be segmented into words. Different segmentation will cause obvious ambiguity. In addition, existing Chinese data augmentation is more suitable for tasks that rely on context semantics. However, named entity recognition often needs to annotate words. Based on this, the sample data generation method of some embodiments of the present disclosure can make full use of the label information of each character in the sample data to generate fine-grained synthetic data. That is, a sample annotated word group library is generated according to a sample annotated word library to realize data expansion at the granularity of Chinese characters. At the same time, the demand for additional knowledge base is low. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals can denote the same or similar elements. It should be understood that the drawings are schematic, and elements and elements are not necessarily drawn to scale.

[0022] Figure 1 is an architecture diagram of an exemplary system to which some embodiments of the present disclosure can be applied;

[0023] Figure 2 is a flowchart of some embodiments of the sample data generation method according to the present disclosure;

[0024] Figure 3 is a structural schematic diagram of some embodiments of the sample data generation device according to the present disclosure;

[0025] Figure 4 is a flowchart of some embodiments of the training method of the corpus model according to the present disclosure;

[0026] Figure 5 is a structural schematic diagram of some embodiments of a corpus generation apparatus according to the present disclosure;

[0027] Figure 6 is a flow chart of some embodiments of a corpus generation method according to the present disclosure;

[0028] Figure 7 is a schematic diagram of an application scenario of a corpus generation method according to some embodiments of the present disclosure;

[0029] Figure 8 is a structural schematic diagram of some embodiments of a corpus generation apparatus according to the present disclosure;

[0030] Figure 9 is a structural schematic diagram of an electronic device suitable for use to implement some embodiments of the present disclosure. DETAILED DESCRIPTION

[0031] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly and completely understood. It should be understood that the drawings of the present disclosure are only for illustrative purposes and should not be construed as limiting the scope of protection of the present disclosure.

[0032] It should also be noted that, for the sake of brevity, only the parts of the drawings that are relevant to the present disclosure are shown. The embodiments in the present disclosure and the features in the embodiments can be combined with each other in the case of no conflict.

[0033] It should be noted that the terms “first”, “second”, and the like in the present disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules, or units.

[0034] It should be noted that the terms “one”, “multiple” in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that, unless otherwise explicitly stated in the context, it should be understood as “one or more”.

[0035] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0036] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0037] Figure 1An exemplary system architecture 100 of a sample data generation method or apparatus, a corpus model training method or apparatus, and a corpus generation method or apparatus to which some embodiments of the present disclosure can be applied is shown.

[0038] As shown in Figure 1 The system architecture 100 can include a terminal device 101, a network 102, a database server 103, and servers 104, 105. The network 102 can be a medium to provide a communication link between the terminal device 101, the database server 103, and the servers 104, 105. The network 102 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0039] A user can use the terminal device 101 to interact with the database server 103 and the servers 104, 105 through the network 102 to receive or send messages, etc. Various client applications can be installed on the terminal device 101, such as data collection applications, model training applications, corpus generation applications, shopping applications, web browsers, and instant messaging tools, etc.

[0040] The terminal device 101 herein can be hardware or software. When the terminal device 101 is hardware, it can be various electronic devices with a display screen, including but not limited to smartphones, tablet computers, e-book readers, laptop computers, and desktop computers, etc. When the terminal device 101 is software, it can be installed in the above-mentioned electronic devices. It can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made herein.

[0041] The database server 103 can be a server for storing sample corpus, initial model, and other data. The servers 104, 105 can be servers providing various services, such as background servers providing support for applications installed on the terminal device 101. When receiving a generation instruction sent by a user, the background server can process sample corpus obtained from the database server 103, and can send the processing result (such as sample annotated phrases) to the terminal device 101 (for display) and the database server 103 (for storage).

[0042] The database server 103 and the servers 104, 105 herein can also be hardware or software. When the database server 103 and the servers 104, 105 are hardware, they can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the database server 103 and the servers 104, 105 are software, they can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made herein.

[0043] It should be noted that the method provided by the embodiments of the present disclosure can be executed by the server 104, 105, or by the terminal device 101. Accordingly, the apparatus can be arranged in the server 104, 105, or in the terminal device 101. No specific limitation is made here.

[0044] It should be noted that in the case where the server 104 has the functions of the database server 103 and the server 105, the database server 103 and the server 105 can also not be arranged in the system architecture 100.

[0045] It should be understood that Figure 1 The number of terminal devices, networks, servers, and database servers in

[0046] With reference to Figure 2 , a flow 200 of some embodiments of the sample data generation method according to the present disclosure is shown. The method includes the following steps:

[0047] Step 201, performing word segmentation on the sample corpus in the acquired Chinese sample corpus, and obtaining a candidate sample vocabulary according to the word segmentation result.

[0048] In some embodiments, the execution subject (for example, the server 104 or 105 shown in Figure 1 ) of the sample data generation method can acquire the Chinese sample corpus in various ways. For example, the execution subject can acquire it from a database server (for example, the database server 103 shown in Figure 1 ) or the cloud through wired or wireless connection. For another example, the execution subject can directly acquire the Chinese sample corpus pre-stored on the local.

[0049] Here, the execution subject can first utilize various Chinese word segmentation tools (such as Jieba word segmentation, language cloud, etc.) to perform word segmentation processing on the sample corpus in the Chinese sample corpus. Then, according to the word segmentation result, the execution subject can obtain a candidate sample vocabulary.

[0050] It should be noted that the above-mentioned Chinese sample corpus can be any Chinese corpus. As an example, the above-mentioned corpus can be a Chinese named entity (such as a fresh commodity name) corpus. At this time, the execution subject can take the sample words obtained by word segmentation as candidate sample words, thereby obtaining a candidate sample vocabulary.

[0051] Optionally, when the above corpus is another (less suitable or not required) Chinese corpus, the execution entity can also screen out sample words representing preset word meanings from the word segmentation results as candidate sample words to obtain a candidate sample word library. The preset word meanings can be set according to actual needs, such as representing named entities (such as fresh produce). In this way, even when the corpus is lacking or insufficient, the required sample words can be selected from other Chinese sample corpora, thereby expanding the data volume of the candidate sample word library.

[0052] Step 202: For each candidate sample word in the candidate sample word library, label each character in the candidate sample word to generate a sample labeled word, and obtain a sample labeled word library.

[0053] In some embodiments, based on the candidate sample word library obtained in step 201, for each candidate sample word therein, the execution entity can label each character in the candidate sample word to generate a sample labeled word, and obtain a sample labeled word library. The labels can include position labels representing the position of the character in the word and attribute labels representing the attributes of the word.

[0054] As an example, the execution entity can label the position labels in the form of BIOES. B stands for Begin, indicating the character at the beginning of the word; I stands for Intermediate, indicating the character in the middle of the word; E stands for End, indicating the character at the end of the word; S stands for Single, indicating that the word is a single character; O stands for Other, used to mark irrelevant characters.

[0055] It can be understood that compared with traditional Chinese corpus data, the data structure of named entity data (especially item / product name data) is relatively simple. Therefore, in order to simplify the labeling process and improve processing efficiency, the execution entity can also label the position labels in the form of BIOs. Here, the above attribute labels can be labeled in the form of numbers and / or letters. For example, 0 represents the product (commodity) ontology; 1 represents modification; 2 represents the brand; LOC represents the location. As an example, after labeling the above labels for "treasured Chinese cabbage", the labeled words "B-2treasureI-2collectionB-0bigI-0whiteI-0cabbage" or "B2treasureI2collectionB0bigI0whiteI0cabbage" can be obtained. The execution entity can integrate "B-2treasure" or "B2treasure" as one character.

[0056] Optionally, the execution subject can also use the various Chinese word segmentation tools described above to label each character in the candidate sample word with a label to generate a sample labeled word. In order to achieve the uniformity of the label and facilitate the improvement of processing efficiency, the label can also be set in these Chinese word segmentation tools before labeling. Alternatively, according to the correspondence between the labels (required labels and tool labels), the labels in the labeled word obtained by the Chinese word segmentation tool are modified to the required labels.

[0057] In some application scenarios, before labeling each character in the candidate sample word with a label, the execution subject can determine whether the candidate sample word contains a preset character. The preset character here can be a specific character or a character of a preset type (such as a number or a letter), which is not limited here. If it is determined that it contains, the execution subject can modify the preset character to a target character. For example, a number can be changed to a capital Chinese character, or replaced with a uniform character (such as N). For example, some or all letters can be replaced with a symbol, etc. This can effectively avoid the conflict between the characters in the word and the labeled label, help to improve the reading recognition efficiency of the execution subject, and also help to improve the processing efficiency.

[0058] Further, the execution subject can also determine whether a target sample labeled word exists in the sample labeled word library. Wherein, the position label of each character in the target sample labeled word is a preset position label. The preset position label here can be the label O described above for marking irrelevant characters. This can further improve the quality of the data in the sample labeled word library, i.e. the proportion of the truly required words.

[0059] Step 203, generating a sample labeled word group library according to the sample labeled word library.

[0060] In some embodiments, based on the sample labeled word library obtained in step 202, the execution subject can generate a sample labeled word group library. As an example, the execution subject can combine each sample labeled word in the sample labeled word library to obtain at least one word group. The combination method here is not limited, such as random combination. For example, each sample labeled word can be combined according to an entity name library. The entity name library here can be online or in the cloud, or can be created by collecting. Then, the execution subject can store at least one word group in a list, or store it by setting spaces between the word groups to distinguish, thereby generating a sample labeled word group library. In this way, without the need for additional knowledge base, data augmentation can also be achieved on the basis of small sample Chinese corpus, improving the data volume. That is, a low-cost fine-grained data enhancement method is achieved.

[0061] Optionally, for each of the at least one word group, the execution subject can add a head identifier and a tail identifier at the start position and the end position of the word group respectively to generate a sample annotated word group, obtaining a sample annotated word group library. The head identifier and the tail identifier here can be set according to actual needs. For example, a general identifier <s>As a header,< / s> as the tail identifier. In this way, it is not only convenient for most users to understand, but also reduces the complexity of annotation, which is conducive to improving processing efficiency.

[0062] Further, in order to better improve the data quality of the sample annotated word group library without affecting the data volume, before adding the head and tail identifiers, the execution subject can further determine whether the target word group is contained in the at least one word group. Wherein, the position label of each character in the target word group is a preset position label. If it is determined that the target word group is contained, the target word group in the at least one word group can be removed. Then, the head and tail identifiers are added to generate the sample annotated word group library. The sample data obtained in this way can be directly applied to the e-commerce platform to improve the recognition accuracy of the platform.

[0063] Some embodiments of the present disclosure provide a sample data generation method, which realizes a low-cost fine-grained sample data enhancement. Specifically, by performing word segmentation and screening on the sample corpus in the Chinese sample corpus library, the required candidate sample word library can be obtained. Then, by labeling the label of each character in the candidate sample word, the sample annotated word can be generated. After that, according to the obtained sample annotated word library, the sample annotated word group library can be generated. The effective expansion of small sample corpus data is realized, which is especially suitable for the expansion of Chinese named entity recognition data.

[0064] Please refer to Figure 3 as an implementation of the method shown in Figure 2 , the present disclosure provides some embodiments of a sample data generation device, which device embodiments correspond to those method embodiments, and the device can be specifically applied in various electronic devices. Figure 2

[0065] As shown in Figure 3 , the sample data generation device 300 of some embodiments can include: a word segmentation unit 301 configured to perform word segmentation on the sample corpus in the acquired Chinese sample corpus library, and obtain a candidate sample word library according to the word segmentation result; a labeling unit 302 configured to label a label for each character in each candidate sample word in the candidate sample word library to generate a sample annotated word, and obtain a sample annotated word library, wherein the labeled label includes a position label for representing the position of the character in the word and an attribute label for representing the attribute of the word; and a sample generation unit 303 configured to generate a sample annotated word group library according to the sample annotated word library.

[0066] ​In some embodiments, the labeling unit 302 is further configured to determine whether the preset character is contained in the candidate sample word; and in response to determining that the preset character is contained, modify the preset character into the target character.

[0067] In some embodiments, the sample generation unit 303 is further configured to combine each sample labeled word in the sample labeled word library to obtain at least one word group; and for each word group in the at least one word group, add a head identifier and a tail identifier at a starting position and an ending position of the word group respectively to generate a sample labeled word group, thereby obtaining a sample labeled word group library.

[0068] In some embodiments, the sample generation unit 303 is further configured to determine whether the target word group is contained in the at least one word group, wherein each character in the target word group has the preset position label; and in response to determining that the target word group is contained, remove the target word group in the at least one word group.

[0069] In some embodiments, the word segmentation unit 301 is further configured to filter out the sample word representing the preset word meaning from the word segmentation result as the candidate sample word, thereby obtaining a candidate sample word library.

[0070] It can be understood that the units described in the apparatus 300 correspond to the respective steps in the method described with reference to Figure 2 . Therefore, the operations, features and beneficial effects described above with respect to the method also apply to the apparatus 300 and the units contained therein, which will not be described here again.

[0071] With continued reference to Figure 4 , a flow 400 of some embodiments of a training method of a corpus model according to the present disclosure is shown. The method comprises the following steps:

[0072] Step 401, obtaining a predicted labeled word group based on the sample labeled word group and an initial model.

[0073] In some embodiments, the execution subject (for example, the server 104 or 105 shown in Figure 1 ) of the training method can obtain the sample labeled word group through wired connection or wireless connection. The sample labeled word group here can be the sample labeled word group in the sample labeled word group library obtained in the above Figure 2 embodiments, which will not be described here again. The sample labeled word group library can be stored locally on the execution subject, or on a database server or other electronic device.

[0074] Here, the execution subject can input the sample labeled word group into the initial model. The initial model can output a predicted labeled word group corresponding to the input sample labeled word group. The initial model here can be a model capable of predicting the character at the current position according to the input character, and predicting the subsequent characters in time sequence.

[0075] As an example, the initial model can be composed of two parts. The sample annotated word group is input into the first part. The first part outputs the corresponding sample annotated word vector, and the output sample annotated word vector is transmitted to the second part, i.e., the input of the second part. The second part can output the predicted annotated word group corresponding to the sample annotated word group. The first part here can adopt a Bert (Bidirectional Encoder Representations from Transformers) model (a self-encoding language model). In this way, the accuracy of generating the sample annotated word vector can be improved. Alternatively, the first part can only adopt the Transformer Encoder end (used to convert input text into vector representation) in the Bert model. In this way, not only the accuracy of the word vector can be improved, but also the complexity of the structure can be reduced, and the generation efficiency can be improved.

[0076] It can be understood that, in order to improve the training efficiency of the initial model, the first part can be pre-trained. And the second part can adopt at least one of the following: a long short-term memory (LSTM) based model, a time recurrent neural network model, and a hidden Markov model (HMM). Among them, the LSTM model is more suitable for processing and predicting important events with very long intervals and delays in time series, and the processing effect is better.

[0077] Optionally, in order to simplify the structure of the initial model and improve the training efficiency of the model, the initial model can not contain the above-mentioned first part. In this case, the execution subject can first generate a sample annotated word vector according to the sample annotated word group. For example, the nn.embedding() function (such as random initialization with a dimension of 300) provided by PyTorch (an open source Python machine learning library) or the above-mentioned Bert model can be used to generate the word vector. Then, the sample annotated word vector can be input into the initial model.

[0078] It should be noted that the internal mechanism of LSTM usually adjusts the information flow through three gates, namely the input gate, the output gate and the forget gate, to understand which data in the sequence needs to be retained or discarded, as follows:

[0079] C t =z f *C t-1 +z i *z;

[0080] Among them, C t is the current cell state; C t-1 is the previous cell state; z f is the forget gate; z iand z is the operation in the input gate, representing the forget stage of the model, selectively forgetting the input from the previous node.

[0081] h t t );

[0082] where h t represents the current hidden state; z0 represents the previous operation in the output gate, representing the selective memory stage of the model, selectively memorizing the input C t . Those that are important are recorded, and those that are not important are recorded less.

[0083] y t = σ (W' · h t ) ;

[0084] where y t represents the output stage of the model, obtained by some changes of the current hidden state h t . Therefore, the subsequent words can be predicted according to the input words in time sequence. σ (·) represents the sigmoid function (sigmoid growth curve); W' is the internal weight of the model.

[0085] Before training, some key parameters in the model can be set, such as 'epochs (training rounds)': 30, 'emb_dim (input word embedding data dimension)': 300, 'rnn_size (node number)': 512, 'dropout (dropping node proportion)': 0.5, 'batch_size (batch size)': 32.

[0086] Optionally, in order to further improve the generation effect of the model, the initial model can also be provided with a linear output layer at the end to output the characters predicted at the current position.

[0087] Step 402, analyze the predicted annotated word group and the sample annotated word group to obtain a target function value.

[0088] In some embodiments, the execution subject can analyze the predicted annotated word group and the corresponding sample annotated word group to obtain a target function value. And the target function value can be compared with a preset threshold. The target function here can be adjusted according to the type of the initial model. Correspondingly, the preset threshold can be different according to the different target functions.

[0089] ​As an example, if the initial model is a Hidden Markov Model, in order to achieve the expected training learning effect, the objective function can adopt the Baum-Welch algorithm (a kind of Expectation-Maximum algorithm (EM)) and the Reversed Viterbi algorithm.

[0090] As another example, if the initial model is an LSTM model, the objective function can be relative entropy (KL divergence) and perplexity (PPL). Specifically:

[0091]

[0092] wherein D KL (p||q) is the relative entropy value; p(x i ) is the true distribution, that is, the sample annotated phrase; q(x i ) is the predicted distribution, that is, the predicted annotated phrase. The closer the relative entropy value is to 1, the better the model fitting effect.

[0093]

[0094] wherein PPL(W) is the perplexity value; P(w1w2…w N ) represents the probability of a sentence (equivalent to the phrase in the present application), and the better the sentence (the greater the probability), the smaller the perplexity, that is, the model is less perplexed by the sentence; N is the number of words contained in the sentence.

[0095] Step 403, determining whether the initial model is trained according to the comparison result of the objective function value and the preset threshold value.

[0096] In some embodiments, the execution subject can determine whether the initial model is trained according to the comparison result of the two. For example, if the objective function value is greater than (or does not reach) the preset threshold value, it means that the initial model is not trained. At this time, step 405 can be continued. On the contrary, if the objective function value is less than or equal to (or reaches) the preset threshold value, it means that the initial model is trained.

[0097] Optionally, if the initial model is trained, the execution subject can store the trained initial model as a corpus model.

[0098] Further, in order to avoid the situation of infinite loop training of the model, the number of training rounds of the model can be set as described above. The execution subject can count the number of training rounds of the initial model, and the number of training rounds is increased by 1 each time. At the same time, the execution subject can determine whether the number of training rounds reaches the preset value (such as 30). In this case, if the objective function value is greater than (or does not reach) the preset threshold value, and the number of training rounds does not reach the preset value, it means that the training is not completed.

[0099] At step 404, in response to determining that the initial model is not trained, the related parameters in the initial model are adjusted.

[0100] In some embodiments, if it is determined that the initial model is not trained, the subject performing can adjust the related parameters in the initial model. Then the initial model after adjustment is continuously trained.

[0101] Some embodiments of the present disclosure provide a training method, which can train the initial model by using the sample annotated word groups obtained by the above-mentioned embodiments, so as to obtain a corpus model. Since the above-mentioned sample annotated word group library is obtained by the sample data generation method, the data is often the required data (i.e., the data quality is better), and the data is also effectively expanded (i.e., the data quantity can also meet certain requirements). Therefore, the corpus model trained by these data is more targeted and can achieve the expected training effect. That is, the corpus model can be better applied to Chinese named entity recognition.

[0102] Further referring to Figure 5 , as an implementation of the method shown in Figure 4 , the present disclosure provides some embodiments of a training device of a corpus model, which device embodiments correspond to the method embodiments shown in Figure 4 , and the device can be specifically applied to various electronic devices.

[0103] As shown in Figure 5 , the training device 500 of some embodiments can include: an input unit 501 configured to obtain a predicted annotated word group based on a sample annotated word group and an initial model, wherein the sample annotated word group is obtained by the above-mentioned sample data generation method; an analysis unit 502 configured to analyze the predicted annotated word group and the sample annotated word group to obtain a target function value; a determination unit 503 configured to determine whether the initial model is trained according to a comparison result of the target function value and a preset threshold; and an adjustment unit 504 configured to adjust the related parameters in the initial model in response to determining that the initial model is not trained.

[0104] In some embodiments, the initial model includes a model based on a long short-term memory network, and the target function value includes a relative entropy value and a perplexity value; and the input unit 501 is further configured to generate a sample annotated word vector according to the sample annotated word group; and input the sample annotated word vector into the initial model to obtain the predicted annotated word group.

[0105] In some embodiments, the device 500 further includes a model generation unit (not shown in Figure 5 ) configured to take the initial model as a corpus model in response to determining that the training is completed.

[0106] It can be understood that the units described in the apparatus 500 correspond to the steps in the method described above. Therefore, the operations, features and advantages described above for the method also apply to the apparatus 500 and the units contained therein, and are not repeated here. Figure 4 The steps in the described method correspond to each other. Therefore, the operations, features and advantages described above for the method also apply to the apparatus 500 and the units contained therein, and are not repeated here.

[0107] Referring to Figure 6 , a flow 600 of some embodiments of the corpus generation method according to the present disclosure is shown. The method includes the following steps:

[0108] Step 601, setting configuration parameters of the corpus model.

[0109] In some embodiments, the execution subject (for example, the server 104 or 105 shown in Figure 1 ) of the corpus generation method can initially set the configuration parameters of the corpus model. Among them, the configuration parameters can include at least one of the following: maximum sentence length (max_sent_length), random seed (random seed), batch size (batch_size). For example, the maximum sentence length can be set to 15, the random seed number to 3435, and the batch size to 100. Through the initial parameter setting, the data generated by the corpus model subsequently can be more in line with the requirements, and the data quality can be improved. The corpus model here can be obtained by the training method described in the Figure 4 embodiments.

[0110] Step 602, input the first identifier and sample annotated words into the corpus model to generate new annotated word groups, and obtain a new annotated word group library.

[0111] In some embodiments, the execution subject can input the first identifier and sample annotated words into the corpus model. Then, through the corpus model, new annotated word groups can be generated. Among them, the first identifier represents the starting position of the word group. The sample annotated words here can be the sample annotated words in the sample annotated word library described in the Figure 2 embodiments. That is, only the first identifier of the word group is input, and the model can determine the character at the beginning of the word group from the sample annotated word library, thereby automatically generating new annotated word groups.

[0112] As shown in Figure 7 , a schematic diagram of one application scenario of the corpus generation method of the present disclosure is shown. In this application scenario, the user can use the terminal device to send a generation instruction to the server. Among them, the generation instruction can include the first identifier <s>and a storage location of the sample annotation word library. The server stores a corpus model. After receiving the generation instruction, the server can obtain the sample annotation word library according to the storage location, such as from a database server. The server can identify the first mark <s>and the sample annotation words in the sample annotation word library are input into the corpus model, so that a new annotation word group library can be generated. Figure 7 As can be seen from the above, the new annotation word group library can include the following annotation word groups: <s>B-1 present I-1 West I-0 guava< / s> <s>B-2 bag I-2 load B-2 hemp I-2 hot B-0 large I-0 shrimp< / s> <s>B-1 right I-1 subject B-0 black I-0 pig< / s> In this way, the original data is effectively expanded.

[0113] Further, the generation effect of the corpus model can also be verified. Here, the subject can mix the sample annotation word library with the new annotation word group library generated by the corpus model. Then, the named entity recognition can be performed by using the Bert model, so as to evaluate the expansion effect. The evaluation results are shown in the following table:

[0114] Not expanded Expanded Label 0 acc: 0.8337 - recall: 0.8379 - f1 : 0.8358 acc: 0.8320 - recall: 0.8277 - f1 : 0.8298 Label 1 acc: 0.6512 - recall: 0.6421 - f1 : 0.6466 acc: 0.6620 - recall: 0.6667 - f1 : 0.6643 Label 2 acc: 0.5794 - recall: 0.4856 - f1 : 0.5284 acc: 0.5837 - recall: 0.4892 - f1 : 0.5323

[0115] As can be seen from the above table, due to the large number of label 0 products, there is almost no effect after expansion. However, for label 1 modification and label 2 brand, there is an improvement after expansion.

[0116] The corpus generation method provided by some embodiments of the present disclosure can effectively expand the sample annotation word library (original data) by using the above corpus model. A new low-cost fine-grained Chinese data set expansion method is implemented, which can conveniently and quickly generate new corpus. Compared with the unexpanded case, the effect of named entity recognition is improved. The corpus generation method of the present disclosure remedies the shortcomings of traditional data enhancement methods, such as introducing a large amount of noise and losing fine-grained semantic characteristics. Moreover, without the need for an additional knowledge base, the quality of the synthesized corpus is improved. In addition, this method can also effectively expand different types of data sets. For example, it can be applied to various scenarios based on entities such as goods, thereby improving the accuracy of search and recommendation.

[0117] With reference to Figure 8 , as an implementation of the method shown in Figure 6 , the present disclosure provides some embodiments of a corpus generation device, which device embodiments correspond to the method embodiments shown in Figure 6 , and the device can be specifically applied to various electronic devices.

[0118] As Figure 8 As shown, the corpus generation apparatus 800 of some embodiments can include a setting unit 801 configured to set configuration parameters of a corpus model, wherein the corpus model is obtained by using the training method described above, and the configuration parameters include at least one of the following: maximum phrase length, random number seed, batch size; and a corpus generation unit 802 configured to input a first identifier and a sample labeled word into the corpus model to generate a new labeled word phrase, and obtain a new labeled word phrase library, wherein the first identifier represents the starting position of a word phrase, and the sample labeled word is obtained by using the sample data generation method described above.

[0119] It can be understood that the units described in the apparatus 800 correspond to the respective steps in the method described above. Therefore, the operations, features and advantages described above for the method also apply to the apparatus 800 and the units included therein, and are not repeated here. Figure 6 It can be understood that the units described in the apparatus 800 correspond to the respective steps in the method described above. Therefore, the operations, features and advantages described above for the method also apply to the apparatus 800 and the units included therein, and are not repeated here.

[0120] Reference is made below to Figure 9 , which shows a structural schematic diagram of an electronic device (e.g., a server in Figure 1 ) suitable for implementing some embodiments of the present disclosure. Figure 9 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0121] As shown in Figure 9 , the electronic device 900 can include a processing apparatus (e.g., a central processor, a graphics processor, etc.) 901, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 902 or loaded into a random access memory (RAM) 903 from a storage apparatus 908. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing apparatus 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0122] Generally, the following apparatuses can be connected to the I / O interface 905: input apparatuses 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output apparatuses 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage apparatuses 908 including, for example, a magnetic tape, a hard disk, etc.; and communication apparatuses 909. The communication apparatuses 909 can allow the electronic device 900 to communicate with other devices wirelessly or through wires to exchange data. Although Figure 9 The electronic device 900 with various apparatuses is shown, but it should be understood that it is not required to implement or have all the apparatuses shown. More or less apparatuses can alternatively be implemented or included. Figure 9 Each block in the flow diagrams represents one or more operations that can be implemented as hardware or software in embodiments. The representations are the functions described above and are intended to contribute to an improved understanding of embodiments of the technology.

[0123] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to some embodiments of the present disclosure. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are performed.

[0124] It should be noted that the computer readable medium in some embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In some embodiments of the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to, wire, cable, RF (radio frequency), or the like, or any suitable combination thereof.

[0125] The computer readable medium can be included in the electronic device, or can exist separately from the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the following operations: performing word segmentation on a sample corpus in a Chinese sample corpus obtained, and obtaining a candidate sample word library according to a result of the word segmentation; for each candidate sample word in the candidate sample word library, labeling each character in the candidate sample word with a label to generate a sample labeled word, and obtaining a sample labeled word library, wherein the label includes a position label representing a position of the character in the word and an attribute label representing an attribute of the word; and generating a sample labeled word group library according to the sample labeled word library.

[0126] Alternatively, based on a sample labeled word group and an initial model, a predicted labeled word group is obtained, wherein the sample labeled word group is obtained by using the sample data generation method; the predicted labeled word group and the sample labeled word group are analyzed to obtain a target function value; whether the initial model is trained is determined according to a comparison result of the target function value and a preset threshold; and in response to a determination that the initial model is not trained, a related parameter in the initial model is adjusted.

[0127] Alternatively, configuration parameters of a corpus model are set, wherein the corpus model is obtained by using the training method, and the configuration parameters include at least one of the following: a maximum word group length, a random number seed, and a batch size; a first identifier and a sample labeled word are input into the corpus model to generate a new labeled word group, and a new labeled word group library is obtained, wherein the first identifier represents a starting position of the word group, and the sample labeled word is obtained by using the sample data generation method.

[0128] Furthermore, computer program code to carry out operations of some embodiments of the disclosure can written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may include a word segmentation unit, a labeling unit, and a sample generation unit. Alternatively, a processor may include an input unit, an analysis unit, a determination unit, and an adjustment unit. Yet another processor may include a setting unit and a corpus generation unit. The names of these units do not necessarily limit the unit itself; for example, a word segmentation unit may also be described as "a unit that performs word segmentation on sample corpora in an acquired Chinese sample corpus."

[0131] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0132] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.< / s> < / s>

Claims

1. A method for generating sample data, wherein, The method includes: The sample corpus in the acquired Chinese sample corpus is segmented into words, and a candidate sample vocabulary is obtained based on the segmentation results; For each candidate sample word in the candidate sample vocabulary, each character in the candidate sample word is labeled to generate a sample labeled word, thus obtaining a sample labeled vocabulary. The labeled labels include positional labels representing the position of the character in the word and attribute labels representing the attributes of the word. Based on the sample labeled vocabulary, a sample labeled phrase library is generated, wherein the sample labeled phrases in the sample labeled phrase library are used as training samples to generate a corpus model. The corpus model predicts the current position and subsequent characters in chronological order based on the input first identifier and the sample labeled words in the candidate sample vocabulary to obtain new labeled phrases. The first identifier indicates the starting position of the phrase. The objective function of the corpus model includes relative entropy and perplexity.

2. The method according to claim 1, wherein, Before labeling each character in the candidate sample words, the method further includes: Determine whether the candidate sample words contain preset characters; In response to the determination of inclusion, the preset character is modified to the target character.

3. The method according to claim 1, wherein, The step of generating a sample labeled phrase library based on the sample labeled vocabulary library includes: The sample labeled words in the sample labeled word library are combined to obtain at least one word group; For each of the at least one word groups, a first identifier and a last identifier are added to the beginning and end positions of the word group respectively to generate a sample labeled word group, thus obtaining a sample labeled word group library.

4. The method according to claim 3, wherein, Before adding a start identifier and a end identifier to the start and end positions of the phrase to generate sample labeled phrases, the method further includes: Determine whether the at least one word group contains a target word group, wherein the position labels of each character in the target word group are preset position labels; In response to determining inclusion, the target phrase is removed from the at least one phrase.

5. The method according to any one of claims 1-4, wherein, The process of obtaining the candidate sample vocabulary based on the word segmentation results includes: From the word segmentation results, sample words that represent the preset word meanings are selected as candidate sample words to obtain the candidate sample word library.

6. A sample data generation apparatus, wherein, The device includes: The word segmentation unit is configured to segment the sample corpus in the acquired Chinese sample corpus and obtain a candidate sample word library based on the segmentation results. The annotation unit is configured to label each character in each candidate sample word in the candidate sample word library to generate a sample labeled word, thereby obtaining a sample labeled word library. The labeled labels include positional labels for characterizing the position of a character in a word and attribute labels for characterizing the attributes of a word. The sample generation unit is configured to generate a sample labeled phrase library based on the sample labeled vocabulary library, wherein the sample labeled phrases in the sample labeled phrase library are used as training samples to generate a corpus model, and the corpus model predicts the current position and subsequent characters in chronological order based on the input first identifier and the sample labeled words in the candidate sample vocabulary library to obtain new labeled phrases, wherein the first identifier indicates the starting position of the phrase, and the objective function of the corpus model includes relative entropy and perplexity.

7. A training method for a corpus model, wherein, The method includes: Based on the sample labeled word groups and the initial model, predicted labeled word groups are obtained, wherein the sample labeled word groups are obtained using the sample data generation method as described in any one of claims 1-5; The predicted labeled word groups and the sample labeled word groups are analyzed to obtain the objective function value, wherein the objective function value includes the relative entropy value and the perplexity value; Based on the comparison result between the objective function value and the preset threshold, it is determined whether the initial model has been trained successfully. In response to the determination that training is not complete, the relevant parameters in the initial model are adjusted; In response to the determination that training is complete, the initial model is used as a corpus model, wherein the corpus model predicts the current position and subsequent characters in chronological order based on the input first identifier and the sample labeled words in the candidate sample word library to obtain new labeled word groups.

8. The method according to claim 7, wherein, The initial model includes a model based on long short-term memory networks; as well as The predicted labeled word groups obtained based on the sample labeled word groups and the initial model include: Generate sample labeled word vectors based on sample labeled word groups; The sample labeled word vectors are input into the initial model to obtain the predicted labeled word groups.

9. A training device for a corpus model, wherein, The device includes: The input unit is configured to obtain predicted labeled words based on sample labeled words and an initial model, wherein the sample labeled words are obtained using the sample data generation method as described in any one of claims 1-5; The analysis unit is configured to analyze the predicted labeled word group and the sample labeled word group to obtain an objective function value, wherein the objective function value includes a relative entropy value and a perplexity value; The determining unit is configured to determine whether the initial model has been trained successfully based on a comparison between the objective function value and a preset threshold. The adjustment unit is configured to adjust relevant parameters in the initial model in response to determining that training is not yet complete; and to use the initial model as a corpus model in response to determining that training is complete, wherein the corpus model predicts the current position and subsequent characters in chronological order based on the input first identifier and the sample labeled words in the candidate sample vocabulary to obtain new labeled word groups.

10. A corpus generation method, wherein, The method includes: Configure the corpus model's parameters, wherein the corpus model is obtained using the training method described in claim 7 or 8, and the configuration parameters include at least one of the following: maximum phrase length, random number seed, and batch size; The first identifier and sample labeled words are input into the corpus model to generate new labeled word groups, resulting in a new labeled word group library. The first identifier indicates the starting position of the word group, and the sample labeled words are obtained using the sample data generation method as described in any one of claims 1-5.

11. A corpus generation apparatus, wherein, The device includes: The setting unit is configured to set configuration parameters for a corpus model, wherein the corpus model is obtained using the training method described in claim 7 or 8, and the configuration parameters include at least one of the following: maximum phrase length, random number seed, and batch size; The corpus generation unit is configured to input the initial identifier and sample labeled words into the corpus model to generate new labeled word groups and obtain a new labeled word group library, wherein the initial identifier indicates the starting position of the word group, and the sample labeled words are obtained by the sample data generation method as described in any one of claims 1-5.

12. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5, 7-8, and 10.

13. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-5, 7-8, and 10.

Images