Labeling method and device, electronic equipment and storage medium

Abstract

The application relates to the field of artificial intelligence, and provides a labeling method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a text to be labeled; based on a labeling model, applying semantic features of each character in the text to be labeled, and part-of-speech features and / or keyword features of a part of speech to which each character belongs, labeling each character to obtain a labeling result of the text to be labeled; the keyword features represent whether the corresponding part of speech is a preset keyword, and the labeling model is obtained based on sample texts and labeling labels of the sample texts. The labeling method provided in the embodiment of the application can realize that the labeling model can capture more feature information related to the labeling label, thereby improving the accuracy of the labeling result of the text to be labeled. Furthermore, the labeling model is used to label the text to be labeled, manual labeling is cancelled, labeling cost is reduced, and labeling efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a labeling method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, in the aspect of data use, supervised learning is still superior to unsupervised learning in terms of effect, and supervised learning needs to label the data correspondingly to complete the training of the model.

[0003] However, the existing data labels still mainly rely on manual labeling. When facing a large amount of data required for training the model, a large amount of manpower is needed to label the data, resulting in high labor cost and low efficiency. Moreover, due to the instability of manual operation, such as human negligence and attention reduction due to long-time work, the accuracy of the labeled data labels will be affected. SUMMARY

[0004] The embodiments of the present application provide a labeling method and device, electronic equipment and storage medium to solve the technical problems of high cost, long time and low accuracy of manual labeling.

[0005] In a first aspect, the embodiments of the present application provide a labeling method, comprising:

[0006] determining a text to be labeled;

[0007] applying semantic features of each character in the text to be labeled and part-of-speech features and / or keyword features of each part-of-speech to which the character belongs based on a labeling model to label each character, and obtaining a labeling result of the text to be labeled;

[0008] The keyword feature represents whether the corresponding part-of-speech is a preset keyword, and the labeling model is trained based on a sample text and a labeling label of the sample text.

[0009] In one embodiment, according to the labeling method provided by the present application, the application of the semantic features of each character in the text to be labeled and the part-of-speech features and / or keyword features of each part-of-speech to which the character belongs based on the labeling model to label each character, and obtaining the labeling result of the text to be labeled, comprises:

[0010] extracting the semantic features of each character in the text to be labeled based on an embedding layer in the labeling model;

[0011] performing feature extraction on the semantic features of each character based on an independent feature layer in the labeling model to obtain independent features of each character;

[0012] based on the feature fusion layer in the labeling model, fusing semantic features of the characters, part-of-speech features and / or keyword features of the segmented words to which the characters belong, and performing feature extraction on the fused features to obtain joint features of the characters;

[0013] based on the labeling layer in the labeling model, applying the independent features and the joint features of the characters to label the characters to obtain a labeling result of the text to be labeled.

[0014] In one embodiment, according to the labeling method provided in the present application, based on the feature fusion layer in the labeling model, fusing semantic features of the characters, part-of-speech features and / or keyword features of the segmented words to which the characters belong, and performing feature extraction on the fused features to obtain joint features of the characters, comprises:

[0015] based on the feature enhancement layer in the feature fusion layer, applying part-of-speech feature weights and keyword weights to weight the part-of-speech features and the keyword features of the segmented words to which the characters belong to obtain enhanced features of the characters; the part-of-speech feature weights and the keyword weights are obtained by training;

[0016] based on the enhanced feature extraction layer in the feature fusion layer, fusing the semantic features of the characters and the enhanced features of the characters, and performing feature extraction on the fused features to obtain joint features of the characters.

[0017] In one embodiment, according to the labeling method provided in the present application, based on the labeling layer in the labeling model, applying the independent features and the joint features of the characters to label the characters to obtain a labeling result of the text to be labeled, comprises:

[0018] based on the joint feature labeling layer in the labeling layer, applying the joint features of the characters to label the characters to obtain a joint labeling result of the text to be labeled;

[0019] based on the independent feature labeling layer in the labeling layer, applying the joint labeling result of the text to be labeled and the independent features of the characters to label the characters to obtain an independent labeling result of the text to be labeled, and determining the independent labeling result as the labeling result.

[0020] In one embodiment, according to the labeling method provided in the present application, the labeling model is obtained by training based on the following steps:

[0021] determining an initial model, the initial model comprising an embedding layer, an independent feature layer, a feature fusion layer and a labeling layer;

[0022] based on the initial model, determining a joint labeling result and an independent labeling result of the sample text;

[0023] determine a training loss value based on the joint annotation result and the independent annotation result of the sample text and the annotation label of the sample text;

[0024] update the parameters of the initial model based on the training loss value until the training loss value converges, to obtain the annotation model.

[0025] In one embodiment, according to the annotation method provided in the present application, the annotation label is determined based on the following steps:

[0026] determine the sample text and the preset keyword;

[0027] perform semantic role annotation on the sample text to obtain a coarse division text containing preset role content;

[0028] perform word segmentation on the coarse division text, and perform part-of-speech annotation on each word segment of the coarse division text to obtain each word segment of the coarse division text and the part-of-speech annotation of each word segment;

[0029] determine the annotation label based on the preset interference part-of-speech, the preset keyword, and the part-of-speech annotation of each word segment of the coarse division text.

[0030] In one embodiment, according to the annotation method provided in the present application, the annotation label is determined based on the part-of-speech annotation of each word segment of the coarse division text, the preset interference part-of-speech, and the preset keyword.

[0031] filter each word segment of the coarse division text based on the length of each word segment of the coarse division text, the preset interference part-of-speech, and the preset keyword, to obtain a fine division text of the sample text;

[0032] perform annotation on the fine division text based on the labels in the annotation label library to obtain the annotation label.

[0033] In a second aspect, the embodiments of the present application provide an annotation device, comprising:

[0034] a determination module configured to determine a text to be annotated;

[0035] an annotation module configured to perform annotation on each character in the text to be annotated based on the semantic features of the character, and the part-of-speech features and / or keyword features of the word segment to which the character belongs, to obtain an annotation result of the text to be annotated, based on an annotation model;

[0036] The keyword features represent whether the corresponding word segment is a preset keyword, and the annotation model is obtained based on sample texts and annotation labels of the sample texts.

[0037] In a third aspect, an electronic device is provided, which includes a processor and a memory storing a computer program. The processor implements the labeling method of the first aspect when executing the program.

[0038] In a fourth aspect, a non-transitory computer-readable storage medium is provided, which stores a computer program. The computer program, when executed by a processor, implements the labeling method of the first aspect.

[0039] The labeling method, device, electronic device, and storage medium provided by the embodiments of the present application increase the part-of-speech features and / or keyword features of the word division to which each character in the text to be labeled belongs in the labeling process, so that the labeling model can capture more feature information related to the labeling label, thereby improving the accuracy of the labeling result of the text to be labeled. Furthermore, the labeling model is used to label the text to be labeled, which eliminates manual labeling, reduces labeling cost, and improves labeling efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0041] Figure 1 is one of the flowcharts of the labeling method provided by the present application;

[0042] Figure 2 is another flowchart of the labeling method provided by the present application;

[0043] Figure 3 is a flowchart of the joint feature acquisition method of each character provided by the present application;

[0044] Figure 4 is a flowchart of the labeling result acquisition method provided by the present application;

[0045] Figure 5 is a flowchart of the labeling model training method provided by the present application;

[0046] Figure 6 is a flowchart of the labeling label acquisition method provided by the present application;

[0047] Figure 7 is a flowchart of the labeling label verification method provided by the present application;

[0048] Figure 8 is a structural diagram of the labeling device provided by the present application;

[0049] Figure 9 FIG. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] For the purposes of the present application, the technical solutions and advantages will be more apparent from the following description of the embodiments of the present application, which will be made with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0051] In view of the problems of high cost, long time and low accuracy caused by manual annotation of a large amount of data. How to automatically annotate a large amount of data to reduce cost, annotation time and improve annotation accuracy is a problem to be solved in the field.

[0052] In view of the above technical problems, an embodiment of the present application provides an annotation method. Figure 1 FIG. 1 is a flow schematic diagram of an annotation method provided by the present application. As shown in FIG. 1, the method is for automatic annotation of text data, and the method can be applied to a scenario in which an institution name is used as annotation information. The following embodiments are described with reference to the scenario in which an institution name is used as annotation information. In addition, the method can also be applied to a scenario in which the same type of information is used as annotation information in similar text data, for example, a scenario in which a real estate name or a commodity name is used as annotation information. The method comprises the following steps. Figure 1 Step 110, determining a text to be annotated;

[0053] Specifically, the text to be annotated is text data with one or more annotation information, for example, the text to be annotated contains one or more institution names. The text to be annotated can be input by a user, can be a text copied from a network, or can be obtained based on voice data recognition, and the present application is not limited in this regard.

[0054] Step 120, based on an annotation model, applying semantic features of each character in the text to be annotated, and part-of-speech features and / or keyword features of each part-of-speech to which the character belongs, to annotate each character to obtain an annotation result of the text to be annotated;

[0055] The keyword feature represents whether the corresponding part-of-speech is a preset keyword, and the annotation model is trained based on a sample text and annotation labels of the sample text.

[0056]

[0057] ​In view of the fact that if only the semantic features of the text to be annotated are used for annotation, the same semantics but not the content to be annotated can be easily obtained, for example, pronouns used to represent the names of institutions, and thus the annotated content needs to be filtered twice, the embodiments of the present application add the part-of-speech features and / or keyword features of the word to which each character belongs to the semantic features of each character in the text to be annotated to assist in annotating the text to be annotated. In a specific scenario, the part-of-speech features and the keyword features in the scenario can enable the annotation model to capture more feature information related to the annotation label, that is, the annotation model can more accurately locate and annotate the annotation target in the text to be annotated.

[0058] Specifically, the annotation model encodes the text to be annotated at the character granularity to obtain the semantic features of each character in the text to be annotated, performs word segmentation on the text to be annotated, determines the part-of-speech features corresponding to each word, and determines the keyword features of each word according to whether each word is a preset keyword. Then, according to the obtained semantic features of each character in the text to be annotated and the part-of-speech features and / or keyword features of the word to which each character belongs, each character is annotated to obtain the annotation result of the text to be recognized.

[0059] It should be noted that the preset keyword is a word containing common features in the scenario to which the text to be annotated belongs, for example, the suffix words such as “certain city” or “company”, “office” and “shop” in the scenario of using the name of an institution as annotation information. The annotation model can annotate the text to be annotated according to the semantic features of each character in the text to be annotated and the part-of-speech features of the word to which each character in the text to be annotated belongs, and perform secondary annotation on the text to be annotated by combining the semantic features of each character in the text to be annotated and the keyword features of the word to which each character in the text to be annotated belongs to obtain the annotation result of the text to be annotated. The annotation model can also annotate the text to be annotated according to the semantic features of each character in the text to be annotated, and fuse the part-of-speech features and / or keyword features of the word to which each character belongs to obtain the enhanced features of each word, to obtain the annotation result of the text to be annotated, and the embodiments of the present application do not limit this. The part-of-speech features are features with a feature dimension number equal to the number of preset part-of-speech categories, for example, if the preset part-of-speech categories are name, verb, adjective, adverb, preposition and conjunction, the part-of-speech features are features with a feature dimension of 6. The keyword features of each word can be determined according to whether each word is a preset keyword, that is, if a word is determined to be a preset keyword, the word is annotated as a keyword, otherwise, the word is annotated as a non-keyword, and the feature annotated by the word is the keyword feature of the word.

[0060] The labeling method provided in the embodiments of the present application increases the part-of-speech features and / or keyword features of the word to which each character in the text to be labeled belongs in the labeling process through the labeling model, so that the labeling model can capture more feature information related to the labeling label, thereby improving the accuracy of the labeling result of the text to be labeled. Furthermore, the labeling model is used to label the text to be labeled, manual labeling is cancelled, labeling cost is reduced, and labeling efficiency is improved.

[0061] Based on the above embodiments, Figure 2 is a second flowchart of the labeling method provided in the present application. As shown in Figure 2 , step 120 includes:

[0062] Step 121, based on the embedding layer in the labeling model, extracting semantic features of each character in the text to be labeled;

[0063] Step 122, based on the independent feature layer in the labeling model, performing feature extraction on the semantic features of each character to obtain independent features of each character;

[0064] Step 123, based on the feature fusion layer in the labeling model, fusing the semantic features of each character, the part-of-speech features and / or keyword features of the word to which each character belongs, and extracting the fused features to obtain joint features of each character;

[0065] Step 124, based on the labeling layer in the labeling model, applying the independent features and joint features of each character to label each character to obtain the labeling result of the text to be labeled.

[0066] Considering that if the labeling model only uses a single feature extraction network to extract a single feature, the labeling model has poor robustness when performing sequence prediction because the single feature has poor representativeness. Therefore, the embodiments of the present application use two different feature extraction networks to respectively extract the semantic features of each character and the features obtained by fusing the semantic features of each character and the part-of-speech features and / or keyword features of the word to which each character belongs, and then label each character based on the two extracted features.

[0067] Specifically, in step 121, the embedding layer in the labeling model performs feature extraction on the to-be-labeled text based on the word granularity size to obtain semantic features of each word in the to-be-labeled text. In step 122, the independent feature layer in the labeling model directly performs feature extraction on the semantic features of each word obtained in step 121 to obtain independent features of each word. In step 123, the semantic features of each word are first fused with the part-of-speech features and / or keyword features of the word to which each word belongs, and feature extraction is performed on the fused features to obtain joint features of each word. Finally, in step 124, the labeling layer in the labeling model labels each word in the to-be-labeled text based on the independent features of each word obtained in step 122 and the joint features of each word obtained in step 123 to obtain the labeling result of the to-be-labeled text.

[0068] It should be noted that the independent features of each word are feature representations obtained by the labeling model performing feature extraction on the semantic features of each word, and the joint features of each word are feature representations obtained by the labeling model performing feature extraction on the features obtained by fusing the semantic features of each word, the part-of-speech features and / or keyword features of the word to which each word belongs. The labeling layer in the labeling model can be a single labeling network for labeling each word in the to-be-labeled text based on the independent features and joint features of each word, or two labeling networks, one of which is used to label each word in the to-be-labeled text based on the joint labeling features of each word to obtain intermediate labeling results of each word, and the other of which is used to perform secondary labeling on each word in the to-be-labeled text based on the intermediate labeling results of each word and the independent features of each word to obtain the labeling result of the to-be-labeled text. The embodiments of the present application do not limit this. The structure of the feature extraction network used by the independent feature layer in the labeling model can be the same as or different from the structure of the feature extraction network used by the feature fusion layer in the labeling model, and the embodiments of the present application do not limit this.

[0069] The labeling method provided in the embodiments of the present application realizes that the labeling layer in the labeling model can predict labeling labels based on more diverse feature forms by respectively performing feature extraction on the features before fusion and the features after fusion by the independent feature layer and the feature fusion layer in the labeling model, thereby improving the accuracy of labeling.

[0070] Based on the above embodiments, Figure 3 is a flowchart of a method for obtaining joint features of each word provided by the present application. As shown in Figure 3 step 123 includes:

[0071] In step 123-1, based on the feature enhancement layer in the feature fusion layer, the part-of-speech features and keyword features of the word to which each word belongs are weighted by applying a part-of-speech feature weight and a keyword weight to obtain enhanced features of each word; the part-of-speech feature weight and the keyword weight are obtained by training.

[0072] Step 123-2, based on the enhanced feature extraction layer in the feature fusion layer, fusing the semantic features of each word and the enhanced features of each word, and extracting the fused features to obtain the joint features of each word.

[0073] Considering that the part-of-speech features and keyword features of each word are fused with the semantic features of each word extracted by the embedding layer in the labeling model, the labeling model can effectively capture more implicit entity information contained in the text to be labeled, the labeling model can obtain better interpretability, and more accurate labeling results can be obtained.

[0074] Specifically, in step 123-1, the feature enhancement layer in the feature fusion layer fuses the part-of-speech features and keyword features of the word according to the part-of-speech feature weight and keyword weight obtained when the labeling model is trained, to obtain the enhanced features of each word. In step 123-2, the enhanced feature extraction layer in the feature fusion layer fuses the enhanced features of each word obtained in step 123-1 and the semantic features of each word, and extracts the fused features to obtain the joint features of each word.

[0075] The labeling method provided by the embodiment of the present application fuses the enhanced features of each word and the semantic features of each word through the feature fusion layer in the labeling model, and extracts the fused features, so that the labeling model can effectively capture more implicit entity information contained in the text to be labeled, the labeling model can obtain better interpretability, and more accurate labeling results can be obtained.

[0076] Based on the above embodiments, Figure 4 is a flowchart of the labeling result obtaining method provided by the present application. As shown in Figure 4 , step 124 includes:

[0077] Step 124-1, based on the joint feature labeling layer in the labeling layer, applying the joint features of each word to label each word to obtain a joint labeling result of the text to be labeled;

[0078] Step 124-2, based on the independent feature labeling layer in the labeling layer, applying the joint labeling result of the text to be labeled and the independent features of each word to label each word to obtain an independent labeling result of the text to be labeled, and determining the independent labeling result as the labeling result.

[0079] Considering that the annotation layer uses only one annotation network to annotate each character in the text to be annotated according to the independent features and the joint features of each character, that is, the label annotation is predicted through a single feature expression, and that the label annotation cannot be predicted from multiple feature expressions, the robustness of the annotation model is low, and the accuracy of the annotation result is also low. Therefore, the embodiment of the present application annotates each character in the text to be annotated by constructing two annotation networks in the annotation layer and annotating each character in the text to be annotated based on different feature expressions.

[0080] Specifically, in step 124-1, the joint feature annotation layer in the annotation layer annotates each character in the text to be annotated according to the input joint features of each character, and obtains the joint annotation result of the text to be annotated. In step 124-2, the independent feature annotation layer in the annotation layer annotates each character in the text to be annotated again according to the joint annotation result obtained in step 124-1 and in combination with the independent features of each character, obtains the independent annotation result of the text to be annotated, and takes the independent annotation result as the final output annotation result.

[0081] The annotation method provided by the embodiment of the present application annotates each character in the text to be annotated twice by the joint feature annotation layer and the independent feature annotation layer in the annotation model, realizes that the features in the annotation model can be transmitted across layers, the annotation layer in the annotation model can predict the annotation label of the text to be annotated according to multiple feature expressions, improves the robustness of the annotation model, and improves the accuracy of the annotation result.

[0082] Based on the above embodiment, the present application further provides a preferred embodiment, in which the annotation model is applied to annotate the organization name from the text to be annotated, and the specific hierarchical result of the annotation model includes:

[0083] (1) The embedding layer of the annotation model uses the pre-trained model BERT (Bidirectional Encoder Representations from Transformers) to extract the semantic features (Word Embedding) of the text to be annotated.

[0084] (2) The feature enhancement layer in the feature fusion layer of the annotation model uses the part-of-speech feature T1 and the keyword feature T2 of each character in the text to be annotated to enhance the semantic features of each character.

[0085] Specifically, the embodiment of the present application annotates and classifies the parts of speech of each word in the text through the LTP (Language Technology Platform) language technology platform to obtain the part-of-speech features of the text to be annotated, mainly annotates and classifies six types of part-of-speech categories, namely, name, verb, adjective, adverb, preposition and conjunction, and sets the feature dimension of the part-of-speech features to 6. The content is segmented through a Chinese segmentation tool, and in the text to be annotated, the key words in the annotation agency name scene are annotated to obtain the key word features of each word in the text to be annotated.

[0086] And the enhanced features of each word in the text to be annotated are calculated by the following formula:

[0087] T f =W1T1+W2T2

[0088] In the formula, W1 and W2 are weight parameters, which are obtained by training the annotation model, T1 is the part-of-speech feature, T2 is the key word feature, and T f is the enhanced feature.

[0089] (3) The enhanced feature extraction layer in the feature fusion layer of the annotation model uses a BiGRU (Bidirectional Gated Recurrent Unity) network, inputs the semantic features and enhanced features of each word in the text to be annotated into the BiGRU network, and obtains the joint features of each word output by the BiGRU network.

[0090] (4) The independent feature layer of the annotation model uses an IDCNN (Iterated Dilated Convolutional Neural Networks) network, inputs the semantic features of each word in the text to be annotated into the IDCNN network, and obtains the independent features of each word output by the IDCNN network.

[0091] (5) The annotation layer of the annotation model uses two CRF (Conditional Random Field) layers, one of which uses the joint features of each word to annotate the text to be annotated to obtain a joint annotation result, and the other uses the joint annotation result and the independent features of each word to annotate the text to be annotated again to obtain an independent annotation result of the text to be annotated, and the independent annotation result is taken as the final annotation result of the text to be annotated.

[0092] Specifically, the joint features and independent features of each word are calculated by the following formulas, respectively:

[0093]

[0094]

[0095] wherein, is an output of the BiGRU network at time t, is an output of the IDCNN network at time t, GRU is a label score of the output feature of the BiGRU network at time t, id is a label score of the output feature of the IDCNN network at time t, G is a weight parameter for calculating P GRU is a bias parameter for calculating P G is a weight parameter for calculating P GRU is a bias parameter for calculating P I is a weight parameter for calculating P id is a bias parameter for calculating P I is a weight parameter for calculating P id is a bias parameter for calculating P G , b G , W I and b I are obtained through training of the labeling model. P GRU and P id at each time constitute joint features and independent features of each word.

[0096] Based on the above embodiment, Figure 5 is a flowchart of a labeling model training method provided by the present application. As shown in Figure 5 , the labeling model is trained based on the following steps:

[0097] Step 510, determining an initial model, the initial model including an embedding layer, an independent feature layer, a feature fusion layer and a labeling layer;

[0098] Specifically, the embedding layer in the initial model is used to extract semantic features of each word in a sample text, the independent feature layer in the initial model is used to extract independent features of each word according to the semantic features of each word in the sample text, the feature fusion layer in the initial model is used to first fuse the semantic features of each word, the part-of-speech features of the word in which each word is located and the keyword features, and then extract joint features of each word according to the fused features; the labeling layer is used to respectively perform label prediction on the sample text according to the independent features and the joint features of each word in the sample text, and calculate a loss value according to the independent labeling result, the joint labeling result and the labeled label of the sample text, and update the parameters of the initial model according to the loss value.

[0099] It should be noted that the feature fusion layer in the initial model can include a feature enhancement layer and an enhanced feature extraction layer, and the labeling layer in the initial model can include a joint feature labeling layer and an independent feature labeling layer, which are not limited by the embodiments of the present application.

[0100] Step 520, determining the joint labeling result and the independent labeling result of the sample text based on the initial model.

[0101] In step 530, a training loss value is determined based on the joint labeling result and the independent labeling result of the sample text and the labeling label of the sample text.

[0102] In step 540, the parameters of the initial model are updated based on the training loss value until the training loss value converges, and the labeling model is obtained.

[0103] Considering that the labeling layer in the initial model jointly labels the sample text based on the feature distribution of the joint features of the sample text and the feature distribution of the independent features of the sample text, the labeling layer can fully understand and master the feature distribution of the label. Therefore, the embodiments of the present application calculate the loss value by using the joint labeling result and the independent labeling result, and update the parameters of the initial model by using the loss value.

[0104] Specifically, after obtaining the joint labeling result and the independent labeling result of the sample text, the labeling layer of the initial model calculates the loss value in combination with the labeling label of the sample text, obtains the training loss value, and then updates the parameters of the initial model according to the training loss value until the training loss value converges, the training of the initial model is completed, and the labeling model is obtained by fixing the parameters of the initial model.

[0105] The labeling method provided by the embodiments of the present application can realize that the initial model can fully understand and master the feature distribution of the label of the sample text according to the various feature forms of the sample text in the training process, so that the labeling model obtained by the training can obtain more accurate labeling results of the to-be-labeled text.

[0106] Based on the above embodiments, the present application provides a preferred embodiment, and the loss function used by the initial model to calculate the loss value is as follows:

[0107]

[0108] In the formula, loss GRU loss ID loss is a weight value, which is generally set to 0.01.

[0109] Based on the above embodiments, Figure 6is a flowchart of the method for obtaining a labeling tag provided by the present application. As shown in Figure 6 The labeling tag is determined based on the following steps.

[0110] Step 610, determining a sample text and a preset keyword.

[0111] Specifically, the sample text is text data with a labeling tag; the preset keyword is a word containing common features in a scenario to which the sample text belongs

[0112] Step 620, performing semantic role labeling on the sample text to obtain a coarse text containing preset role content.

[0113] Step 630, performing word segmentation on the coarse text, and performing part-of-speech tagging on each word segment of the coarse text to obtain each word segment of the coarse text and part-of-speech tagging of each word segment.

[0114] Step 640, determining the labeling tag based on the preset part-of-speech of the interference word, the preset keyword, and each word segment of the coarse text and part-of-speech tagging of each word segment.

[0115] Considering that the labeling tag of the sample text is extracted in an automatic manner, the problems of high cost, low efficiency, and low accuracy caused by manual labeling can be avoided. Therefore, the labeling tag of the sample text is determined by the semantic role labeling, part-of-speech tagging, and preset keyword of the sample text.

[0116] Specifically, the semantic role labeling is performed on the sample text to determine the coarse text containing the preset role content, wherein the preset role represents role information that needs to be focused on, for example, an agent or a patient. In step 630, the coarse text obtained in step 620 is segmented and each word segment is tagged with part-of-speech. In step 640, the word segments with the same part-of-speech as the preset part-of-speech of the interference word are removed, and the word segments containing the preset keyword are retained to obtain the labeling tag of the sample text. Taking the scenario of using an institution name as labeling information as an example, the institution name is mainly a noun, an adjective, or a verb. Therefore, the preset part-of-speech of the interference word is a preposition, a pronoun, and a conjunction. The word segments of the sample text that belong to the preset part-of-speech of the interference word are deleted, and the results containing the preset keyword such as "company", "office", and "store" are retained.

[0117] It should be noted that step 640 can also include deleting the segmented words that do not meet the length range according to the length of the segmented words, and the embodiments of the present application do not limit this. The sample text can also be preprocessed before step 620, for example: clearing special symbols, punctuation marks or English letters in the sample text, in addition, some short sentences or the beginning and end of the sample text can also be removed, and the labeled tags can be verified after step 640 to improve the accuracy of the labeled tags, and the embodiments of the present application do not limit this.

[0118] The labeling method provided by the embodiments of the present application extracts the labeled tags of the sample text automatically, reduces manual labeling, reduces cost, and improves efficiency and the accuracy of the labeled tags.

[0119] Based on the above embodiments, the present application provides a preferred embodiment, and the labeled tags of the sample text are obtained based on the following steps:

[0120] Step one, semantic role labeling of the sample text: based on the LTP natural language processing tool, the subject and core predicate of each sentence are analyzed, and the purpose of semantic role labeling is to find the corresponding semantic role components of the predicate in the sentence, including core semantic roles (such as agent, patient, etc.) and auxiliary semantic roles (such as place, time, manner, reason, etc.). The task of the embodiments of the present application is to automatically label the nouns of various organization names, so it is necessary to find the subject of the sentence, that is, the agent, in the predicate. Through semantic role labeling of the sample text, a coarse text containing the agent is obtained.

[0121] Step two, Chinese word segmentation and part-of-speech tagging of the coarse text obtained in step one: since the semantic role labeling result may have errors or irrelevant words in the result, the agent content still needs to be analyzed by Chinese word segmentation and part-of-speech tagging to obtain the word segmentation and part-of-speech results in the coarse text.

[0122] Step three, feature selection based on the coarse text and the word segmentation and part-of-speech results obtained in step two: based on the word segmentation and part-of-speech tagging results in the coarse text, the agent content is screened, and first, the results of words with too short length, such as "the company", "the station" and the like, are removed. Through part-of-speech analysis, the part-of-speech of the name is mostly noun, adjective or verb, so the interfering words such as prepositions and pronouns are deleted. Finally, the content containing the preset keywords, such as "company", "place" and "store" and the like, is reserved, and this part of the content is taken as the labeled tag of the sample text.

[0123] Based on the above embodiments, Figure 7 is a flowchart of the labeled tag verification method provided by the present application. As shown in Figure 7 , step 640 includes:

[0124] Step 641, based on the length of each word in the rough division text, the preset interference word and the preset key word, filtering each word in the rough division text to obtain the fine division text of the sample text.

[0125] Step 642, based on the label in the labeled label library, labeling the fine division text to obtain the labeled label.

[0126] Considering that the correct label in the labeled label library can verify whether the labeled label obtained from the sample text is correct, the accuracy of the labeled label from the sample text is further improved.

[0127] Specifically, according to the preset interference word, the word with the same word type as the preset interference word is removed, and the word containing the preset key word is retained to obtain the fine division text of the sample text. Step 642 verifies and labels the fine division text according to the label in the labeled label library to obtain the final labeled label.

[0128] It should be noted that the fine division text can be verified by the label in the labeled label library. If the fine division text matches a label in the labeled label library, the fine division label is used as the labeled label of the sample text, otherwise, the sample text can be discarded or manually labeled, and the embodiments of the present application do not limit this.

[0129] In addition, the labeled label library is updated in real time. Taking the scenario of using the name of an institution as a labeled label as an example, through an enterprise information query tool or an interface provided by the query tool, newly established institution names can be obtained in real time, and these institution names can be updated to the labeled label library. In other scenarios, information query tools or interfaces provided by the query tools in other scenarios can also be called to update the labeled label library in real time.

[0130] The labeling method provided by the embodiments of the present application verifies the labeled label extracted from the sample text by the label in the labeled label library, improves the accuracy of the labeled label of the sample text, and further improves the accuracy of the labeling result of the to-be-labeled text output by the labeling model trained based on the sample text and the labeled label of the sample text.

[0131] The labeling device provided by the embodiments of the present application is described below. The labeling device described below can be correspondingly referred to the labeling method described above.

[0132] Figure 8 is a structural schematic diagram of the labeling device provided by the present application. As Figure 8 shown, the device includes a determination module 810 and a labeling module 820.

[0133] Among them,

[0134] The determination module 810 is configured to determine a to-be-labeled text.

[0135] The labeling module 820 is configured to label each character in the text to be labeled based on the semantic features of the character, and the part-of-speech features and / or keyword features of the word to which the character belongs, to obtain a labeling result of the text to be labeled.

[0136] The keyword feature represents whether the corresponding word is a preset keyword, and the labeling model is trained based on the sample text and the labeling labels of the sample text.

[0137] In the embodiments of the present application, the labeling model can capture more feature information related to the labeling label, thereby improving the accuracy of the labeling result of the text to be labeled, and the labeling model is used to label the text to be labeled, which cancels manual labeling, reduces labeling cost and improves labeling efficiency.

[0138] Based on any of the above embodiments, the labeling module 820 includes:

[0139] The semantic extraction sub-module is configured to extract the semantic features of each character in the text to be labeled based on the embedding layer in the labeling model.

[0140] The independent feature extraction sub-module is configured to extract the independent features of each character based on the independent feature layer in the labeling model.

[0141] The feature fusion sub-module is configured to fuse the semantic features of each character, the part-of-speech features and / or keyword features of the word to which each character belongs, and extract the fused features to obtain the joint features of each character based on the feature fusion layer in the labeling model.

[0142] The labeling sub-module is configured to label each character based on the independent features and joint features of each character based on the labeling layer in the labeling model, to obtain a labeling result of the text to be labeled.

[0143] Based on any of the above embodiments, the feature fusion sub-module includes:

[0144] The feature enhancement sub-module is configured to weight the part-of-speech features and keyword features of the word to which each character belongs based on the feature enhancement layer in the feature fusion layer, to obtain enhanced features of each character based on the part-of-speech feature weight and keyword weight; the part-of-speech feature weight and keyword weight are obtained by training.

[0145] The joint feature extraction submodule is configured to fuse the semantic features of the characters and the enhanced features of the characters based on the enhanced feature extraction layer in the feature fusion layer, and extract the fused features to obtain the joint features of the characters.

[0146] The labeling submodule includes:

[0147] The joint labeling submodule is configured to apply the joint features of the characters to the characters based on the joint feature labeling layer in the labeling layer to obtain a joint labeling result of the text to be labeled.

[0148] The independent labeling submodule is configured to apply the joint labeling result of the text to be labeled and the independent features of the characters to the characters based on the independent feature labeling layer in the labeling layer to obtain an independent labeling result of the text to be labeled, and determine the independent labeling result as the labeling result.

[0149] The labeling module 820 further includes:

[0150] The initial model determination submodule is configured to determine an initial model, which includes the embedding layer, the independent feature layer, the feature fusion layer, and the labeling layer.

[0151] The labeling result determination submodule is configured to determine the joint labeling result and the independent labeling result of the sample text based on the initial model.

[0152] The loss value calculation submodule is configured to determine a training loss value based on the joint labeling result and the independent labeling result of the sample text and the labeling label of the sample text.

[0153] The model parameter update submodule is configured to update the parameters of the initial model based on the training loss value until the training loss value converges to obtain a labeling model.

[0154] The labeling module 820 further includes:

[0155] The sample text determination submodule is configured to determine a sample text and a preset keyword.

[0156] The coarse division submodule is configured to perform semantic role labeling on the sample text to obtain a coarse division text containing preset role content.

[0157] The part-of-speech labeling submodule is configured to perform word segmentation on the coarse division text, and perform part-of-speech labeling on each segmented word of the coarse division text to obtain each segmented word of the coarse division text and the part-of-speech labeling of each segmented word.

[0158] The labeling label determination submodule is configured to determine a labeling label based on the preset interfering part-of-speech, the preset keyword, and each segmented word of the coarse division text and the part-of-speech labeling of each segmented word.

[0159] Based on any of the above embodiments, the submodule for extracting annotation tags includes:

[0160] The sub-module is used to filter each word in the coarse text based on the length of each word in the coarse text, the preset interfering parts of speech, and the preset keywords, to obtain the subdivided text of the sample text.

[0161] The verification submodule is used to annotate the subdivided text based on the tags in the annotation tag library to obtain annotation tags.

[0162] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communication interface 920, a memory 930, and a communication bus 940. The processor 910, communication interface 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can call a computer program in the memory 930 to execute an annotation method. This method includes: determining the text to be annotated; based on an annotation model, applying the semantic features of each character in the text to be annotated, as well as the part-of-speech features and / or keyword features of the word segments to which each character belongs, to annotate each character, thereby obtaining the annotation result of the text to be annotated; the keyword features characterize whether the corresponding word segment is a preset keyword. The annotation model is trained based on sample text and the annotation labels of the sample text, enabling the annotation model to capture more feature information related to the annotation labels, thereby improving the accuracy of the annotation result of the text to be annotated. Furthermore, by using the annotation model to annotate the text, manual annotation is eliminated, reducing annotation costs and improving annotation efficiency.

[0163] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0164] In another aspect, the embodiments of the present application further provide a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program is executable by a processor to enable the computer to perform the labeling method provided by the above embodiments.

[0165] In another aspect, the embodiments of the present application further provide a processor-readable storage medium, which stores a computer program, and the computer program is used to enable a processor to perform the labeling method provided by the above embodiments.

[0166] The processor-readable storage medium can be any available medium or data storage device that can be accessed by a processor, including but not limited to a magnetic storage (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage (such as a CD, a DVD, a BD, a HVD, etc.), and a semiconductor memory (such as a ROM, an EPROM, an EEPROM, a NAND FLASH, a solid state disk (SSD), etc.).

[0167] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiments according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0168] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform the methods described in the various embodiments or some parts of the embodiments.

[0169] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A labeling method characterized by comprising: include: Identify the text to be annotated; Based on the annotation model, the semantic features of each character in the text to be annotated, as well as the part-of-speech features and / or keyword features of the word segment to which each character belongs, are applied to annotate each character to obtain the annotation result of the text to be annotated; The keyword feature characterizes whether the corresponding word segment is a preset keyword, and the annotation model is trained based on the sample text and the annotation labels of the sample text; The annotation-based model applies the semantic features of each character in the text to be annotated, as well as the part-of-speech features and / or keyword features of the word segments to which each character belongs, to annotate each character, thereby obtaining the annotation result of the text to be annotated, including: Based on the embedding layer in the annotation model, the semantic features of each character in the text to be annotated are extracted; Based on the independent feature layer in the annotation model, the semantic features of each character are extracted to obtain the independent features of each character; Based on the feature fusion layer in the annotation model, the semantic features of each character, the part-of-speech features of the word to which each character belongs, and / or the keyword features are fused, and feature extraction is performed on the fused features to obtain the joint features of each character; Based on the annotation layer in the annotation model, the independent and joint features of each character are applied to annotate each character, thereby obtaining the annotation result of the text to be annotated.

2. The labeling method of claim 1, wherein The feature fusion layer in the annotation model fuses the semantic features of each character, the part-of-speech features of the word segment to which each character belongs, and / or keyword features, and extracts the fused features to obtain the joint features of each character, including: Based on the feature enhancement layer in the feature fusion layer, part-of-speech feature weights and keyword weights are applied to weight the part-of-speech features and keyword features of the word segment to which each character belongs, thereby obtaining the enhanced features of each character; the part-of-speech feature weights and keyword weights are obtained through training; Based on the enhanced feature extraction layer in the feature fusion layer, the semantic features and enhanced features of each character are fused, and the fused features are extracted to obtain the joint features of each character.

3. The labeling method according to claim 1 or 2, characterized in that, The annotation layer based on the annotation model applies the independent and joint features of each character to annotate each character, obtaining the annotation result of the text to be annotated, including: Based on the joint feature annotation layer in the annotation layer, the joint features of each character are applied to annotate each character, and the joint annotation result of the text to be annotated is obtained; Based on the independent feature annotation layer in the annotation layer, the joint annotation result of the text to be annotated and the independent features of each character are applied to annotate each character to obtain the independent annotation result of the text to be annotated, and the independent annotation result is determined as the annotation result.

4. The labeling method according to claim 3, wherein The annotation model was trained based on the following steps: Determine an initial model, which includes an embedding layer, an independent feature layer, a feature fusion layer, and a labeling layer; Based on the initial model, the joint annotation results and independent annotation results of the sample text are determined; Based on the joint annotation results and independent annotation results of the sample text, as well as the annotation labels of the sample text, the training loss value is determined; Based on the training loss value, the parameters of the initial model are updated until the training loss value converges, thus obtaining the labeled model.

5. The labeling method according to claim 1 or 2, characterized by, The labeling was determined based on the following steps: Determine the sample text and the preset keywords; Semantic role labeling is performed on the sample text to obtain coarsely segmented text containing content of preset roles; The coarse text is segmented into words, and each segment of the coarse text is tagged with part-of-speech tags to obtain each segment of the coarse text and its part-of-speech tags. Based on preset interference parts of speech, preset keywords, and the parts of speech tags of each word in the coarsely segmented text, the tagging labels are determined.

6. The labeling method of claim 5, wherein The step of determining the tagging labels based on each word segment and part-of-speech tag of the coarsely segmented text, preset interference parts of speech, and preset keywords includes: Based on the length of each word in the coarsely segmented text, the preset interfering parts of speech and preset keywords, the words in the coarsely segmented text are filtered to obtain the subdivided text of the sample text. The subdivided text is annotated based on the tags in the annotation tag library to obtain the annotation tags.

7. A marking device, characterized by include: The determination module is used to determine the text to be annotated; The annotation module is used to annotate each character in the text to be annotated based on the annotation model, and to annotate each character by applying the semantic features of each character, as well as the part-of-speech features and / or keyword features of the word segment to which each character belongs, so as to obtain the annotation result of the text to be annotated. The keyword feature characterizes whether the corresponding word segment is a preset keyword, and the annotation model is trained based on the sample text and the annotation labels of the sample text; The annotation module includes: The semantic extraction submodule is used to extract the semantic features of each character in the text to be annotated based on the embedding layer in the annotation model. An independent feature extraction submodule is used to extract semantic features of each character based on the independent feature layer in the annotation model, so as to obtain the independent features of each character. The feature fusion submodule is used to fuse the semantic features of each character, the part-of-speech features of the word to which each character belongs, and / or the keyword features based on the feature fusion layer in the annotation model, and to extract features from the fused features to obtain the joint features of each character; The annotation submodule is used to annotate each character based on the annotation layer in the annotation model, applying the independent and joint features of each character, to obtain the annotation result of the text to be annotated.

8. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the annotation method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the annotation method as described in any one of claims 1 to 6.

Images