Data processing method, device, electronic device and readable storage medium

Abstract

The embodiments of the present disclosure disclose a data processing method, device, electronic device and readable storage medium. The data processing method includes: obtaining text data, the text data includes reliable data and noise data; obtaining a label of the noise data, the label includes one or more sub-labels; determining a candidate label for the noise data according to the confidence of the multiple sub-labels; generating a label sequence of the text data according to the candidate label of the noise data and the determined label of the reliable data; and training a conditional random field model using the label sequence.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, electronic device and readable storage medium. Background Art

[0002] Named entity recognition is a very important task in the field of natural language processing. Its goal is to extract named entities mentioned in unstructured text, including names of people, organizations, places, medical terms, time, quantity, monetary value, etc.

[0003] The conditional random field model is a probabilistic graphical model commonly used in sequence labeling tasks. It is used to model the conditional probability distribution of another set of output random variables under the condition of a given set of input random variables. The conditional random field model can be used for named entity recognition. A piece of text is input and the probability of various possible label sequences consisting of the labels of each word in the text is output. The label sequence with the highest probability can be determined as the label sequence of the text, and then the entities in the text can be identified based on the label sequence. Word labels can be divided into entity labels and non-entity labels. Entity labels consist of two parts: location information and type information. For example, in the label "B-LOC", "B" indicates the beginning of an entity (location information), and "LOC" indicates that this entity is an entity representing a location (type information). The non-entity label "O" indicates that the word is not an entity.

[0004] Named entity recognition tasks often require a large amount of high-quality labeled data to train the model. In order to obtain this data, a lot of time and cost are consumed. In comparison, a large amount of noisy labeled data can be automatically obtained through remote supervision and other methods. In order to make better use of this noisy data and reduce the negative impact of the noise data, the processing of the noise data is very important. Summary of the invention

[0005] In order to solve the problems in the related art, the embodiments of the present disclosure provide a data processing method, device, electronic device and readable storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides a data processing method, including:

[0007] Acquiring text data, wherein the text data includes reliable data and noise data;

[0008] Obtaining a label of the noise data, where the label includes one or more sub-labels;

[0009] Determining candidate tags for the noise data according to the confidences of the multiple sub-tags;

[0010] Generate a label sequence for the text data according to the candidate labels for the noise data and the determined labels for the reliable data;

[0011] The conditional random field model is trained using the label sequence.

[0012] In combination with the first aspect, in a first implementation of the first aspect of the present disclosure, if the label of the noise data is an entity label, the sub-labels include but are not limited to: a position sub-label, and / or a type sub-label, the position sub-label indicating the position of the noise data in the corresponding entity, and the type sub-label indicating the type of the corresponding entity of the noise data.

[0013] In combination with the first aspect or the first implementation of the first aspect, in a second implementation of the first aspect of the present disclosure, the confidences of the multiple sub-tags are calculated in the following manner:

[0014] For each sub-tag, the confidence of the sub-tag is determined according to the corresponding probabilities that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong.

[0015] In combination with the second implementation manner of the first aspect, in a third implementation manner of the first aspect of the present disclosure, determining the confidence of the sub-tag according to the corresponding probability that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong includes:

[0016] The confidence of the sub-label is obtained by dividing the sum of the corresponding probabilities that the noise data has each label containing the sub-label by the total number of sub-labels in the category to which other sub-labels in the label of the noise data belong.

[0017] In combination with the third implementation of the first aspect, in a fourth implementation of the first aspect of the present disclosure, the method further includes:

[0018] Obtain a word vector representation of the noise data;

[0019] The corresponding probabilities of the noise data having each label including the sub-label are determined according to the word vector representation.

[0020] In combination with the first aspect, in a fifth implementation of the first aspect of the present disclosure, determining a candidate label for the noise data includes:

[0021] The sub-tag whose confidence meets the preset condition is used as the designated sub-tag, and the tag in the tag set containing the designated sub-tag and any sub-tag of the tag not containing the noise data is used as the candidate tag, and the tag set includes all tags that the text data may have; or

[0022] The sub-tag whose confidence meets the preset condition is taken as the designated sub-tag, and the tag in the tag set including the designated sub-tag is taken as the candidate tag.

[0023] In combination with the second implementation manner of the first aspect, in the sixth implementation manner of the first aspect of the present disclosure, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the confidence higher than a preset threshold in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence and higher than a preset threshold in the label of the noise data.

[0024] In combination with the first aspect, in a seventh implementation manner of the first aspect of the present disclosure, determining a candidate label for the noise data includes:

[0025] For a tag containing one subtag and / or a tag without a subtag that meets a preset condition, all tags in a tag set are used as candidate tags, and the tag set includes all tags that the words in the plurality of text data may have.

[0026] In combination with the first aspect, in an eighth implementation of the first aspect of the present disclosure, the step of training a conditional random field model using the label sequence includes:

[0027] The conditional random field model is trained with the goal of maximizing the total probability of the label sequence of multiple text data.

[0028] In combination with the first aspect, in a ninth implementation of the first aspect of the present disclosure, the method further includes:

[0029] The text data is input into a trained conditional random field model, and a predicted label sequence of the text data is determined according to the probability of each label sequence output by the trained conditional random field model.

[0030] In a second aspect, an embodiment of the present disclosure provides a data processing device, including:

[0031] A first acquisition module is configured to acquire text data, wherein the text data includes reliable data and noise data;

[0032] A second acquisition module is configured to acquire a label of the noise data, where the label includes one or more sub-labels;

[0033] A first determination module, configured to determine a candidate tag for the noise data according to the confidences of the multiple sub-tags;

[0034] A generation module, configured to generate a label sequence for the text data according to the candidate labels for the noise data and the determined labels for the reliable data;

[0035] The training module is configured to train a conditional random field model using the label sequence.

[0036] The device is combined with the second aspect. In a first implementation method of the second aspect of the present disclosure, if the label of the noise data is an entity label, the sub-labels include but are not limited to: a position sub-label, and / or a type sub-label, the position sub-label indicates the position of the noise data in the corresponding entity, and the type sub-label indicates the type of the corresponding entity of the noise data.

[0037] In combination with the second aspect or the first implementation of the second aspect, in a second implementation of the second aspect of the present disclosure, the confidences of the multiple sub-tags are calculated in the following manner:

[0038] For each sub-tag, the confidence of the sub-tag is determined according to the corresponding probabilities that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong.

[0039] In combination with the second implementation manner of the second aspect, in a third implementation manner of the second aspect of the present disclosure, determining the confidence of the sub-tag according to the corresponding probability that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong includes:

[0040] The confidence of the sub-label is obtained by dividing the sum of the corresponding probabilities that the noise data has each label containing the sub-label by the total number of sub-labels in the category to which other sub-labels in the label of the noise data belong.

[0041] In combination with the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect of the present disclosure, the device further includes:

[0042] A second acquisition module is configured to acquire a word vector representation of the word with the wrong label;

[0043] The second determination module is configured to determine the corresponding probability that the word has each tag containing the sub-tag according to the word vector representation.

[0044] In combination with the second aspect, in a fifth implementation of the second aspect of the present disclosure, the device determines a candidate label for the noise data, including:

[0045] The sub-tag whose confidence meets the preset condition is used as the designated sub-tag, and the tag in the tag set containing the designated sub-tag and any sub-tag of the tag not containing the noise data is used as the candidate tag, and the tag set includes all tags that the text data may have; or

[0046] The sub-tag whose confidence meets the preset condition is taken as the designated sub-tag, and the tag in the tag set including the designated sub-tag is taken as the candidate tag.

[0047] In combination with the second implementation manner of the second aspect, in a sixth implementation manner of the second aspect of the present disclosure, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the confidence higher than a preset threshold in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence and higher than a preset threshold in the label of the noise data.

[0048] In combination with the second aspect, in a seventh implementation manner of the second aspect of the present disclosure, the device determines a candidate label for the noise data, including:

[0049] For a tag containing one subtag and / or a tag without a subtag that meets a preset condition, all tags in a tag set are used as candidate tags, and the tag set includes all tags that the words in the plurality of text data may have.

[0050] In combination with the second aspect, in an eighth implementation of the second aspect of the present disclosure, the device uses the label sequence to train a conditional random field model, including:

[0051] The conditional random field model is trained with the goal of maximizing the total probability of the label sequence of multiple text data.

[0052] In combination with the second aspect, in a ninth implementation of the second aspect of the present disclosure, the device further includes:

[0053] The input module is configured to input the text data into a trained conditional random field model, and determine a predicted label sequence of the text data according to the probability of each label sequence output by the trained conditional random field model.

[0054] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspect to the ninth implementation manner of the first aspect.

[0055] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the methods described in the first aspect to the ninth implementation manner of the first aspect are implemented.

[0056] According to the technical solution provided by the embodiments of the present disclosure, text data is obtained, wherein the text data includes reliable data and noise data; a label of the noise data is obtained, wherein the label includes one or more sub-labels; candidate labels are determined for the noise data based on the confidence of the multiple sub-labels; a label sequence of the text data is generated based on the candidate labels of the noise data and the determined labels of the reliable data; and a conditional random field model is trained using the label sequence.

[0057] According to the embodiments of the present disclosure, for labels with multiple sub-labels, sub-labels with higher confidence in the wrong labels can be retained, while reducing the negative impact of the wrong labels, and retaining as much reliable information as possible, thereby improving the utilization efficiency of training data in the presence of wrong labels, reducing the amount of computation and time cost of model training. At the same time, since the useful information in the training data is retained as much as possible, the trained model can have better performance.

[0058] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0060] Figure 1 A flow chart showing a data processing method according to an embodiment of the present disclosure.

[0061] Figure 2 A schematic diagram according to an embodiment of the present disclosure is shown.

[0062] Figure 3 A structural block diagram of a data processing device according to an embodiment of the present disclosure is shown.

[0063] Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0064] Figure 5 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0065] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0066] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0067] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0068] In the present disclosure, the acquisition of user information or user data is an operation authorized and confirmed by the user, or actively selected by the user.

[0069] As mentioned above, named entity recognition tasks often require a large amount of high-quality labeled data to train the model. In order to obtain this data, a lot of time and cost are consumed. In comparison, a large amount of noisy labeled data can be automatically obtained through remote supervision and other methods. In order to make better use of these noisy data and reduce the negative impact of noise data on model training, the processing of noise data is very important.

[0070] One possible way to reduce the negative impact of noisy data on model training is to eliminate the impact of incorrect labels by setting them to all possible labels. However, this method ignores the fact that in named entity recognition tasks, there may still be valid information in incorrectly labeled entity labels. Directly discarding incorrect labels will lead to the loss of valid information.

[0071] The present disclosure provides a data processing method, including:

[0072] Acquiring text data, wherein the text data includes reliable data and noise data;

[0073] Obtaining a label of the noise data, where the label includes one or more sub-labels;

[0074] Determining candidate tags for the noise data according to the confidences of the multiple sub-tags;

[0075] Generate a label sequence for the text data according to the candidate labels for the noise data and the determined labels for the reliable data;

[0076] The conditional random field model is trained using the label sequence.

[0077] According to an embodiment of the present disclosure, text data may include, for example, multiple words, wherein words with correct labels are reliable data, and words with incorrect labels are noise data.

[0078] According to the embodiments of the present disclosure, for labels with multiple sub-labels, sub-labels with higher confidence in the wrong labels can be retained, while reducing the negative impact of the wrong labels, and retaining as much reliable information as possible, thereby improving the utilization efficiency of training data in the presence of wrong labels, reducing the amount of computation and time cost of model training. At the same time, since the useful information in the training data is retained as much as possible, the trained model can have better performance.

[0079] For example, we can take advantage of the fact that entity labels in named entity recognition include both location information and type information, retain more reliable location information or type information in erroneous entity labels, and retain as much information about related entities as possible while reducing the negative impact of erroneous labels. This will improve the efficiency of training data utilization and reduce the amount of computation and time cost of model training in the presence of erroneous labels. At the same time, by retaining as much useful information in the training data as possible, the trained model can have a better named entity recognition effect.

[0080] The solution disclosed in the present invention can be used in many fields such as e-commerce, logistics, news, and medical treatment. In the e-commerce field, the e-commerce platform can identify the product name, quantity, price, etc. based on the product description entered by the user. In the logistics field, the logistics platform can identify the place name, person name, etc. based on the address information on the order. In the news field, the text automatic recognition system can identify the name of a person, organization name, place name, etc. from the news text. In the medical field, the smart medical system can identify the name of a person, medical terminology, disease name, drug name, etc. from the medical record.

[0081] Figure 1 A flow chart showing a data processing method according to an embodiment of the present disclosure.

[0082] like Figure 1 As shown, the data processing method includes the following steps S101-S105:

[0083] In step S101, text data is obtained, wherein the text data includes reliable data and noise data;

[0084] In step S102, a label of the noise data is obtained, where the label includes one or more sub-labels;

[0085] In step S103, determining candidate tags for the noise data according to the confidences of the multiple sub-tags;

[0086] In step S104, a label sequence of the text data is generated according to the candidate labels of the noise data and the determined labels of the reliable data;

[0087] In step S105, the conditional random field model is trained using the label sequence.

[0088] According to an embodiment of the present disclosure, text data may include, for example, multiple words, wherein words with correct labels are reliable data, and words with incorrect labels are noise data.

[0089] According to the embodiments of the present disclosure, for labels with multiple sub-labels, sub-labels with higher confidence in the wrong labels can be retained, while reducing the negative impact of the wrong labels, and retaining as much reliable information as possible, thereby improving the utilization efficiency of training data in the presence of wrong labels, reducing the amount of computation and time cost of model training. At the same time, since the useful information in the training data is retained as much as possible, the trained model can have better performance.

[0090] According to an embodiment of the present disclosure, the multiple sub-tags are two sub-tags; and / or if the label of the noise data is an entity label, the sub-tags include but are not limited to: a position sub-tag, and / or a type sub-tag, the position sub-tag indicates the position of the noise data in the corresponding entity, and the type sub-tag indicates the type of the corresponding entity of the noise data.

[0091] Figure 2 A schematic diagram according to an embodiment of the present disclosure is shown.

[0092] Understandable, although Figure 2 Named entity recognition is taken as an example to illustrate the principle of the present disclosure, but the present disclosure is not limited thereto, but can be applied to various practical application scenarios for processing text data with incorrect labels.

[0093] As an example, the position subtag of the entity tag may include "B" and "I", which respectively indicate that the corresponding word is located at the start position of the entity and the non-start position of the entity (including the middle position and the end), and the type subtag of the entity tag may include "PER", "LOC", "ORG", etc., which respectively indicate that the type of the entity to which the corresponding word belongs is a person's name, a location, an organization, etc. The non-entity tag may be "O", which indicates that the corresponding word does not belong to an entity.

[0094] like Figure 2 As shown in the figure, the label sequence of the text data "Brooklyn and Mary live in New York" is "B-LOC, O, B-PER, O, O, O, B-LOC", while the correct label sequence of the text data should be "B-PER, O, B-PER, O, O, B-LOC, I-LOC". In other words, there are wrong labels in the label sequence of the text data, that is, the labels of the words "Brooklyn", "New", and "York" are wrong labels. Words with wrong labels are called noise data, and words with correct labels are called reliable data.

[0095] According to an embodiment of the present disclosure, for an erroneous label including multiple sub-labels, namely, the erroneous label "B-LOC" of "Brooklyn" and the erroneous label "B-LOC" of "York", the confidence of each sub-label is calculated, and its candidate label is determined according to the confidence of the sub-label.

[0096] For example, assuming that the confidence of the sub-label "B" in the wrong label "B-LOC" of the noise data "Brooklyn" is higher than the confidence of the sub-label "LOC", the labels "B-PER" and "B-LOC" containing the sub-label "B" in the label set, and the label "O" containing neither "B" nor "LOC" can be used as candidate labels for "Brooklyn". According to an embodiment of the present disclosure, for a specified text annotation task, a label set can be pre-set, and the label set contains all labels that the words in the text to be annotated may have.

[0097] Assuming that the confidence of the sub-label "LOC" in the wrong label "B-LOC" of the noise data "York" is higher than the confidence of the sub-label "B", the labels "B-LOC" and "I-LOC" containing the sub-label "LOC" in the label set, and the label "O" containing neither "B" nor "LOC" can be taken as candidate labels for "York".

[0098] For an erroneous label including only one sub-label, ie, an erroneous label “O” of “New”, its candidate labels may be determined as all possible labels in the label set.

[0099] like Figure 2 As shown, after determining the candidate labels for the noise data, multiple label sequences can be generated for the text data "Brooklyn and Mary live in New York" based on the labels of the reliable data in the text data and the candidate labels for the noise data.

[0100] exist Figure 2In the example shown, the first label of the label sequence can be one of "B-PER", "B-LOC", "O", the second label is "O", the third label is "B-PER", the fourth label is "O", the fifth label is "O", the sixth label is one of "B-PER", "I-PER", "B-LOC", "I-LOC", "O", and the seventh label is one of "B-LOC", "I-LOC", "O". Therefore, Figure 2 The text data "Brooklyn and Mary live in New York" shown has a total of 3*1*1*1*1*1*5*3=45 tag sequences.

[0101] In these label sequences, the information of sub-labels with high confidence, i.e., more reliable, in the wrong entity labels is retained as much as possible, which improves the utilization efficiency of training data and reduces the amount of computation and time cost of model training. At the same time, since the useful information in the training data is retained as much as possible, the trained model can have better performance.

[0102] According to an embodiment of the present disclosure, the confidences of the multiple sub-labels are calculated in the following manner:

[0103] For each sub-tag, the confidence of the sub-tag is determined according to the corresponding probabilities that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong.

[0104] According to an embodiment of the present disclosure, determining the confidence of the sub-tag according to the corresponding probability that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong includes:

[0105] The confidence of the sub-label is obtained by dividing the sum of the corresponding probabilities that the noise data has each label containing the sub-label by the total number of sub-labels in the category to which other sub-labels in the label of the noise data belong.

[0106] For example, define a label y i The form is y i p -y i t , for example: if y i =B-LOC, then y i p =B,y i t =LOC. At the same time, define Y p is the set of all position sub-labels in the label set (e.g. {B, I}), Yt is the set of all sub-tags of the tag set (e.g. {PER,LOC}). For an incorrect entity tag y p -y t ,y p ∈Y p ,y t ∈Y t , in order to determine which sub-label is more reliable, the confidence p of the position sub-label and the type sub-label can be calculated separately pos and p type ,

[0107]

[0108]

[0109] in, Indicates that the noise data has label y i The probability of Indicates that the noise data has sub-labels y p Label y i The probability sum of Indicates that the noise data has sub-labels y t Label y i The probability sum of |Y t | represents the set Y t The number of elements in , that is, the total number of type subtags, |Y p | represents the set Y p The number of elements in , that is, the total number of positional sublabels.

[0110] It should be noted that, although the embodiments of the present disclosure are described by taking the case where a tag includes two sub-tags as an example, the embodiments of the present disclosure are also applicable to the case where a tag includes multiple sub-tags.

[0111] According to an embodiment of the present disclosure, the method further includes: obtaining a word vector representation of the noise data; and determining, based on the word vector representation, a corresponding probability that the noise data has each label containing the sub-label.

[0112] For example, for each word in the text data, we can first obtain its word vector representation x by, for example, a convolutional neural network or a recurrent neural network model. t Then, through the bidirectional LSTM (Long Short-Term Memory Neural Network) or convolutional neural network or GRU model or Transformer model, based on the word vector representation x t Get the word representation h containing the context information of the word t =BiLSTM(x t). Then, the probability p of the word belonging to each label is obtained through the multi-layer perceptron t =softmax(MLP(h t )).

[0113] According to an embodiment of the present disclosure, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence in the tag of the noise data. For example, when a tag contains two sub-tags with different confidences, the sub-tag whose confidence meets the preset conditions is the sub-tag with higher confidence, and it is used as the designated sub-tag. In this way, the information of the sub-tag with higher confidence in the tag of the noise data can be retained, thereby improving the utilization rate of the training data.

[0114] According to an embodiment of the present disclosure, the sub-tag whose confidence meets the preset conditions includes the sub-tag whose confidence is higher than the preset threshold in the tag of the noise data. When the confidence of the sub-tag is higher than the preset threshold, the sub-tag can be considered to be credible and used as the designated sub-tag. In this way, the more credible information in the tag of the noise data can be retained as much as possible, and the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence in the tag of the noise data and higher than the preset threshold.

[0115] According to an embodiment of the present disclosure, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence in the tag of the noise data and higher than the preset threshold, which is used as the designated sub-tag. For example, when a tag contains two sub-tags, if the confidence of both sub-tags is not higher than the preset threshold, even the sub-tag with the highest confidence is not considered to be credible. Only when the confidence of the sub-tag is the highest and higher than the preset threshold, the sub-tag is used as the designated sub-tag. In this way, the credibility of the sub-tags retained in the erroneous tags can be further improved, thereby improving the accuracy of the trained model.

[0116] According to an embodiment of the present disclosure, a sub-tag in the label of the noise data whose confidence meets a preset condition is taken as a designated sub-tag, and a label in a label set that includes the designated sub-tag and a label that does not include any sub-tag in the label of the noise data is taken as the candidate label, and the label set includes all labels that the words in the multiple text data may have.

[0117] For example, Figure 2As shown, for the noise data "Brooklyn", assuming that the sub-tag in its label that meets the preset conditions is "B", "B-LOC", "B-PER" and "O" are used as its candidate labels. In the embodiment where the sub-tag with the highest confidence is used as the designated sub-tag, it is possible that the credibility of the sub-tag with the highest confidence is still not high, so in addition to using the label containing the sub-tag as a candidate label, the label "O" of any sub-tag in the label that does not contain noise data is also used as a candidate label.

[0118] Alternatively, according to an embodiment of the present disclosure, a sub-tag in the erroneous tag whose confidence meets a preset condition is used as a designated sub-tag, and a tag in a tag set that includes the designated sub-tag is used as the candidate tag.

[0119] For example, for the noise data "Brooklyn", assuming that the sub-label in its wrong label that meets the preset conditions is "B", "B-LOC" and "B-PER" are used as its candidate labels. For sub-labels with confidence higher than the preset threshold, it can be considered that their credibility is high enough, so only the label containing the sub-label is used as a candidate label to improve the model training efficiency and reduce the consumption of computing and storage resources.

[0120] According to an embodiment of the present disclosure, determining candidate tags for the noise data includes: for a tag containing a sub-tag and / or a tag that does not have a sub-tag that meets a preset condition, all tags in a tag set are used as candidate tags, and the tag set includes all tags that the words in the multiple text data may have.

[0121] For example, if the noise data has a label "O" which has only one sub-label, all labels in the label set are used as candidate labels for the noise data. If none of the sub-labels of the noise data meets the preset conditions, for example, their confidences are all lower than the preset minimum confidence, then it is considered that none of the sub-labels of the noise data is reliable, so all labels in the label set are used as candidate labels for the noise data.

[0122] According to an embodiment of the present disclosure, the method of training a conditional random field model using the label sequence includes: training the conditional random field model with maximizing the total probability of the label sequence of multiple text data as a training goal.

[0123] For example, the loss function of the conditional random field model is:

[0124]

[0125] Among them, C(Y p ) is the set of all possible label sequences, P(y|S) is the probability that the label sequence of text data S is y, Represents all text data The calculation formula of the probability of the label sequence is the same as the probability calculation method of the label sequence in the conditional random field (CRF) of the prior art, and will not be repeated here.

[0126] According to an embodiment of the present disclosure, the method further includes: inputting the text data into a trained conditional random field model, and determining a predicted label sequence for the text data according to the probability of each label sequence output by the trained conditional random field model. For example, the label sequence corresponding to the maximum probability is used as the predicted label sequence for the text data.

[0127] Figure 3 The structural block diagram of the data processing device according to the embodiment of the present disclosure is shown. The device can be implemented as part or all of the electronic device through software, hardware or a combination of both.

[0128] like Figure 3 As shown, the data processing device 300 includes a first acquisition module 310 , a second acquisition module 320 , a first determination module 330 , a generation module 340 , and a training module 350 .

[0129] The first acquisition module 310 is configured to acquire text data, wherein the text data includes reliable data and noise data;

[0130] The second acquisition module 320 is configured to acquire a label of the noise data, where the label includes one or more sub-labels;

[0131] The first determination module 330 is configured to determine a candidate tag for the noise data according to the confidences of the plurality of sub-tags;

[0132] The generating module 340 is configured to generate a label sequence for the text data according to the candidate labels for the noise data and the determined labels for the reliable data;

[0133] The training module 350 is configured to train a conditional random field model using the label sequence.

[0134] According to an embodiment of the present disclosure, if the label of the noise data is an entity label, the sub-labels include but are not limited to: a position sub-label, and / or a type sub-label, the position sub-label indicating the position of the noise data in the corresponding entity, and the type sub-label indicating the type of the corresponding entity of the noise data.

[0135] According to an embodiment of the present disclosure, the confidences of the multiple sub-labels are calculated in the following manner:

[0136] For each sub-tag, the confidence of the sub-tag is determined according to the corresponding probabilities that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong.

[0137] According to an embodiment of the present disclosure, determining the confidence of the sub-tag according to the corresponding probability that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong includes:

[0138] The confidence of the sub-label is obtained by dividing the sum of the corresponding probabilities that the noise data has each label containing the sub-label by the total number of sub-labels in the category to which other sub-labels in the label of the noise data belong.

[0139] According to an embodiment of the present disclosure, the device 300 further includes:

[0140] A third acquisition module 360 ​​is configured to obtain a word vector representation of the noise data;

[0141] The second determination module 370 is configured to determine the corresponding probability that the noise data has each label containing the sub-label according to the word vector representation.

[0142] According to an embodiment of the present disclosure, determining candidate tags for the noise data includes:

[0143] The sub-tag whose confidence meets the preset condition is used as the designated sub-tag, and the tag in the tag set containing the designated sub-tag and any sub-tag of the tag not containing the noise data is used as the candidate tag, and the tag set includes all tags that the text data may have; or

[0144] The sub-tag whose confidence meets the preset condition is taken as the designated sub-tag, and the tag in the tag set including the designated sub-tag is taken as the candidate tag.

[0145] According to an embodiment of the present disclosure, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the confidence higher than a preset threshold in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence and higher than a preset threshold in the label of the noise data.

[0146] According to an embodiment of the present disclosure, determining candidate tags for the noise data includes:

[0147] For a tag containing one subtag and / or a tag without a subtag that meets a preset condition, all tags in a tag set are used as candidate tags, and the tag set includes all tags that the words in the plurality of text data may have.

[0148] According to an embodiment of the present disclosure, the training of the conditional random field model using the label sequence includes:

[0149] The conditional random field model is trained with the goal of maximizing the total probability of the label sequence of multiple text data.

[0150] According to an embodiment of the present disclosure, the device 300 further includes:

[0151] The input module 380 is configured to input the text data into the trained conditional random field model, and determine the predicted label sequence of the text data according to the probability of each label sequence output by the trained conditional random field model.

[0152] The present disclosure also discloses an electronic device, Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0153] like Figure 4 As shown, the electronic device 400 includes a memory 401 and a processor 402, wherein the memory 401 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 402 to implement the method according to an embodiment of the present disclosure.

[0154] Specifically, the data processing method includes:

[0155] Acquiring text data, wherein the text data includes reliable data and noise data;

[0156] Obtaining a label of the noise data, where the label includes one or more sub-labels;

[0157] Determining candidate tags for the noise data according to the confidences of the multiple sub-tags;

[0158] Generate a label sequence for the text data according to the candidate labels for the noise data and the determined labels for the reliable data;

[0159] The conditional random field model is trained using the label sequence.

[0160] According to an embodiment of the present disclosure, if the label of the noise data is an entity label, the sub-labels include but are not limited to: a position sub-label, and / or a type sub-label, the position sub-label indicating the position of the noise data in the corresponding entity, and the type sub-label indicating the type of the corresponding entity of the noise data.

[0161] According to an embodiment of the present disclosure, the confidences of the multiple sub-labels are calculated in the following manner:

[0162] For each sub-tag, the confidence of the sub-tag is determined according to the corresponding probabilities that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong.

[0163] According to an embodiment of the present disclosure, determining the confidence of the sub-tag according to the corresponding probability that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong includes:

[0164] The confidence of the sub-label is obtained by dividing the sum of the corresponding probabilities that the noise data has each label containing the sub-label by the total number of sub-labels in the category to which other sub-labels in the label of the noise data belong.

[0165] According to an embodiment of the present disclosure, the method further includes:

[0166] Obtain a word vector representation of the noise data;

[0167] The corresponding probabilities of the noise data having each label including the sub-label are determined according to the word vector representation.

[0168] According to an embodiment of the present disclosure, determining candidate tags for the noise data includes:

[0169] The sub-tag whose confidence meets the preset condition is used as the designated sub-tag, and the tag in the tag set containing the designated sub-tag and any sub-tag of the tag not containing the noise data is used as the candidate tag, and the tag set includes all tags that the text data may have; or

[0170] The sub-tag whose confidence meets the preset condition is taken as the designated sub-tag, and the tag in the tag set including the designated sub-tag is taken as the candidate tag.

[0171] According to an embodiment of the present disclosure, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the confidence higher than a preset threshold in the label of the noise data; and / or, the sub-tag whose confidence meets the preset conditions includes the sub-tag with the highest confidence and higher than a preset threshold in the label of the noise data.

[0172] According to an embodiment of the present disclosure, determining candidate tags for the noise data includes:

[0173] For a tag containing one subtag and / or a tag without a subtag that meets a preset condition, all tags in a tag set are used as candidate tags, and the tag set includes all tags that the words in the plurality of text data may have.

[0174] According to an embodiment of the present disclosure, the training of the conditional random field model using the label sequence includes:

[0175] The conditional random field model is trained with the goal of maximizing the total probability of the label sequence of multiple text data.

[0176] According to an embodiment of the present disclosure, the method further includes:

[0177] The text data is input into a trained conditional random field model, and a predicted label sequence of the text data is determined according to the probability of each label sequence output by the trained conditional random field model.

[0178] Figure 5 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0179] like Figure 5 As shown, the computer system 500 includes a processing unit 501, which can perform various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The processing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0180] The following components are connected to the I / O interface 505: an input part 506 including a keyboard, a mouse, etc.; an output part 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part 508 including a hard disk, etc.; and a communication part 509 including a network interface card such as a LAN card, a modem, etc. The communication part 509 performs communication processing via a network such as the Internet. The drive 510 is also connected to the I / O interface 505 as needed. Removable media 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive 510 as needed, so that the computer program read therefrom is installed into the storage part 508 as needed. Among them, the processing unit 501 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0181] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a computer-readable storage medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 509, and / or installed from a removable medium 511.

[0182] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0183] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0184] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0185] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A data processing method, comprising: Acquiring text data, wherein the text data includes reliable data and noise data; Obtaining a label of the noise data, where the label includes one or more sub-labels; Determining candidate tags for the noise data according to the confidences of the multiple sub-tags; Generate a label sequence for the text data according to the candidate labels for the noise data and the determined labels for the reliable data; Training a conditional random field model using the label sequence; Among them, determining candidate tags for the noise data based on the confidence of the multiple sub-tags includes: determining candidate tags for the noise data based on sub-tags with higher confidence among the tags, wherein the sub-tag with higher confidence is a sub-tag with the highest confidence and / or a confidence higher than a preset threshold.

2. The method according to claim 1, wherein: If the label of the noise data is an entity label, the sub-labels include but are not limited to: a position sub-label, and / or a type sub-label, wherein the position sub-label indicates the position of the noise data in the corresponding entity, and the type sub-label indicates the type of the corresponding entity of the noise data.

3. The method according to claim 1 or 2, wherein: The confidence of the multiple sub-labels is calculated as follows: For each sub-tag, the confidence of the sub-tag is determined according to the corresponding probabilities that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong.

4. The method according to claim 3, wherein: The determining the confidence of the sub-tag according to the corresponding probability that the noise data has each tag containing the sub-tag and the total number of sub-tags in the category to which other sub-tags in the tag of the noise data belong comprises: The confidence of the sub-label is obtained by dividing the sum of the corresponding probabilities that the noise data has each label containing the sub-label by the total number of sub-labels in the category to which other sub-labels in the label of the noise data belong.

5. The method according to claim 4, further comprising: Obtain a word vector representation of the noise data; The corresponding probabilities of the noise data having each label including the sub-label are determined according to the word vector representation.

6. The method according to claim 1, wherein: The determining of candidate labels for the noise data comprises: The sub-tag whose confidence meets the preset condition is used as the designated sub-tag, and the tag containing the designated sub-tag and any sub-tag of the tag not containing the noise data in the tag set is used as the candidate tag, and the tag set includes all tags of the text data; or The sub-tag whose confidence meets the preset condition is taken as the designated sub-tag, and the tag in the tag set including the designated sub-tag is taken as the candidate tag.

7. The method according to claim 3, wherein: The sub-tags whose confidences meet the preset conditions include the sub-tags with the highest confidences in the tags of the noise data; and / or, the sub-tags whose confidences meet the preset conditions include the sub-tags with confidences higher than a preset threshold in the tags of the noise data; And / or, the sub-tags whose confidences meet preset conditions include sub-tags whose confidences are the highest and higher than a preset threshold in the tags of the noise data.

8. The method according to claim 1, wherein: The determining of candidate labels for the noise data comprises: For a tag containing one subtag and / or a tag having no subtag that meets a preset condition, all tags in a tag set are used as candidate tags, and the tag set includes all tags of words in the plurality of text data.

9. The method according to claim 1, wherein the training of the conditional random field model using the label sequence comprises: The conditional random field model is trained with the goal of maximizing the total probability of the label sequence of multiple text data.

10. The method according to claim 1, further comprising: The text data is input into a trained conditional random field model, and a predicted label sequence of the text data is determined according to the probability of each label sequence output by the trained conditional random field model.

11. A data processing device, comprising: A first acquisition module is configured to acquire text data, wherein the text data includes reliable data and noise data; A second acquisition module is configured to acquire a label of the noise data, where the label includes one or more sub-labels; A first determination module is configured to determine a candidate tag for the noise data according to the confidences of the multiple sub-tags; A generation module, configured to generate a label sequence for the text data according to the candidate labels for the noise data and the determined labels for the reliable data; A training module, configured to train a conditional random field model using the label sequence; Among them, the first determination module is also used to determine a candidate tag for the noise data based on a sub-tag with higher confidence in the tag, wherein the sub-tag with higher confidence is a sub-tag with the highest confidence and / or a confidence higher than a preset threshold.

12. An electronic device comprising a memory and a processor; wherein: The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1-10.

13. A readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the method steps described in any one of claims 1 to 10.

Images