A method, apparatus and electronic device for expanding labeled data

Abstract

This invention provides a method, apparatus, and electronic device for expanding labeled data, relating to the field of knowledge graph technology. The method includes: acquiring various basic labeled data to be expanded, and determining preset replacement data belonging to various types of elements; for each basic labeled data, determining each set of replacement content included in the basic labeled data; for each set of replacement content, determining multiple sets of target content matching the set of replacement content in each set of replacement data; for each set of target content, using each target data in the set of target content to replace the basic data in the basic labeled data containing the replacement content corresponding to the set of target content that belongs to the set of replacement content and matches the target data, thereby obtaining target labeled data. Compared with the prior art, the solution provided by this invention can expand labeled data using less manually labeled labeled data to obtain a large amount of labeled data.

Description

Technical Field

[0001] This invention relates to the field of knowledge graph technology, and in particular to a method, apparatus, and electronic device for expanding labeled data. Background Technology

[0002] A knowledge graph is a knowledge network that includes nodes and edges. Nodes represent entities, and edges represent the relationships between connected entities. Furthermore, entities and relationships often have multiple attributes.

[0003] In related technologies, when constructing knowledge graphs from unstructured data, a knowledge extraction model based on supervised deep learning algorithms is typically used to extract entity elements, attribute elements, and relational elements representing the relationship between two entity elements from the unstructured data. Then, the extracted elements are used to construct the knowledge graph. The process of extracting these various types of elements can be called knowledge extraction.

[0004] In the aforementioned related technologies, a large amount of labeled data is required to train the knowledge extraction model. Each labeled data includes data content and labeled content. The data content includes at least one of the following: entity elements, attribute elements, and text data representing the relationship between two entities. The labeled content includes: the entity subtype to which the entity element belongs, the attribute subtype to which the attribute element belongs, and the relationship subtype to which the relationship element belongs. Specifically, the relationship subtype to which the relationship element belongs in the labeled content refers to the relationship subtype corresponding to the text data representing the relationship between two entities in the data content.

[0005] For example, the data content of the labeled data is: [22 years old] [Xiaoming] and [Xiaohong] got married. Here, since [22 years old] belongs to the [age] subtype under the attribute element, [Xiaoming] and [Xiaohong] belong to the [person] subtype under the entity element, and the relationship between [Xiaoming] and [Xiaohong] belongs to the [couple] subtype under the relationship element, the labeled content of the above labeled data is: [22 years old] is labeled as the attribute [age], [Xiaoming] and [Xiaohong] are labeled as the entity [person], and the relationship between [Xiaoming] and [Xiaohong] is labeled as the relationship [couple].

[0006] However, in the aforementioned related technologies, the labeled data is generated through manual annotation. Therefore, when a large amount of labeled data needs to be generated, it will take a lot of time, resulting in low efficiency in the generation of labeled data.

[0007] Therefore, how to expand the labeled data using a small amount of manually labeled data to obtain a large amount of labeled data is a problem that urgently needs to be solved. Summary of the Invention

[0008] The purpose of this invention is to provide a method, apparatus, and electronic device for expanding labeled data, so as to expand labeled data with less manual annotation to obtain a large amount of labeled data. The specific technical solution is as follows:

[0009] In a first aspect, embodiments of the present invention provide a method for expanding labeled data, the method comprising:

[0010] Obtain the basic annotation data to be expanded and determine the preset replacement data belonging to each type of element; wherein, each replacement data is labeled with its element type and the subtype under that element type;

[0011] For each set of basic labeled data, determine the replacement content included in the basic labeled data; wherein, each set of replacement content includes: at least one set of basic data in the relation subtype to which the entity element, attribute element and relation element to be replaced in the basic labeled data belongs, and the replacement content of each set is not completely the same;

[0012] For each set of replacement content, multiple sets of target content that match the set of replacement content are determined in each set of replacement data; wherein, each set of target content includes each set of target data that matches each set of basic data in the set of replacement content, each set of target data and the matching basic data belong to the same type of element, the sets of target content are not completely the same, and each set of target content is completely different from the set of replacement content.

[0013] For each set of target content, the target data is replaced by the basic data in the basic annotation data that belongs to the replacement content and matches the target data, using the target data in the target content.

[0014] Optionally, in one specific implementation, for each group of replacement content, the determined multiple groups of target content that match the group of replacement content include: multiple groups of first-class content and / or multiple groups of second-class content;

[0015] In each group of the first category of content, each target data and its matching basic data belong to the same subtype under the same element, while each target data and its matching basic data belong to different subtypes under the same element.

[0016] Optionally, in one specific implementation, the method further includes:

[0017] Multiple sets of annotation data to be merged are obtained from the various basic annotation data and / or the various target annotation data obtained; wherein, each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same;

[0018] Based on the common entity elements in each set of labeled data to be merged, the labeled data in each set of labeled data to be merged are combined to obtain new target labeled data.

[0019] Optionally, in one specific implementation, the method further includes:

[0020] Select multiple annotation data to be processed from the various basic annotation data and / or the various target annotation data obtained;

[0021] Using a preset data augmentation method, at least one augmented annotation data is generated for each annotation data to be processed, serving as new target annotation data; wherein, the data augmentation method includes at least one of: back translation, random insertion, and random deletion.

[0022] Optionally, in one specific implementation, the step of determining the preset replacement data belonging to various types of elements includes:

[0023] Obtain candidate data belonging to various elements in the preset target knowledge graph, and calculate the number of times each candidate data appears in the various basic labeled data;

[0024] For each piece of data to be replaced in the various basic labeled data, the similarity between the piece of data to be replaced and each of the corresponding specified data is calculated; wherein, each of the specified data corresponding to each piece of data to be replaced is data of the same subtype under the same element as the piece of data to be replaced in the various candidate data.

[0025] Based on the calculated occurrence counts and similarities, candidate data corresponding to each data to be replaced are determined, and the element type and subtypes under the element type are labeled for each candidate data, thus obtaining each replacement data.

[0026] Optionally, in one specific implementation, the step of determining each candidate data corresponding to each data to be replaced based on the calculated occurrence counts and similarities includes:

[0027] For each piece of data to be replaced, arrange the specified data corresponding to the data to be replaced according to a preset arrangement method; wherein, the preset arrangement method includes: if the similarity is different, arrange them in order of increasing similarity and if the similarity is the same, arrange them in order of increasing frequency of occurrence.

[0028] For each piece of data to be replaced, retrieve the specified data that precedes the specified position from the sorted specified data, and use them as candidate data corresponding to each piece of data to be replaced.

[0029] Optionally, in one specific implementation, before the step of determining the sets of replacement content included in each set of basic annotation data, the method further includes:

[0030] Based on the metadata of the target knowledge graph, a target annotation data template is generated; wherein the structural features between entity elements, attribute elements and relation elements represented by the target annotation data template are different from the structural features between entity elements, attribute elements and relation elements represented by each basic annotation data.

[0031] The target annotation data template is filled with each candidate data to obtain new basic annotation data.

[0032] Optionally, in one specific implementation, before the step of determining the sets of replacement content included in each set of basic annotation data, the method further includes:

[0033] From the target knowledge graph, at least one sub-graph is extracted, and based on the various elements included in each sub-graph, basic annotation data corresponding to each sub-graph is generated.

[0034] The method further includes:

[0035] Multiple sets of annotation data to be merged are obtained from the target annotation data generated using the basic annotation data corresponding to each sub-map; each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same.

[0036] Based on the common entity elements in each set of labeled data to be merged, the labeled data in each set of labeled data to be merged are combined to obtain new target labeled data.

[0037] Optionally, in one specific implementation, the step of obtaining the various basic annotation data to be expanded includes:

[0038] Obtain the preset initial annotation data;

[0039] Based on the metadata of the target knowledge graph, erroneous labeled data in each of the initial labeled data are identified, and the identified erroneous labeled data is deleted to obtain each of the basic labeled data to be expanded.

[0040] Optionally, in one specific implementation, after the step of deleting the determined erroneous labeled data and before the step of obtaining the various basic labeled data to be expanded, the method further includes:

[0041] For each initial labeled data remaining after deleting the identified erroneous labeled data, determine whether there is any unlabeled data to be labeled in the labeled data; wherein, the data to be labeled is: entity elements and / or attribute elements;

[0042] If it exists, for the data to be labeled, determine whether there is candidate data including the data to be labeled among the candidate data; otherwise, determine the initial labeled data as the basic labeled data.

[0043] If there exists candidate data that includes the element to be labeled and there is only one candidate data, then label the element type to which the candidate data belongs and the subtype under that element type for the element to be labeled.

[0044] If there are multiple candidate data that include the element to be labeled, then the candidate data with the longest length among the existing candidate data is determined, and the element type and subtype under that element type are labeled to the element to be labeled.

[0045] If no candidate data containing the element to be labeled exists, the initial labeled data is determined as the base labeled data.

[0046] Secondly, embodiments of the present invention provide an apparatus for expanding labeled data, the apparatus comprising:

[0047] The data acquisition module is used to acquire the basic labeled data to be expanded and to determine the preset replacement data belonging to each type of element; wherein, each replacement data is labeled with its element type and the subtype under that element type;

[0048] The replacement content determination module is used to determine the replacement content of each set of basic labeled data for each basic labeled data; wherein each set of replacement content includes: at least one basic data in the relation subtype to which the entity element, attribute element and relation element to be replaced in the basic labeled data belongs, and the replacement content of each set is not completely the same;

[0049] The target content determination module is used to determine multiple sets of target content that match each set of replacement content in the various replacement data for each set of replacement content; wherein, each set of target content includes each set of target data that matches each set of basic data in the set of replacement content, each set of target data and the matching basic data belong to the same type of element, the sets of target content are not completely the same, and each set of target content is completely different from the set of replacement content.

[0050] The content replacement module is used to replace the basic data in the basic annotation data that belongs to the replacement content and matches the target data for each group of target content, using the target data in the target content of that group of target content, so as to obtain the target annotation data.

[0051] Optionally, in one specific implementation, for each group of replacement content, the determined multiple groups of target content that match the group of replacement content include: multiple groups of first-class content and / or multiple groups of second-class content;

[0052] In each group of the first category of content, each target data and its matching basic data belong to the same subtype under the same element, while each target data and its matching basic data belong to different subtypes under the same element.

[0053] Optionally, in one specific implementation, the apparatus further includes:

[0054] The first data acquisition module is used to acquire multiple sets of annotation data to be merged from the various basic annotation data and / or the various target annotation data obtained; wherein each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same;

[0055] The first data merging module is used to combine the various annotation data in each group of annotation data to be merged based on the common entity elements of each annotation data in the group of annotation data to be merged, so as to obtain new target annotation data.

[0056] Optionally, in one specific implementation, the apparatus further includes:

[0057] The data acquisition module is used to select multiple data to be processed from the various basic annotation data and / or the various target annotation data obtained;

[0058] The data augmentation module is used to generate at least one augmented annotation data corresponding to each annotation data to be processed, as a new target annotation data, using a preset data augmentation device; wherein, the data augmentation device includes at least one of: back translation, random insertion and random deletion.

[0059] Optionally, in one specific implementation, the data acquisition module includes:

[0060] The first calculation submodule is used to obtain each candidate data belonging to various types of elements that appear in the preset target knowledge graph, and to calculate the number of times each candidate data appears in the various basic annotation data.

[0061] The second calculation submodule is used to calculate the similarity between each data to be replaced and each corresponding specified data for each data to be replaced in the various basic labeled data; wherein, each specified data corresponding to each data to be replaced is data of the same subtype under the same element as the data to be replaced in the various candidate data.

[0062] The data acquisition submodule is used to determine each candidate data corresponding to each data to be replaced based on the calculated occurrence frequency and similarity, and to label each candidate data with the element type to which the candidate data belongs and the subtype under the element type, so as to obtain each replacement data.

[0063] Optionally, in one specific implementation, the data acquisition submodule is specifically used for:

[0064] For each piece of data to be replaced, arrange the specified data corresponding to the data to be replaced according to a preset arrangement method; wherein, the preset arrangement method includes: if the similarity is different, arrange them in order of increasing similarity and if the similarity is the same, arrange them in order of increasing frequency of occurrence.

[0065] For each piece of data to be replaced, retrieve the specified data that precedes the specified position from the sorted specified data, and use them as candidate data corresponding to each piece of data to be replaced.

[0066] Optionally, in one specific implementation, the apparatus further includes:

[0067] The template generation module is used to generate a target annotation data template based on the metadata of the target knowledge graph before determining the sets of replacement content included in each set of basic annotation data; wherein the structural features between entity elements, attribute elements and relation elements represented by the target annotation data template are different from the structural features between entity elements, attribute elements and relation elements represented by each set of basic annotation data.

[0068] The data generation module is used to fill the target annotation data template with each candidate data to obtain new basic annotation data.

[0069] Optionally, in one specific implementation, the apparatus further includes:

[0070] The subgraph extraction module is used to extract at least one subgraph from the target knowledge graph before determining the replacement content included in each set of basic annotation data for each basic annotation data, and to generate basic annotation data corresponding to each subgraph based on the various elements included in each subgraph.

[0071] The device further includes:

[0072] The second data acquisition module is used to acquire multiple sets of annotation data to be merged from the target annotation data generated using the basic annotation data corresponding to each sub-map; wherein each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same.

[0073] The second data merging module is used to combine the various annotation data in each group of annotation data to be merged based on the common entity elements of each annotation data in the group of annotation data to be merged, so as to obtain new target annotation data.

[0074] Optionally, in one specific implementation, the data acquisition module is specifically used for:

[0075] Obtain the preset initial annotation data;

[0076] Based on the metadata of the target knowledge graph, erroneous labeled data in each of the initial labeled data are identified, and the identified erroneous labeled data is deleted to obtain each of the basic labeled data to be expanded.

[0077] Optionally, in one specific implementation, the data acquisition module is specifically used for:

[0078] After deleting the identified erroneous labeled data and before obtaining the various basic labeled data to be expanded, for each initial labeled data remaining after deleting the identified erroneous labeled data, it is determined whether there is any unlabeled data to be labeled in the labeled data; wherein, the data to be labeled is: entity elements and / or attribute elements;

[0079] If it exists, for the data to be labeled, determine whether there is candidate data including the data to be labeled among the candidate data; otherwise, determine the initial labeled data as the basic labeled data.

[0080] If there exists candidate data that includes the element to be labeled and there is only one candidate data, then label the element type to which the candidate data belongs and the subtype under that element type for the element to be labeled.

[0081] If there are multiple candidate data that include the element to be labeled, then the candidate data with the longest length among the existing candidate data is determined, and the element type and subtype under that element type are labeled to the element to be labeled.

[0082] If no candidate data containing the element to be labeled exists, the initial labeled data is determined as the base labeled data.

[0083] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0084] Memory, used to store computer programs;

[0085] When a processor executes a program stored in memory, it implements the steps of the method for expanding any of the annotation data provided in the first aspect above.

[0086] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for expanding any of the labeled data provided in the first aspect.

[0087] Fifthly, embodiments of the present invention provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of the method for expanding any of the labeled data provided in the first aspect.

[0088] Beneficial effects of the embodiments of the present invention:

[0089] As can be seen from the above, when determining the labeled data for training the knowledge extraction model using the solution provided in this embodiment of the invention, the various basic labeled data to be expanded can be obtained first, and the preset replacement data belonging to various types of elements can be determined. Then, for each basic standard data, the sets of replacement content included in that basic standard data can be determined, thereby identifying multiple sets of target content matching each set of replacement content in each set of replacement data. Afterwards, the target data in each set of target content can be used to replace the basic data in the basic labeled data containing that set of target content that matches the target data of that set of replacement content, thus obtaining the target labeled data.

[0090] Based on this, by applying the solution provided in the embodiments of the present invention, multiple target annotation data can be obtained by replacing the basic data in the basic annotation data with preset replacement data. In this way, when obtaining annotation data for training the knowledge extraction model, the annotation data can be expanded by replacing the data in the annotation data with a small amount of manually annotated annotation data, so as to obtain a large amount of annotation data for training the knowledge extraction model. Attached Figure Description

[0091] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0092] Figure 1 A flowchart illustrating a method for expanding labeled data according to an embodiment of the present invention;

[0093] Figure 2 A schematic diagram of a sub-map;

[0094] Figure 3 A flowchart illustrating another method for expanding labeled data provided in an embodiment of the present invention;

[0095] Figure 4 A flowchart illustrating another method for expanding labeled data provided in an embodiment of the present invention;

[0096] Figure 5 for Figure 1 A flowchart illustrating a specific implementation of S101 in the middle;

[0097] Figure 6 A flowchart illustrating another method for expanding labeled data provided in an embodiment of the present invention;

[0098] Figure 7 A flowchart illustrating another method for expanding labeled data provided in an embodiment of the present invention;

[0099] Figure 8 A flowchart illustrating another method for expanding labeled data provided in an embodiment of the present invention;

[0100] Figure 9 This is a schematic diagram of the structure of a data augmentation device provided in an embodiment of the present invention;

[0101] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0102] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0103] In related technologies, the labeled data used to train knowledge extraction models is generated manually. Therefore, when a large amount of labeled data needs to be generated, it consumes a lot of time, resulting in low efficiency in data generation. Therefore, how to expand the labeled data using a smaller amount of manually labeled data to obtain a large amount of labeled data is a pressing problem that needs to be solved.

[0104] To address the aforementioned technical problems, embodiments of the present invention provide a method for expanding labeled data.

[0105] The method for expanding labeled data provided in this embodiment of the invention can be applied to any application scenario where labeled data is determined for model training. Furthermore, the method for expanding labeled data can be applied to various types of electronic devices, such as desktop computers, tablet computers, and mobile phones. The electronic device can be an independent electronic device or one or more electronic devices in a device cluster.

[0106] Therefore, this embodiment of the invention does not limit the application scenario and execution subject of the method for expanding the labeled data. Hereinafter, the execution subject of the method for expanding the labeled data is simply referred to as an electronic device.

[0107] One embodiment of the present invention provides a method for expanding labeled data, which may include the following steps:

[0108] Obtain the basic annotation data to be expanded and determine the preset replacement data belonging to each type of element; wherein, each replacement data is labeled with its element type and the subtype under that element type;

[0109] For each set of basic labeled data, determine the replacement content included in the basic labeled data; wherein, each set of replacement content includes: at least one set of basic data in the relation subtype to which the entity element, attribute element and relation element to be replaced in the labeled data belongs, and the replacement content of each set is not completely the same;

[0110] For each set of replacement content, multiple sets of target content that match the set of replacement content are determined in each set of replacement data; wherein, each set of target content includes each set of target data that matches each set of basic data in the set of replacement content, each set of target data and the matching basic data belong to the same type of element, the sets of target content are not completely the same, and each set of target content is completely different from the set of replacement content.

[0111] For each set of target content, the target data is replaced by the basic data in the basic annotation data that belongs to the replacement content and matches the target data, using the target data in the target content.

[0112] As can be seen from the above, when determining the labeled data for training the knowledge extraction model using the solution provided in this embodiment of the invention, the various basic labeled data to be expanded can be obtained first, and the preset replacement data belonging to various types of elements can be determined. Then, for each basic standard data, the sets of replacement content included in that basic standard data can be determined, thereby identifying multiple sets of target content matching each set of replacement content in each set of replacement data. Afterwards, the target data in each set of target content can be used to replace the basic data in the basic labeled data containing that set of target content that matches the target data of that set of replacement content, thus obtaining the target labeled data.

[0113] Based on this, by applying the solution provided in the embodiments of the present invention, multiple target annotation data can be obtained by replacing the basic data in the basic annotation data with preset replacement data. In this way, when obtaining annotation data for training the knowledge extraction model, the annotation data can be expanded by replacing the data in the annotation data with a small amount of manually annotated annotation data, so as to obtain a large amount of annotation data for training the knowledge extraction model.

[0114] The following description, in conjunction with the accompanying drawings, details a method for expanding labeled data according to an embodiment of the present invention.

[0115] Figure 1 This is a flowchart illustrating a method for expanding labeled data according to an embodiment of the present invention, as shown below. Figure 1 As shown, the expansion method may include the following steps:

[0116] S101: Obtain the basic annotation data to be expanded and determine the preset replacement data belonging to each type of element;

[0117] Each replacement data item is labeled with its element type and its subtypes.

[0118] When expanding the labeled data, we can first obtain the basic labeled data to be expanded and determine the preset replacement data belonging to each type of element. Each replacement data can replace a certain data in a basic labeled data. After the replacement is completed, new labeled data can be obtained. This new labeled data is the result obtained after expanding the basic labeled data.

[0119] In this process, after a replacement is completed, the resulting labeled data is designated as the new labeled data. This can be considered as generating new labeled data based on the replaced base labeled data, while the replaced base labeled data still exists. Therefore, this base labeled data can be replaced again to obtain new labeled data once more. In this way, multiple replacements can be performed on the same base labeled data to obtain multiple new labeled data sets, and the original base labeled data still exists.

[0120] For example, if the basic annotation data A is replaced M times, then after the M replacements are completed, the existing annotation data includes: M new annotation data and the basic annotation data A.

[0121] Optionally, the various basic annotation data to be expanded can be obtained through manual annotation.

[0122] Optionally, the basic annotation data to be expanded can be the annotation data corresponding to subgraphs in a preset knowledge graph. That is, subgraphs can be extracted from the preset knowledge graph. Then, for each extracted subgraph, based on at least one type of element among entity elements, attribute elements, and relation elements included in the subgraph, the data content and annotation content of the annotation data corresponding to the subgraph are determined, and the annotation data corresponding to the subgraph is obtained as the basic annotation data.

[0123] For example, sub-maps such as Figure 2 As shown, the subgraph includes entity elements Lao Zhang and Xiao Zhang, and relation elements parent and child. The data content of the obtained labeled data is: Lao Zhang and Xiao Zhang are parent and child, and the label content is: [Lao Zhang] and [Xiao Zhang] are labeled as entity [person], and [Lao Zhang] and [Xiao Zhang] are labeled as relation [parent and child].

[0124] Furthermore, the preset replacement data belonging to various types of elements can include at least one of the following: replacement data belonging to entity elements, replacement data belonging to attribute elements, and replacement data belonging to relation elements. For example, it may only include replacement data belonging to entity elements, replacement data belonging to attribute elements, or replacement data belonging to relation elements; or it may include replacement data belonging to entity elements, replacement data belonging to attribute elements, and replacement data belonging to relation elements; or it may include any two of the following: replacement data belonging to entity elements, replacement data belonging to attribute elements, and replacement data belonging to relation elements.

[0125] Furthermore, to ensure that the annotation content in the new labeled data matches the data content after replacement, if the replacement data used for replacement belongs to the same subtype under the same element type as the replaced data, then there is no need to change the annotation content of the basic annotation data. Otherwise, it is necessary to use the element type and subtypes under that element type of the replacement data to replace the annotation content of the basic annotation data related to the replaced data. Based on this, each replacement data can be labeled with its element type and subtypes under that element type.

[0126] Among these, each of the above-mentioned entity elements, attribute elements, and relation elements can have multiple different subtypes.

[0127] For example, people and companies are both entity elements, but they belong to different subtypes of entity elements; age and gender are both attribute elements, but they belong to different subtypes of attribute elements; parent-child relationship and spouse relationship are both relationship elements, but they belong to different subtypes of relationship elements.

[0128] It should be emphasized that the examples above are merely illustrative of different subtypes under each element type, and not limitations.

[0129] Optionally, the preset replacement data for each type of element can be manually set.

[0130] Optionally, the preset replacement data for each type of element can be extracted from the preset target text according to the attribute category.

[0131] Therefore, the embodiments of the present invention do not specifically limit the methods for obtaining the above-mentioned basic annotation data or the methods for determining the above-mentioned replacement data.

[0132] S102: For each set of basic annotation data, determine the replacement content included in that set of basic annotation data;

[0133] Each set of replacement content includes at least one basic data from the relation subtype to which the entity element, attribute element, and relation element to be replaced in the labeled data belongs, and the replacement content of each set is not exactly the same;

[0134] For each basic annotation data, when performing data replacement, it is reasonable to replace only one data in the basic annotation data, replace multiple data belonging to the same type of element in the basic annotation data, or replace multiple data belonging to multiple types of element in the basic annotation data.

[0135] Furthermore, for each set of basic labeled data, in some replacement processes, one piece of data in the basic labeled data can be replaced; in other replacement processes, multiple pieces of data belonging to the same category in the basic labeled data can be replaced; and in still other replacement processes, multiple pieces of data belonging to multiple categories in the basic labeled data can be replaced. Even in the replacement method where only one piece of data in the basic labeled data is replaced, the data replaced in each replacement can be different.

[0136] For example, [Xiaoming] and [Xiaohong], whose data is labeled as [22 years old], are married. [22 years old] is labeled as the attribute [age], [Xiaoming] and [Xiaohong] are labeled as entities [people], and the relationship between [Xiaoming] and [Xiaohong] is labeled as the relationship [couple]. In the process of multiple replacements, [Xiaoming] can be replaced, [Xiaohong] can be replaced, [Xiaoming] and [Xiaohong] can be replaced at the same time, and [22 years old] can also be replaced, etc.

[0137] In other words, for each basic labeled data, multiple replacement methods can be used to replace the data in the basic labeled data at least once to obtain multiple new labeled data.

[0138] Based on this, for each basic annotation data, the replacement content of each group of basic annotation data is determined according to the replacement method of the basic annotation data, and the replacement content of each group is not completely the same.

[0139] Each set of replacement content may include: the basic data to be replaced in the basic annotation data, and the basic data may include: at least one basic data of the relation subtype to which the entity element, attribute element and relation element to be replaced in the basic annotation data belongs.

[0140] In other words, each set of replacement content includes: at least one basic data from the relation subtype to which the entity element, attribute element, and relation element to be replaced in the basic annotation data belongs.

[0141] Optionally, for each set of basic annotation data, one set of replacement content included in the basic annotation data can be determined, or multiple sets of replacement content included in the basic annotation data can be determined.

[0142] For example, [Xiaoming] and [Xiaohong], whose data is labeled as [22 years old], are married. [22 years old] is labeled as the attribute [age], [Xiaoming] and [Xiaohong] are labeled as entities [people], and the relationship between [Xiaoming] and [Xiaohong] is labeled as the relationship [couple]. Three sets of replacement content can be determined. The first set of replacement content includes [Xiaoming] and [Xiaohong], the second set of replacement content includes [Xiaoming], [Xiaohong], and [22 years old], and the third set of replacement content includes [Xiaoming], [Xiaohong], [22 years old], and [couple], etc.

[0143] S103: For each group of replacement content, determine multiple groups of target content that match the group of replacement content in each replacement data;

[0144] Each set of target content includes target data that matches each basic data in the set of replacement content. Each target data and its matching basic data belong to the same type of element. The target content in each set is not completely the same, and each set of target content is completely different from the set of replacement content.

[0145] During the replacement process, in order to ensure that the structural features of entity elements, attribute elements and relation elements in the basic annotation data and the new annotation data remain unchanged before and after the replacement, the replacement data used and the data being replaced should belong to the same type of element when replacing the data in the basic replacement data. Furthermore, when replacing the basic data included in each group of replacement content, it is necessary to use the replacement data to replace all the basic data included in that group of replacement content in the basic annotation data.

[0146] Based on this, for each set of replacement content, target data that matches each basic data in that set of replacement content can be identified within the replacement data. Furthermore, each target data belongs to the same element category as its matching basic data, thus yielding the target content that matches that set of replacement content. The obtained target content is completely different from the original set of replacement content.

[0147] Furthermore, each set of replacement content can represent a replacement method for the underlying labeled data containing that set of replacement content. Therefore, this replacement method can be used to replace the underlying labeled content multiple times, and each replacement will not be exactly the same.

[0148] Based on this, for each set of replacement content, multiple sets of target content that match that set of replacement content can be identified in each set of replacement data, and the target content in each set is not exactly the same.

[0149] Optionally, the number of target content groups determined for each group of replacement content can be set by technicians according to the needs of model training, or it can be determined based on the number of elements belonging to each category included in each replacement data, or it can be random. This embodiment of the invention does not specifically limit this.

[0150] S104: For each group of target content, use the target data in the target content of that group to replace the basic data in the basic annotation data that belongs to the replacement content of that group and matches the target data, and obtain the target annotation data.

[0151] After obtaining multiple sets of target content that match each set of replacement content, the target data in each set of target content can be used to replace the basic annotation data where the replacement content is located, so as to obtain new annotation data, that is, to obtain target annotation data.

[0152] For each group of target content, the basic annotation data containing the replacement content can be determined first. Then, for each target data in the group, basic data belonging to the replacement content and matching the target data can be identified from the determined basic annotation data. This target data can then replace the identified basic data. After all target data in the group has been replaced, the new annotation data, i.e., the target annotation data, can be obtained.

[0153] For each set of target content, after the target data is an entity element or attribute element, if the subtype of the target element is different from the subtype of the replaced base data after data replacement, the annotation content of the replaced base data in the annotation content of the base annotation data needs to be updated simultaneously. Thus, the updated annotation content is used as the annotation content of the target annotation data obtained after replacement, so as to ensure the accuracy of the annotation content of the target annotation data.

[0154] For example, the data content of a certain basic labeled data is: [22 years old] [Xiaoming] and [Xiaohong] went on a trip. The labeled content is: [22 years old] is labeled as the attribute [age], [Xiaoming] and [Xiaohong] are labeled as entities [people], and the relationship between [Xiaoming] and [Xiaohong] is labeled as the relationship [couple].

[0155] The first set of replacement content includes [Xiaoming] and [Xiaohong], and the target content that matches the first set of replacement content is [Xiaohei] and [Xiaolan]. After the replacement, the data content of the target labeled data is: [22 years old] [Xiaohei] and [Xiaolan] go on a trip. The labeled content is: [22 years old] is labeled as the attribute [age], [Xiaohei] and [Xiaolan] are labeled as entities [people], and the relationship between [Xiaohei] and [Xiaolan] is labeled as the relationship [couple].

[0156] The second set of replacement content includes [Xiaoming], [Xiaohong], and [22 years old]. The target content that matches the second set of replacement content is [Xiaohei], [Xiaolan], and [30 years old]. After the replacement, the data content of the target labeled data is: [30 years old] [Xiaohei] and [Xiaolan] go on a trip. The labeled content is: [30 years old] is labeled as the attribute [age], [Xiaohei] and [Xiaolan] are labeled as entities [people], and the relationship between [Xiaohei] and [Xiaolan] is labeled as the relationship [couple].

[0157] The third set of replacement content includes [Xiaoming], [Xiaohong], [22 years old], and [couple]. The target content that matches the second set of replacement content is [Zhang Da], [Zhang Er], [18 years old], and [brothers]. After the replacement, the data content of the target labeled data is: [18 years old] [Zhang Da] and [Zhang Er] go on a trip. The labeled content is: [18 years old] is labeled as the attribute [age], [Zhang Da] and [Zhang Er] are labeled as entities [people], and the relationship between [Zhang Da] and [Zhang Er] is labeled as the relationship [brothers].

[0158] Optionally, after obtaining the labeled data for each target, one can directly use the obtained labeled data for model training, or one can use both the basic labeled data and the labeled data for each target for model training, both of which are reasonable.

[0159] Based on this, by applying the solution provided in the embodiments of the present invention, multiple target annotation data can be obtained by replacing the basic data in the basic annotation data with preset replacement data. In this way, when obtaining annotation data for training the knowledge extraction model, the annotation data can be expanded by replacing the data in the annotation data with a small amount of manually annotated annotation data, so as to obtain a large amount of annotation data for training the knowledge extraction model.

[0160] Optionally, in one specific implementation, for each group of replacement content, among the target content groups that match that group of replacement content, each target data and the matching basic data can belong to the same subtype under the same type of element, or belong to different subtypes under the same type of element.

[0161] Among them, target content that belongs to the same subtype of the same element under the same category of the included target data and the matching basic data is called the first type of content, and target content that belongs to different subtypes under the same element under the included target data and the matching basic data is called the second type of content.

[0162] Furthermore, for each set of replacement content, the multiple sets of target content that match the set of replacement content can include only multiple sets of first-class content, or only multiple sets of second-class content, or simultaneously include at least one set of first-class content and at least one set of second-class content.

[0163] Based on this, in this embodiment of the invention, step S104 may include the following two cases:

[0164] The first scenario: For each group of target content, if the target content belongs to the first category, then in the target annotation data obtained after the replacement, only the data content of the replaced entity elements and / or attribute elements is changed, while the annotation content of the replaced entity elements and / or attribute elements remains unchanged. That is, there is no need to change the annotation content of the basic annotation data. In this way, the obtained target annotation data can be regarded as positive annotation data.

[0165] The second scenario: For each group of target content, if the target content belongs to the second category, then in the new labeled data obtained after the replacement, not only are the data contents of the replaced entity elements and / or attribute elements changed, but the labeled content of the replaced entity elements and / or attribute elements is also changed. Therefore, the labeled content of the basic labeled data needs to be changed, and the changed labeled content is used as the labeled content of the new labeled data obtained after the replacement. In this way, the obtained target labeled data can be regarded as negative labeled data.

[0166] In the second case mentioned above, for negatively labeled data, the data content may be: not frequently appearing in the user's usual language context, expressing ambiguous semantics, or expressing semantics that are invalid, etc., which do not conform to the user's usual language habits. However, the content included in its labeled content is correct, that is, the element type and subtype under the element type of each data in the data content included in its labeled content are correct.

[0167] For example, the basic labeled data content is "[Company xx was founded in 2000]", and the label content is "[Company xx] is labeled as entity [Company]". The replaced negative labeled data content is "[Xiaoming was founded in 2000]", and the label content is "[Xiaoming] is labeled as entity [Person]". It can be seen that although the negative labeled data content "[Xiaoming was founded in 2000]" does not conform to the user's usual language habits, the element type and subtypes under the element type of "[Xiaoming]" in the label content of the negative labeled data are correct.

[0168] For example, the basic labeled data content is "[Xiaoming is 18 years old this year]", and the label content "[18 years old]" is labeled as the attribute "[age]". The data content of the resulting negative labeled data is "[Xiaoming is male this year]", and the label content "[male]" is labeled as the attribute "[gender]". It can be seen that although the data content "[Xiaoming is male this year]" of the negative labeled data does not conform to the user's usual language habits, the element type and subtypes under the element type of the label content "[male]" in the negative labeled data are correct.

[0169] For example, the basic labeled data content is "[Xiaoming and Xiaohong got married]", and the label content is that the relationship between "[Xiaoming]" and "[Xiaohong]" is labeled as "[husband and wife]". The replaced negative labeled data content is "[Xiaoming and xx company got married]", and the label content is that the relationship between "[Xiaoming]" and "[xx company]" is labeled as "[no relationship]". It can be seen that although the data content of the negative labeled data "[Xiaoming and xx company got married]" does not conform to the user's usual language habits, the element type and subtype of the relationship between "[Xiaoming]" and "[xx company]" in the label content of the negative labeled data are correct.

[0170] Based on this, negative labeled data can be understood as: labeled data in which there are errors in the data content, but the element types and subtypes under the element types of each data in the labeled content are correct.

[0171] Furthermore, since the element types and subtypes of each data element in the negative annotation data are correct, negative annotation data cannot be equated with erroneous annotation data.

[0172] In this way, training the knowledge extraction model using negatively labeled data can enable the obtained knowledge extraction model to correctly identify the element type and subtype of each data in the erroneous data content, thereby improving the accuracy of the obtained knowledge extraction model.

[0173] For multiple labeled data sets, if they contain the same entity element, they can be associated through this shared entity element. This allows them to be merged into a single labeled data set, which can then be used as the new target labeled data set. The merged labeled data set includes the aforementioned shared entity element.

[0174] Based on this, in one optional implementation, such as Figure 3 As shown, the method for expanding labeled data provided in this embodiment of the invention may further include the following steps S105-S106:

[0175] S105: Obtain multiple sets of annotation data to be merged from each basic annotation data and / or each target annotation data obtained;

[0176] Each group of labeled data to be merged includes multiple labeled data with the same entity elements, and the labeled data to be merged in each group are not completely the same.

[0177] S106: Based on the common entity elements of each annotation data in each group of annotation data to be merged, combine the annotation data in the group of annotation data to be merged to obtain new target annotation data.

[0178] Based on this, in this specific implementation, after obtaining the target annotation data in step S104 above, multiple annotation data with the same entity elements can be obtained from the various basic annotation data and / or the obtained target annotation data, thereby obtaining a set of annotation data to be merged.

[0179] Furthermore, in order to expand the various basic annotation data to obtain more target annotation data, multiple sets of annotation data to be merged can be obtained from the various basic annotation data and / or the various target annotation data obtained.

[0180] Furthermore, for each group of labeled data to be merged, the labeled data in the group can be combined based on the common entity elements of each labeled data in the group to be merged, to obtain new target labeled data.

[0181] At this point, the target annotation data obtained includes the target annotation data obtained in step S104 and the target annotation data obtained in step S106. That is, the target annotation data obtained at this point includes two types of target annotation data: one is the target annotation data obtained by data replacement in step S104, and the other is the target annotation data obtained by merging annotation data in step S106.

[0182] in, Figure 3 The specific implementation shown is merely an example of the method for expanding the annotation data including the above steps S105 and S106, and is not a limitation. Based on other specific implementations provided in this embodiment of the invention, the above steps S105 and S106 can be added to further expand the annotation data.

[0183] In many cases, new labeled data can be obtained by augmenting existing labeled data.

[0184] Based on this, in one optional implementation, such as Figure 4 As shown, the method for expanding labeled data provided in this embodiment of the invention may further include the following steps S107-S108:

[0185] S107: Select multiple annotation data to be processed from each basic annotation data and / or each target annotation data obtained;

[0186] S108: Using a preset data augmentation method, generate at least one augmented annotation data corresponding to each annotation data to be processed, as a new target annotation data;

[0187] Data augmentation methods include at least one of back translation, random insertion, and random deletion.

[0188] In this specific implementation, after obtaining the target annotation data through data replacement in step S104, a preset data augmentation method can be used to enhance the annotation data of each basic annotation data and / or each obtained target annotation data, so as to obtain new annotation data after enhancement, which serves as the new target annotation data.

[0189] In this way, multiple annotation data to be processed can be selected from each basic annotation data and / or each target annotation data obtained. Then, at least one augmented annotation data corresponding to each annotation data to be processed can be generated using a preset data augmentation method, which can then be used as new target annotation data.

[0190] For each piece of labeled data to be processed, one can use only one preset data augmentation method to generate augmented labeled data corresponding to the labeled data to be processed, which will serve as new target labeled data; or multiple preset data augmentation methods can be used to generate multiple augmented labeled data corresponding to the labeled data to be processed, which will serve as multiple new target labeled data.

[0191] The aforementioned data augmentation methods include at least one of back translation, random insertion, and random deletion.

[0192] Back-translation refers to translating the labeled data from the original language into another language, and then translating it back into the original language. The labeled data obtained after these two translations is the new labeled data. Optionally, if the labeled data to be processed is the target labeled data obtained through data substitution in step S104 above, then during back-translation, it can be ensured that the target data in the target content used for data substitution in this labeled data remains unchanged.

[0193] Random insertion refers to adding stop words randomly at any position in the labeled data to obtain new labeled data. Optionally, if the labeled data to be processed is the target labeled data obtained through data replacement in step S104 above, then when performing random insertion, it can be ensured that the target data in the target content used for data replacement in the labeled data remains unchanged.

[0194] "Deletion at any time" means randomly deleting words at any position in the labeled data to obtain new labeled data after deletion. Optionally, if the labeled data to be processed is the target labeled data obtained through data replacement in step S104 above, then when performing random deletion, it can be ensured that the target data in the target content used for data replacement in the labeled data remains unchanged.

[0195] Optionally, in the above Figure 4 Based on the specific implementation shown, the target annotation data obtained includes the target annotation data obtained in step S104 and the target annotation data obtained in step S108. That is, the target annotation data obtained includes two types of target annotation data: one is the target annotation data obtained by data replacement in step S104, and the other is the target annotation data obtained by data augmentation in step S108.

[0196] Optionally, in the above Figure 3 and Figure 4 Based on the specific implementation shown, the target annotation data obtained includes the target annotation data obtained in step S104, the target annotation data obtained in step S106, and the target annotation data obtained in step S108. That is, the target annotation data obtained includes three types of target annotation data: the first type is the target annotation data obtained by data replacement in step S104; the second type is the target annotation data obtained by merging annotation data in step S106; and the third type is the target annotation data obtained by data augmentation processing in step S108.

[0197] in, Figure 4The specific implementation shown is merely an example of the method for expanding the labeled data including the above steps S107 and S108, and is not a limitation. Based on other specific implementations provided in this embodiment of the invention, the above steps S107 and S108 can be added to further expand the labeled data.

[0198] Since existing knowledge graphs can include entity elements, attribute elements, and relation elements, and the structure of existing knowledge graphs can represent the structural features of real entity elements, attribute elements, and relation elements, it is possible to obtain various replacement data belonging to each type of element from existing knowledge graphs.

[0199] Alternatively, in one specific implementation, such as Figure 5 As shown, step S101 above, which determines the preset replacement data belonging to various types of elements, may include the following steps:

[0200] S1011: Obtain each candidate data belonging to each type of element in the preset target knowledge graph, and calculate the number of times each candidate data appears in each basic labeled data.

[0201] S1012: For each piece of data to be replaced in each set of basic labeled data, calculate the similarity between the piece of data to be replaced and each of the corresponding specified data.

[0202] Among them, each specified data corresponding to each data to be replaced belongs to the same subtype under the same element category as the data to be replaced;

[0203] S1013: Based on the calculated occurrence counts and similarities, determine the candidate data corresponding to each data to be replaced, and label the element type and subtype of each candidate data to be replaced, thus obtaining the replacement data.

[0204] In this specific implementation, when determining the replacement data belonging to each type of element, the target knowledge graph can be determined first, and then the candidate data belonging to each type of element appearing in the target knowledge graph can be obtained.

[0205] This can involve obtaining partial candidate data belonging to various types of elements in the target knowledge graph; obtaining all candidate data belonging to various types of elements in the target knowledge graph; or obtaining partial candidate data belonging to a certain type or several types of elements in the target knowledge graph, and obtaining all candidate data belonging to other types of elements in the target knowledge graph. All of these are reasonable.

[0206] Furthermore, when retrieving candidate data belonging to a certain category of elements from the target knowledge graph, the number of candidate data retrieved can be pre-set, or it can be retrieved from all candidate data belonging to that category of elements appearing in the target knowledge graph according to a specified proportion. In this case, it can be retrieved according to specified rules, such as the frequency of occurrence, or it can be retrieved at any time; both are reasonable. Moreover, the aforementioned specified proportion can be pre-set or randomized; both are reasonable.

[0207] After obtaining each candidate data, we can calculate the number of times each candidate data appears in each basic labeled data.

[0208] Furthermore, for each piece of data to be replaced in the various basic labeled data, data belonging to the same subtype under the same element category as the data to be replaced can be identified from the acquired candidate data and used as the corresponding designated data. In this way, the similarity between the data to be replaced and the corresponding designated data can be calculated.

[0209] Then, based on the calculated occurrence counts and similarities, we can determine the candidate data corresponding to each data to be replaced, and label each candidate data with its element type and subtype, thus obtaining each replacement data.

[0210] Optionally, each piece of data to be replaced in each basic annotation data can be a portion of the data belonging to each type of element in each basic annotation data; it can also be all the data belonging to each type of element in each basic annotation data; or it can be a portion of the data belonging to a certain type of element or several types of elements, or all the data belonging to other types of elements. All of these are reasonable.

[0211] Furthermore, when retrieving a portion of data belonging to a certain type of element from various basic annotation data as replacement data, the quantity of replacement data can be pre-set, or it can be retrieved from all data belonging to that type of element appearing in the various basic annotation data according to a specified proportion. In this case, it can be retrieved according to specified rules, such as the frequency of occurrence, or it can be retrieved at any time; both are reasonable. Moreover, the aforementioned specified proportion can be pre-set or randomized; both are reasonable.

[0212] Furthermore, in this specific implementation, the basic data in each group of replacement content determined in step S102 can be the data to be replaced in step S1012.

[0213] Optionally, in one specific implementation, step S1013 above, which determines each candidate data corresponding to each data to be replaced based on the calculated occurrence counts and similarities, may include the following steps 11-12:

[0214] Step 11: For each piece of data to be replaced, arrange the specified data corresponding to that piece of data according to the preset arrangement.

[0215] The preset arrangement methods include: if the similarity is different, they are arranged in order of increasing similarity; if the similarity is the same, they are arranged in order of increasing frequency of occurrence.

[0216] Step 12: For each piece of data to be replaced, extract the specified data that precedes the specified position from the sorted specified data, and use them as the candidate data corresponding to each piece of data to be replaced.

[0217] In this specific implementation, for each piece of data to be replaced, the specified data corresponding to the data to be replaced can be arranged in ascending order of similarity between the data to be replaced and the corresponding specified data. If there are multiple specified data with the same similarity, they are arranged in ascending order of frequency of occurrence.

[0218] For example, the similarity between the data to be replaced B and the corresponding specified data 1-5 is 0.9, 0.8, 0.5, 0.5 and 0.3 respectively, and the occurrence counts of the specified data 1-5 are 1, 3, 2, 0 and 1 respectively; then the resulting arrangement is: data 5, data 4, data 3, data 2 and data 1.

[0219] In this way, for each piece of data to be replaced, we can obtain the specified data that precedes the specified position in the sorted data, and thus obtain the candidate data corresponding to the data to be replaced.

[0220] In this process, the number of candidate data corresponding to each piece of data to be replaced is the same. Furthermore, the number of candidate data corresponding to each piece of data to be replaced, i.e., the specified position mentioned above, can be preset, which is reasonable.

[0221] In addition, in one optional implementation, when determining candidate data corresponding to each piece of data to be replaced, the data can be obtained directly according to the relationship between the similarity between the data to be replaced and each of the corresponding specified data and a preset similarity threshold. For example, among the specified data corresponding to the data to be replaced, those specified data whose similarity to the data to be replaced is less than the preset similarity threshold are obtained as candidate data corresponding to the data to be replaced. Therefore, in this specific implementation, the number of candidate data corresponding to each piece of data to be replaced can be the same, completely different, or partially the same.

[0222] Knowledge graphs possess metadata, which characterizes the structural features between entity elements, attribute elements, and relation elements within the knowledge graph. For example, the entity elements at both ends of the relation element "[couple]" have subtypes of "[person]" and "[person]", and "[father]" is a unique subtype of the relation element. Similarly, the annotations in each labeled data set characterize the structural features between various elements within that data set. Therefore, if a certain structural feature in the knowledge graph differs from all the structural features represented by the labeled data, then the labeled data whose structural features match those in the knowledge graph will not be used to train the model. Thus, to improve the richness of the labeled data used for model training and enhance the accuracy of the trained model, labeled data whose structural features match those in the knowledge graph can be constructed based on the data representing various elements within the knowledge graph.

[0223] Based on this, in one optional implementation, such as Figure 6 As shown, the method for expanding labeled data provided in this embodiment of the invention may further include the following steps S100A-S1001B:

[0224] S100A: Generates target annotation data templates based on the metadata of the target knowledge graph;

[0225] The structural features between entity elements, attribute elements, and relation elements represented by the target annotation data template are different from the structural features between entity elements, attribute elements, and relation elements represented by each basic annotation data.

[0226] S100B: Fill the target annotation data template with each candidate data to obtain new basic annotation data.

[0227] In this specific implementation, since the metadata of the target knowledge graph can represent subtypes under various element types appearing in the target knowledge graph, the structural features between entity elements, attribute elements, and relation elements in the target knowledge graph can be determined based on the metadata. Furthermore, the structural features between entity elements, attribute elements, and relation elements represented by each basic annotation data can be determined based on the annotation content of each basic annotation data. Subsequently, structural features that differ from the structural features between entity elements, attribute elements, and relation elements represented by each basic annotation data can be identified within the structural features of entity elements, attribute elements, and relation elements in the target knowledge graph. Based on these identified structural features, a target annotation data template is generated.

[0228] Then, based on the subtype to which the various elements to be filled in the target annotation data template belong, data that can be filled into the target annotation data template can be selected from the candidate data of each element in the target knowledge graph. Then, the selected data is used to fill the target annotation data template to obtain new annotation data. Then, the new annotation data can be used as the basic annotation data.

[0229] For example, the metadata of the target knowledge graph is: entity type [person], multiple attribute types [name, age, gender], relationship type [couple], subject type [person] in the relationship type, and object type [person] in the relationship type;

[0230] Thus, a target-labeled data template can be generated: the attribute of [person] is [gender], and [person] and [person] are a married couple.

[0231] Furthermore, if candidate data [Xiaoming: Gender: Male] and [Xiaoming: Couple: Xiaohong] exist in the target knowledge graph, then the basic labeled data can be obtained:

[0232] Basic annotation data 1: The data content is that the gender of [Xiaoming] is [male]. The annotation content is that [Xiaoming] is labeled as the entity [person], and [male] is marked as the attribute [gender].

[0233] Basic labeled data 2: The data content is that [Xiaoming] and [Xiaohong] are a couple. The label content is that [Xiaoming] and [Xiaohong] are marked as entities [people], and the relationship between [Xiaoming] and [Xiaohong] is marked as relationship [couple].

[0234] in, Figure 6The specific implementation shown is merely an example of the method for expanding the annotation data including the above steps S100A and S1001B, and is not a limitation. Based on other specific implementations provided in this embodiment of the invention, the above steps S100A and S1001B can be added to further expand the annotation data.

[0235] A knowledge graph comprises multiple subgraphs, each of which can include any number of data types, such as entity elements, attribute elements, and relation elements. Therefore, based on each subgraph, a corresponding labeled data can be determined. For example, ... Figure 2 The subgraph shown corresponds to the data content: Lao Zhang and Xiao Zhang are father and son, and the annotation content is: [Lao Zhang] and [Xiao Zhang] are labeled as entities [person], and [Lao Zhang] and [Xiao Zhang] are labeled as the relationship [father and son]. Thus, when obtaining the basic annotation data to be expanded, subgraphs can be extracted from the knowledge graph, and the annotation data corresponding to the extracted subgraphs can be used as the basic annotation data to be expanded.

[0236] Based on this, in one optional implementation, step S101 above, obtaining the various basic annotation data to be expanded, may include the following step 21:

[0237] Step 21: Extract at least one sub-graph from the target knowledge graph, and generate basic annotation data corresponding to each sub-graph based on the various elements included in each sub-graph.

[0238] Furthermore, based on the acquisition of some basic annotation data to be expanded through other means, such as manual annotation, additional basic annotation data to be expanded can be generated by utilizing subgraphs in the knowledge graph.

[0239] Based on this, in one optional implementation, such as Figure 7 As shown, the method for expanding labeled data provided in this embodiment of the invention may further include the following step S100C:

[0240] S100C: Extract at least one sub-graph from the target knowledge graph, and generate basic annotation data for each sub-image based on the various elements included in each sub-graph.

[0241] Furthermore, in the above Figure 6 and Figure 7 Based on the specific implementation shown, such as Figure 8 As shown, the method for expanding labeled data provided in this embodiment of the invention may further include the following steps S100A-S100C.

[0242] S100A: Generates target annotation data templates based on the metadata of the target knowledge graph;

[0243] The structural features between entity elements, attribute elements, and relation elements represented by the target annotation data template are different from the structural features between entity elements, attribute elements, and relation elements represented by each basic annotation data.

[0244] S100B: Fill the target annotation data template with each candidate data to obtain new basic annotation data;

[0245] S100C: Extract at least one sub-graph from the target knowledge graph, and generate basic annotation data for each sub-image based on the various elements included in each sub-graph.

[0246] In this specific implementation, the execution order of the above steps S100A and S100C is not limited.

[0247] Optionally, in the above Figure 7 and Figure 8 Based on the specific implementation shown, the method for expanding labeled data provided in this embodiment of the invention may further include the following steps 31-32:

[0248] Step 31: Obtain multiple sets of annotation data to be merged from the target annotation data generated using the basic annotation data corresponding to each sub-map;

[0249] Each group of labeled data to be merged includes multiple labeled data with the same entity elements, and the labeled data to be merged in each group are not completely the same.

[0250] Step 32: Based on the common entity elements in each set of labeled data to be merged, combine the labeled data in the set of labeled data to be merged to obtain new target labeled data.

[0251] Among them, the specific implementation methods of steps 31-32 above are the same as those described above. Figure 3 The specific implementation methods shown in steps S105-S106 are similar and will not be repeated here.

[0252] Optionally, in this specific implementation, in the above... Figure 1 Based on the specific implementation shown, the obtained target annotation data includes the target annotation data obtained in step S104 and the target annotation data obtained in step 32. That is, the obtained target annotation data includes two types of target annotation data: one is the target annotation data obtained by data replacement in step S104, and the other is the target annotation data obtained by merging annotation data in step 32.

[0253] Optionally, in this specific implementation, in the above... Figure 4 Based on the specific implementation shown, the obtained target annotation data includes the target annotation data obtained in step S104, the target annotation data obtained in step S108, and the target annotation data obtained in step 32. That is, the obtained target annotation data includes three types of target annotation data: the first type is the target annotation data obtained by data replacement in step S104, the second type is the target annotation data obtained by data augmentation in step S108, and the third type is the target annotation data obtained by merging annotation data in step 32.

[0254] Optionally, in this specific implementation, in the above... Figure 3 Based on the specific implementation shown, the target annotation data obtained includes the target annotation data obtained in step S104, the target annotation data obtained in step S106, and the target annotation data obtained in step 32. That is, the target annotation data obtained includes three types of target annotation data: the first type is the target annotation data obtained by data replacement in step S104; the second type is the target annotation data obtained by merging annotation data in step S106; and the third type is the target annotation data obtained by merging annotation data in step 32.

[0255] Optionally, in this specific implementation, in the above... Figure 3 and Figure 4 Based on the specific implementation shown, the obtained target annotation data includes the target annotation data obtained in step S104, step S106, step S108, and step 32. That is, the obtained target annotation data includes four types of target annotation data: the first type is the target annotation data obtained by data replacement in step S104; the second type is the target annotation data obtained by merging annotation data in step S106; the third type is the target annotation data obtained by data augmentation processing in step S108; and the fourth type is the target annotation data obtained by merging annotation data in step 32.

[0256] When obtaining labeled data through methods other than knowledge graphs, such as manual annotation, errors may exist in the labeled content. For example, the relationship between two entity elements might be incorrectly labeled, or the subtypes of two entity elements with a specified relationship might be incorrect. Therefore, to ensure the accuracy of the final target labeled data and thus the accuracy of the trained model, it is necessary to ensure the accuracy of the basic labeled data used for expansion.

[0257] Based on this, in one optional implementation, step S101 above, obtaining the various basic annotation data to be expanded, includes the following steps 41-42:

[0258] Step 41: Obtain the preset initial annotation data;

[0259] Step 42: Based on the metadata of the target knowledge graph, identify the erroneous labeled data in each initial labeled data, delete the identified erroneous labeled data, and obtain the basic labeled data to be expanded.

[0260] In this specific implementation, the preset initial annotation data is first obtained, and then the erroneous annotation data in each initial annotation data can be determined based on the metadata of the target knowledge graph.

[0261] Thus, after deleting the erroneous annotation data identified in each initial annotation data, the remaining initial annotation data are the basic annotation data to be expanded.

[0262] Furthermore, in some basic annotation data that needs to be expanded, there may be data belonging to entity elements or attribute elements that have not been annotated. Therefore, these unannotated data can be annotated to enrich the annotation content of the basic annotation data.

[0263] Based on this, optionally, in a specific implementation, after deleting the determined erroneous annotation data in step 42 above, and before obtaining the various basic annotation data to be expanded in step 42 above, step 42 may further include the following steps 421-425:

[0264] Step 421: For each initial annotation data remaining after deleting the identified erroneous annotation data, determine whether there is any unannotated data to be annotated in the annotation data; if there is, proceed to step 422; otherwise, determine the initial annotation data as the basic annotation data.

[0265] The data to be labeled includes entity elements and / or attribute elements.

[0266] Step 422: For the data to be labeled, determine whether there is candidate data that includes the element to be labeled among the candidate data; if there is, and the number of existing candidate data is 1, proceed to step 423; if there is, and the number of existing candidate data is multiple, proceed to step 424; if there is no candidate data that includes the element to be labeled, proceed to step 425.

[0267] Step 423: Label the candidate data to be labeled with the element type and the subtypes under that element type;

[0268] Step 424: Determine the longest candidate data among the existing candidate data, and label the data to be labeled with the element type and subtypes of the longest candidate data.

[0269] Step 425: Determine the initial annotation data as the base annotation data.

[0270] In this specific implementation, after deleting the erroneous labeled data identified in each initial labeled data, for each remaining initial labeled data, it can be first determined whether there is any unlabeled data to be labeled in the labeled data.

[0271] If there are no unlabeled data to be labeled in the labeled data, the initial labeled data can be directly determined as the basic labeled data;

[0272] If there are unlabeled data to be labeled in the labeled data, it can be further determined whether there are candidate data including the aforementioned data to be labeled among the various candidate data appearing in the target knowledge graph.

[0273] If there exists candidate data containing the element to be labeled, and there is only one candidate data, then the element type and subtypes under that element type can be labeled for the candidate data to be labeled. In this way, the labeled initial labeled data can be determined as the basic labeled data;

[0274] If there are multiple candidate data points containing the element to be labeled, then the longest candidate data can be identified, and the element type and subtypes of that longest candidate data can be labeled to the data to be labeled. In this way, the initially labeled data can be determined as the basic labeled data.

[0275] If there is no candidate data that includes the element to be labeled, the initial labeled data can be directly determined as the base labeled data.

[0276] Corresponding to the above-described method for expanding labeled data in the embodiments of the present invention, the present invention provides a device for expanding labeled data.

[0277] Figure 9 This is a schematic diagram of the structure of a data augmentation device provided in an embodiment of the present invention, as shown below. Figure 9 As shown, the device may include the following modules:

[0278] The data acquisition module 910 is used to acquire the basic labeled data to be expanded and to determine the preset replacement data belonging to each type of element; wherein, each replacement data is labeled with the element type to which it belongs and the subtype under that element type;

[0279] The replacement content determination module 920 is used to determine the replacement content of each set of basic annotation data for each basic annotation data; wherein, each set of replacement content includes: at least one basic data in the relation subtype to which the entity element, attribute element and relation element to be replaced in the basic annotation data belongs, and the replacement content of each set is not completely the same;

[0280] The target content determination module 930 is used to determine multiple sets of target content that match each set of replacement content in the various replacement data for each set of replacement content; wherein, each set of target content includes each set of target data that matches each set of basic data in the set of replacement content, each set of target data and the matching basic data belong to the same type of element, the sets of target content are not completely the same, and each set of target content is completely different from the set of replacement content.

[0281] The content replacement module 940 is used to replace the basic data in the basic annotation data of the corresponding replacement content in each group of target content with the basic data of the target content that belongs to the replacement content of the group and matches the target data, so as to obtain the target annotation data.

[0282] Based on this, by applying the solution provided in the embodiments of the present invention, multiple target annotation data can be obtained by replacing the basic data in the basic annotation data with preset replacement data. In this way, when obtaining annotation data for training the knowledge extraction model, the annotation data can be expanded by replacing the data in the annotation data with a small amount of manually annotated annotation data, so as to obtain a large amount of annotation data for training the knowledge extraction model.

[0283] Optionally, in one specific implementation, for each group of replacement content, the determined multiple groups of target content that match the group of replacement content include: multiple groups of first-class content and / or multiple groups of second-class content;

[0284] In each group of the first category of content, each target data and its matching basic data belong to the same subtype under the same element, while each target data and its matching basic data belong to different subtypes under the same element.

[0285] Optionally, in one specific implementation, the apparatus further includes:

[0286] The first data acquisition module is used to acquire multiple sets of annotation data to be merged from the various basic annotation data and / or the various target annotation data obtained; wherein each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same;

[0287] The first data merging module is used to combine the various annotation data in each group of annotation data to be merged based on the common entity elements of each annotation data in the group of annotation data to be merged, so as to obtain new target annotation data.

[0288] Optionally, in one specific implementation, the apparatus further includes:

[0289] The data acquisition module is used to select multiple data to be processed from the various basic annotation data and / or the various target annotation data obtained;

[0290] The data augmentation module is used to generate at least one augmented annotation data corresponding to each annotation data to be processed, as a new target annotation data, using a preset data augmentation device; wherein, the data augmentation device includes at least one of: back translation, random insertion and random deletion.

[0291] Optionally, in one specific implementation, the data acquisition module 910 includes:

[0292] The first calculation submodule is used to obtain each candidate data belonging to various types of elements that appear in the preset target knowledge graph, and to calculate the number of times each candidate data appears in the various basic annotation data.

[0293] The second calculation submodule is used to calculate the similarity between each data to be replaced and each corresponding specified data for each data to be replaced in the various basic labeled data; wherein, each specified data corresponding to each data to be replaced is data of the same subtype under the same element as the data to be replaced in the various candidate data.

[0294] The data acquisition submodule is used to determine each candidate data corresponding to each data to be replaced based on the calculated occurrence frequency and similarity, and to label each candidate data with the element type to which the candidate data belongs and the subtype under the element type, so as to obtain each replacement data.

[0295] Optionally, in one specific implementation, the data acquisition submodule is specifically used for:

[0296] For each piece of data to be replaced, arrange the specified data corresponding to the data to be replaced according to a preset arrangement method; wherein, the preset arrangement method includes: if the similarity is different, arrange them in order of increasing similarity and if the similarity is the same, arrange them in order of increasing frequency of occurrence.

[0297] For each piece of data to be replaced, retrieve the specified data that precedes the specified position from the sorted specified data, and use them as candidate data corresponding to each piece of data to be replaced.

[0298] Optionally, in one specific implementation, the apparatus further includes:

[0299] The template generation module is used to generate a target annotation data template based on the metadata of the target knowledge graph before determining the sets of replacement content included in each set of basic annotation data; wherein the structural features between entity elements, attribute elements and relation elements represented by the target annotation data template are different from the structural features between entity elements, attribute elements and relation elements represented by each set of basic annotation data.

[0300] The data generation module is used to fill the target annotation data template with each candidate data to obtain new basic annotation data.

[0301] Optionally, in one specific implementation, the apparatus further includes:

[0302] The subgraph extraction module is used to extract at least one subgraph from the target knowledge graph before determining the replacement content included in each set of basic annotation data for each basic annotation data, and to generate basic annotation data corresponding to each subgraph based on the various elements included in each subgraph.

[0303] The device further includes:

[0304] The second data acquisition module is used to acquire multiple sets of annotation data to be merged from the target annotation data generated using the basic annotation data corresponding to each sub-map; wherein each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same.

[0305] The second data merging module is used to combine the various annotation data in each group of annotation data to be merged based on the common entity elements of each annotation data in the group of annotation data to be merged, so as to obtain new target annotation data.

[0306] Optionally, in one specific implementation, the data acquisition module 910 is specifically used for:

[0307] Obtain the preset initial annotation data;

[0308] Based on the metadata of the target knowledge graph, erroneous labeled data in each of the initial labeled data are identified, and the identified erroneous labeled data is deleted to obtain each of the basic labeled data to be expanded.

[0309] Optionally, in one specific implementation, the data acquisition module 910 is specifically used for:

[0310] After deleting the identified erroneous labeled data and before obtaining the various basic labeled data to be expanded, for each initial labeled data remaining after deleting the identified erroneous labeled data, it is determined whether there is any unlabeled data to be labeled in the labeled data; wherein, the data to be labeled is: entity elements and / or attribute elements;

[0311] If it exists, for the data to be labeled, determine whether there is candidate data including the data to be labeled among the candidate data; otherwise, determine the initial labeled data as the basic labeled data.

[0312] If there exists candidate data that includes the element to be labeled and there is only one candidate data, then label the element type to which the candidate data belongs and the subtype under that element type for the element to be labeled.

[0313] If there are multiple candidate data that include the element to be labeled, then the candidate data with the longest length among the existing candidate data is determined, and the element type and subtype under that element type are labeled to the element to be labeled.

[0314] If no candidate data containing the element to be labeled exists, the initial labeled data is determined as the base labeled data.

[0315] Corresponding to the method for expanding labeled data provided in the above embodiments of the present invention, the present invention also provides an electronic device, such as... Figure 10As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0316] Memory 1003 is used to store computer programs;

[0317] When the processor 1001 executes the program stored in the memory 1003, it implements the steps of any of the annotation data expansion methods provided in the above embodiments of the present invention.

[0318] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0319] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0320] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0321] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0322] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the annotation data expansion methods provided in the embodiments of the present invention.

[0323] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of any of the annotation data expansion methods provided in the embodiments of the present invention.

[0324] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0325] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0326] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0327] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An augmentation method of annotating data, characterized by, The method includes: Obtain the basic annotation data to be expanded and determine the preset replacement data belonging to each type of element; wherein, each replacement data is labeled with its element type and the subtype under that element type; For each set of basic labeled data, determine the replacement content included in the basic labeled data; wherein, each set of replacement content includes: at least one set of basic data in the relation subtype to which the entity element, attribute element and relation element to be replaced in the basic labeled data belongs, and the replacement content of each set is not completely the same; For each set of replacement content, multiple sets of target content matching that set of replacement content are determined from each set of replacement data; wherein, the target content includes: multiple sets of first-class content and / or multiple sets of second-class content, each set of target content includes each set of target data that matches each set of basic data in that set of replacement content, each set of target data and the matching basic data belong to the same type of element, each set of first-class content includes each set of target data and the matching basic data belong to the same subtype under the same type of element, each set of second-class content includes each set of target data and the matching basic data belong to different subtypes under the same type of element, the target content of each set is not completely the same, and each set of target content is completely different from that set of replacement content; For each set of target content, use the target data in that set of target content to replace the basic data in the basic annotation data that belongs to the set of replacement content and matches the target data, thus obtaining the target annotation data; The method further includes: From the aforementioned basic annotation data, select multiple annotation data to be processed; Using a preset data augmentation method, at least one augmented annotation data is generated for each annotation data to be processed, serving as new target annotation data; wherein, the data augmentation method includes at least one of: back translation, random insertion, and random deletion.

2. The method of claim 1, wherein, The method further includes: Multiple sets of annotation data to be merged are obtained from the various basic annotation data and / or the various target annotation data obtained; wherein, each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same; Based on the common entity elements in each set of labeled data to be merged, the labeled data in each set of labeled data to be merged are combined to obtain new target labeled data.

3. The method of claim 1, wherein, The method further includes: Multiple annotation data to be processed are selected from the various basic annotation data and the various target annotation data obtained, or from the various target annotation data obtained. Using a preset data augmentation method, at least one augmented annotation data is generated for each annotation data to be processed, serving as new target annotation data; wherein, the data augmentation method includes at least one of: back translation, random insertion, and random deletion.

4. The method according to any one of claims 1 to 3, characterized in that, The step of determining the preset replacement data belonging to each type of element includes: Obtain candidate data belonging to various elements in the preset target knowledge graph, and calculate the number of times each candidate data appears in the various basic labeled data; For each piece of data to be replaced in the various basic labeled data, the similarity between the piece of data to be replaced and each of the corresponding specified data is calculated; wherein, each of the specified data corresponding to each piece of data to be replaced is data of the same subtype under the same element as the piece of data to be replaced in the various candidate data. Based on the calculated occurrence counts and similarities, candidate data corresponding to each data to be replaced are determined, and the element type and subtypes under the element type are labeled for each candidate data, thus obtaining each replacement data.

5. The method according to claim 4, characterized in that, The step of determining candidate data corresponding to each piece of data to be replaced based on the calculated occurrence counts and similarities includes: For each piece of data to be replaced, arrange the specified data corresponding to the data to be replaced according to a preset arrangement method; wherein, the preset arrangement method includes: if the similarity is different, arrange them in order of increasing similarity and if the similarity is the same, arrange them in order of increasing frequency of occurrence. For each piece of data to be replaced, retrieve the specified data that precedes the specified position from the sorted specified data, and use them as candidate data corresponding to each piece of data to be replaced.

6. The method according to claim 4, characterized in that, Before the step of determining the sets of replacement content included in each set of basic annotation data, the method further includes: Based on the metadata of the target knowledge graph, a target annotation data template is generated; wherein the structural features between entity elements, attribute elements and relation elements represented by the target annotation data template are different from the structural features between entity elements, attribute elements and relation elements represented by each basic annotation data. The target annotation data template is filled with each candidate data to obtain new basic annotation data.

7. The method according to claim 4, characterized in that, Before the step of determining the sets of replacement content included in each set of basic annotation data, the method further includes: From the target knowledge graph, at least one sub-graph is extracted, and based on the various elements included in each sub-graph, basic annotation data corresponding to each sub-graph is generated. The method further includes: Multiple sets of annotation data to be merged are obtained from the target annotation data generated using the basic annotation data corresponding to each sub-map; each set of annotation data to be merged includes multiple annotation data with the same entity elements, and the representation data to be merged in each set are not completely the same. Based on the common entity elements in each set of labeled data to be merged, the labeled data in each set of labeled data to be merged are combined to obtain new target labeled data.

8. The method according to claim 4, characterized in that, The steps for obtaining the various basic annotation data to be expanded include: Obtain the preset initial annotation data; Based on the metadata of the target knowledge graph, erroneous labeled data in each of the initial labeled data are identified, and the identified erroneous labeled data is deleted to obtain each of the basic labeled data to be expanded.

9. The method according to claim 8, characterized in that, After the step of deleting the identified erroneous labeled data and before the step of obtaining the various basic labeled data to be expanded, the method further includes: For each initial labeled data remaining after deleting the identified erroneous labeled data, determine whether there is any unlabeled data to be labeled in the labeled data; wherein, the data to be labeled is: entity elements and / or attribute elements; If it exists, for the data to be labeled, determine whether there is candidate data including the data to be labeled among the candidate data; otherwise, determine the initial labeled data as the basic labeled data. If there exists candidate data that includes the data to be labeled and there is only one candidate data, then label the data to be labeled with the element type to which the candidate data belongs and the subtype under that element type; If there are multiple candidate data that include the data to be labeled, then the candidate data with the longest length is determined, and the element type and subtype of the determined candidate data with the longest length are labeled for the data to be labeled. If no candidate data including the data to be labeled is found, the initial labeled data is determined as the base labeled data.

10. A device for expanding labeled data, characterized in that, The device includes: The data acquisition module is used to acquire the basic labeled data to be expanded and to determine the preset replacement data belonging to each type of element; wherein, each replacement data is labeled with its element type and the subtype under that element type; The replacement content determination module is used to determine the replacement content of each set of basic labeled data for each basic labeled data; wherein each set of replacement content includes: at least one basic data in the relation subtype to which the entity element, attribute element and relation element to be replaced in the basic labeled data belongs, and the replacement content of each set is not completely the same; The target content determination module is used to determine multiple sets of target content that match each set of replacement content from the various replacement data for each set of replacement content; wherein, the target content includes: multiple sets of first-class content and / or multiple sets of second-class content, each set of target content includes each set of target data that matches each set of basic data in the set of replacement content, each set of target data and the matching basic data belong to the same type of element, each set of first-class content includes each set of target data and the matching basic data belong to the same subtype under the same type of element, each set of second-class content includes each set of target data and the matching basic data belong to different subtypes under the same type of element, the target content of each set of target content is not completely the same, and each set of target content is completely different from the set of replacement content; The content replacement module is used to replace the basic data in the basic annotation data that belongs to the replacement content and matches the target data in each group of target content with the target data in that group of target content, so as to obtain the target annotation data. The device further includes: The data acquisition module is used to select multiple data points to be processed from the various basic labeled data. The data augmentation module is used to generate at least one augmented annotation data corresponding to each annotation data to be processed, as a new target annotation data, using a preset data augmentation device; wherein, the data augmentation device includes at least one of: back translation, random insertion and random deletion.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-9.

Images