Structured text data extraction method, device and computer equipment

Abstract

The application provides a structured text data extraction method, device and computer equipment, and belongs to the computer field.The structured text data extraction method of the application embodiment comprises the following steps: obtaining an entity stored in a first table structure in advance, wherein the first table structure comprises a plurality of attributes belonging to the same entity; converting the entity from the first table structure into a first graph structure; the first graph structure comprises nodes corresponding to each attribute in the plurality of attributes and a connection relationship between the nodes determined based on the association relationship between the attributes; and outputting the first graph structure to label a text sample based on the first graph structure; wherein the labeled text sample is used for training an extraction model, and the extraction model is used for extracting structured text data from a text to be extracted, thereby greatly simplifying the complexity of the definition process in the labeling stage.

Description

Technical Field

[0001] This application belongs to the field of computer technology, specifically relating to a method, apparatus, and computer equipment for extracting structured text data. Background Technology

[0002] The overall processing flow for extracting data from text in related technologies mainly includes: annotation stage, model training stage, and content extraction stage. In the annotation stage, training samples can be generated through methods such as manual annotation to produce the corresponding extraction model according to the annotation intent.

[0003] The annotation stage can be further divided into entity definition, relation definition, and text annotation. As you can see, the definition process is quite complex and not conducive to data analysts obtaining the data they need from the extracted text. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, and computer device for extracting structured text data. Specifically, this application is implemented through the following technical solution:

[0005] Firstly, a method for extracting structured text data is provided, the method comprising:

[0006] Obtain a predefined entity stored in a first table structure, wherein the first table structure includes several attributes belonging to the same entity;

[0007] The entity is converted from the first table structure to the first graph structure; the first graph structure includes nodes corresponding to each of the plurality of attributes and connection relationships between nodes determined based on the association relationships between the attributes;

[0008] Output the first graph structure to annotate text samples based on the first graph structure; wherein the annotated text samples are used to train an extraction model, and the extraction model is used to extract structured text data from the text to be extracted.

[0009] Optionally, the step of extracting structured text from the text to be extracted includes:

[0010] The trained extraction model extracts structured text data stored in a second graph structure from the text to be extracted.

[0011] The structured text data is converted from the second graph structure to the second table structure to obtain structured text data stored in the second table structure.

[0012] Optionally, the attributes stored in the first table structure include a preset primary attribute and a number of secondary attributes, and there is an association relationship between the primary attribute and each of the secondary attributes.

[0013] Optionally, converting the entity from the first table structure to the first graph structure includes:

[0014] Each attribute of the entity in the first table structure is converted into a node in the first graph structure corresponding to each attribute; wherein the name of the node is the name of the corresponding attribute;

[0015] For each of the subordinate attributes, in response to the association relationship between the primary attribute and the subordinate attribute, it is determined that there is a connection relationship between the first node corresponding to the primary attribute and the second node corresponding to the subordinate attribute, and a name for the connection relationship is generated according to a preset name generation rule.

[0016] Optionally, generating the name of the connection relationship according to a preset name generation rule includes:

[0017] Generate the name of the connection relationship based on the name of the attribute;

[0018] or,

[0019] The name of the connection relationship is generated based on the combination of the name of the primary attribute and the name of the secondary attribute.

[0020] Optionally, each node in the second graph structure includes attribute values ​​corresponding to the attributes; the second table structure includes fields corresponding to the attributes respectively; the step of converting the structured text data from the second graph structure to the second table structure includes:

[0021] Obtain the first attribute value of the first node corresponding to the main attribute in the second graph structure;

[0022] Store the first attribute value in the first field corresponding to the primary attribute in the second table structure; query the second attribute value of the second node that has a connection relationship with the first node in the second graph structure; wherein, the second attribute value is the attribute value of the secondary attribute corresponding to the second node;

[0023] The second attribute value is stored in the second field of the second table structure, which corresponds to the second attribute of the second node.

[0024] Optionally, the annotation of text samples based on the first graph structure includes:

[0025] In response to the user's annotation of several target attribute names for several text contents in the text sample, query the target node in the first graph structure that corresponds to the names of the several target attributes;

[0026] When target nodes with a connection relationship are found, the name of the connection relationship between the target nodes with the connection relationship is marked as the connection relationship between the corresponding target attributes.

[0027] Optionally, the structured text data stored in the second table structure is a two-dimensional table, the two-dimensional table includes information columns corresponding to each attribute among several attributes of the entity, and the primary key information column of the two-dimensional table is an information column corresponding to the primary attribute of the entity.

[0028] Secondly, another method for extracting structured text data is provided, which further includes:

[0029] Get the text to be extracted;

[0030] The extraction model extracts structured text data stored in a second graph structure from the text to be extracted. The second graph structure corresponds to a first graph structure obtained by converting entities stored in a first table structure. The first table structure includes several attributes belonging to the same entity. The first graph structure includes nodes corresponding to each attribute and connection relationships between nodes determined based on the association relationships between attributes. The second graph structure includes attribute values ​​extracted from the text to be extracted, corresponding to the attributes of each node in the first graph structure. Text samples used to train the extraction model are pre-annotated based on the first graph structure.

[0031] The structured text data is converted from the second graph structure to the second table structure to obtain structured text data stored in the second table structure.

[0032] Thirdly, a structured text data extraction device is provided, the device comprising:

[0033] A definition module is used to obtain predefined entities stored in a first table structure, wherein the first table structure includes several attributes belonging to the same entity;

[0034] A conversion module is used to convert the entity from the first table structure to a first graph structure; the first graph structure includes nodes corresponding to each of the plurality of attributes and connection relationships between nodes determined based on the association relationships between the attributes;

[0035] An extraction module is used to output the first graph structure to annotate text samples based on the first graph structure; wherein the annotated text samples are used to train an extraction model, and the extraction model is used to extract structured text data from the text to be extracted.

[0036] Optionally, the extraction module is used for:

[0037] The trained extraction model extracts structured text data stored in a second graph structure from the text to be extracted.

[0038] The structured text data is converted from the second graph structure to the second table structure to obtain structured text data stored in the second table structure.

[0039] Optionally, the attributes stored in the first table structure include a preset primary attribute and a number of secondary attributes, and there is an association relationship between the primary attribute and each of the secondary attributes.

[0040] Optionally, the conversion module is used for:

[0041] Each attribute of the entity in the first table structure is converted into a node in the first graph structure corresponding to each attribute; wherein the name of the node is the name of the corresponding attribute;

[0042] For each of the subordinate attributes, in response to the association relationship between the primary attribute and the subordinate attribute, it is determined that there is a connection relationship between the first node corresponding to the primary attribute and the second node corresponding to the subordinate attribute, and a name for the connection relationship is generated according to a preset name generation rule.

[0043] Optionally, the conversion module is used for:

[0044] Generate the name of the connection relationship based on the name of the attribute;

[0045] or,

[0046] The name of the connection relationship is generated based on the combination of the name of the primary attribute and the name of the secondary attribute.

[0047] Optionally, each node in the second graph structure includes attribute values ​​corresponding to the attributes; the second table structure includes fields corresponding to the aforementioned attributes; the extraction module is used for:

[0048] Obtain the first attribute value of the first node corresponding to the main attribute in the second graph structure;

[0049] Store the first attribute value in the first field corresponding to the primary attribute in the second table structure; query the second attribute value of the second node that has a connection relationship with the first node in the second graph structure; wherein, the second attribute value is the attribute value of the secondary attribute corresponding to the second node;

[0050] The second attribute value is stored in the second field of the second table structure, which corresponds to the second attribute of the second node.

[0051] Optionally, the extraction module is used for:

[0052] In response to the user's annotation of several target attribute names for several text contents in the text sample, query the target node in the first graph structure that corresponds to the names of the several target attributes;

[0053] When target nodes with a connection relationship are found, the name of the connection relationship between the target nodes with the connection relationship is marked as the connection relationship between the corresponding target attributes.

[0054] Optionally, the structured text data stored in the second table structure is a two-dimensional table, the two-dimensional table includes information columns corresponding to each attribute among several attributes of the entity, and the primary key information column of the two-dimensional table is an information column corresponding to the primary attribute of the entity.

[0055] Fourthly, another structured text data extraction device is provided, the device comprising:

[0056] The text input module is used to obtain the text to be extracted;

[0057] The text extraction module is used to extract structured text data stored in a second graph structure from the text to be extracted based on an extraction model. The second graph structure corresponds to a first graph structure obtained by converting entities stored in a first table structure. The first table structure includes several attributes belonging to the same entity. The first graph structure includes nodes corresponding to each attribute and connection relationships between nodes determined based on the association relationships between attributes. The second graph structure includes attribute values ​​extracted from the text to be extracted that correspond to the attributes of each node in the first graph structure. Text samples used to train the extraction model are pre-annotated based on the first graph structure.

[0058] The text conversion module is used to convert the structured text data from the second graph structure to the second table structure, so as to obtain structured text data stored in the second table structure.

[0059] Fifthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the structured text data extraction steps of the first or second aspect.

[0060] In a sixth aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the structured text data extraction steps of the first or second aspect.

[0061] This application embodiment uses a predefined entity containing several attributes stored in a first table structure. While obtaining the several attributes, it also obtains the association relationships between the attributes. Then, the entity is transformed into a first graph structure for annotating text samples, so that the nodes in the first graph structure correspond to each attribute, and the connection relationships between each node correspond to the association relationships between each attribute. It can be seen that only entity definition is required in the annotation stage, without the need to define the relationships for several attributes, reducing the steps of manually performing relationship definition and greatly simplifying the complexity of the definition process in the annotation stage. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating an exemplary embodiment of a structured text data extraction method according to this application;

[0063] Figure 2 This is a schematic diagram of a graph structure shown in an exemplary embodiment of this application;

[0064] Figure 3 This is a flowchart illustrating another structured text data extraction method according to an exemplary embodiment of this application;

[0065] Figure 4 This is a schematic diagram of graph data shown in an exemplary embodiment of this application;

[0066] Figure 5 This is a schematic diagram of another graph structure shown in an exemplary embodiment of this application;

[0067] Figure 6 This is a flowchart illustrating another structured text data extraction method according to an exemplary embodiment of this application;

[0068] Figure 7 This is a schematic diagram of the structure of a structured text data extraction device shown in an exemplary embodiment of this application;

[0069] Figure 8 This is a schematic diagram illustrating the structure of another structured text data extraction device according to an exemplary embodiment of this application;

[0070] Figure 9 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0072] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0073] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0074] In the related technologies of the annotation stage, technicians need to define entities and relationships. Entity definition refers to the process of defining several independent attributes, and relationship definition refers to the process of defining the relationship between attributes. However, defining the relationship between attributes is often a large workload and difficult to define, which leads to the complexity of the annotation process.

[0075] This application provides a new entity definition method that defines an entity with several attributes stored in a table structure. The relationships between the attributes can be obtained through the defined entity, which can replace the entity definition and relationship definition in related technologies, thereby reducing the complexity of the initial definition process.

[0076] In implementation, a predefined entity stored in a first table structure is obtained; the entity is converted from the first table structure into a first graph structure; the first graph structure includes nodes corresponding to each attribute of the entity and connection relationships between nodes determined based on the association relationships between each attribute; text samples are labeled based on the first graph structure, and the labeled text samples are used to train an extraction model, and the extraction model is used to extract structured text data from the text to be extracted.

[0077] The above technical solution, by defining entities stored in a first table structure, and based on several attributes belonging to the entity in the first table structure and the association between the attributes, can realize the above entity definition and relationship definition, thereby greatly simplifying the complexity of the definition process in the annotation stage.

[0078] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0079] Please see Figure 1 , Figure 1 This application illustrates a method for extracting structured text data, which includes the following steps.

[0080] S110. Obtain a predefined entity stored in a first table structure, wherein the first table structure includes several attributes belonging to the same entity.

[0081] Users can predefine entities in the form of a first table structure. These predefined entities can include several attributes and are also called attributed entities. For example, taking personnel information as a predefined entity, the personnel information can include attributes such as name, age, gender, and height, as shown in Table 1 below. This entity is more in line with the definition of an entity in real life, referring to objectively existing and distinguishable objects or things in the real world. It should be noted that the entities in the following embodiments are all user-defined entities.

[0082]

[0083] Table 1

[0084] The attributes contained in the first table structure are used as entities defined in the entity definition process described above, thereby completing the entity definition in the aforementioned related technologies. Alternatively, rules can be established based on preset association relationships to obtain the association relationships between the attributes contained in the first table structure, thereby completing the relationship definition in the aforementioned related technologies.

[0085] The rules for establishing the association between the attributes can be determined based on the actual settings of the first table structure. For example, any two attributes in the first table structure can be associated with each other, the association between the attributes can be determined based on the position of each attribute in the first table structure, the attributes can be linked together according to a preset association order, or one attribute can be used as the center to associate with other attributes, etc.

[0086] S120. Convert the entity from the first table structure to the first graph structure; the first graph structure includes nodes corresponding to each of the several attributes and connection relationships between nodes determined based on the association relationships between the attributes.

[0087] Since the machine learning phase in related technologies is mainly expressed in the form of graph structures, and the extraction results obtained when extracting text from the text to be extracted are also output in the form of graph structures, it is necessary to transform the entity definition from a first table structure to a first graph structure.

[0088] There are various ways to convert the first table structure into the first graph structure. For example, a pre-defined entity mapping module can be used to automatically convert user-defined entities from the first table structure into the first graph structure.

[0089] The entity mapping module can map several attributes belonging to the entity to nodes in the first graph structure, and automatically generate the connection relationships between the nodes based on the association relationships between the attributes. Taking the personnel information entity shown in Table 1 above as an example, the resulting first graph structure can be found in [reference needed]. Figure 2 The first graph structure includes nodes corresponding to name, age, gender, and height, respectively. Connection relationships are established between the node corresponding to name and the nodes corresponding to age, gender, and height: Relationship 1, Relationship 2, Relationship 3.

[0090] The names of each node in the first diagram structure can be set according to actual needs, such as the names of corresponding attributes.

[0091] In the first graph structure, the names of the connections between nodes can be obtained according to preset naming rules. For example, they can be a combination of the names of the two nodes connected by the connection, i.e., a combination of the names of the two corresponding attributes. The names of the connections can be expressed in various ways, for example, name1:name2. The name of the connection can represent both the two connected nodes and the direction of the connection. Taking name1:name2 as an example, the direction of the connection is from the node with the first name to the node with the second name.

[0092] As can be seen, unlike related technologies that require separate entity and relation definitions, this application embodiment only defines entities with attributes, eliminating the need for separate entity and relation definitions. The corresponding graph structure can be automatically mapped through the entity mapping module, greatly reducing the complexity of the definition.

[0093] The first graph structure may include one or more entities. In one implementation, when a new user-defined entity is obtained, the existing first graph structure can be updated based on the first table structure of the new entity, by adding nodes corresponding to the attributes of the new entity, or by adding connections between corresponding nodes based on the association relationships between the attributes of the new entity.

[0094] For the sake of simplicity, the following embodiments will be illustrated by taking the first diagram structure as corresponding to one entity.

[0095] S130. Output the first graph structure to annotate the text samples based on the first graph structure; wherein the annotated text samples are used to train the extraction model, and the extraction model is used to extract structured text data from the text to be extracted.

[0096] The first graph structure obtained from the transformation is output to the user, and the text samples used to train the extraction model are labeled based on the first graph structure.

[0097] There are various ways to annotate text samples, such as manual annotation, assisted manual annotation, or automatic annotation; no specific limitations are made here. For the sake of illustration, this application provides only two specific implementation methods as examples.

[0098] In one implementation, manual annotation is performed. After the converted first graph structure is output to the user, the user views the names of the attributes corresponding to each node shown in the first graph structure, and the names of the connection relationships between each node. The user annotates the text content in the text sample with the corresponding attribute names, and annotates the connection relationships between the attributes corresponding to nodes with connection relationships.

[0099] In another implementation, manual annotation is used as an aid. Users can manually annotate the text content in the text sample with the names of the corresponding target attributes based on the names of the attributes corresponding to each node shown in the first graph structure.

[0100] In response to the user's annotation of several target attribute names for several text contents in the text sample, the system automatically queries the target nodes in the first graph structure that correspond to the names of the several target attributes to determine whether there are target nodes with connection relationships.

[0101] In one implementation, after the user completes the annotation of the text sample, the names of the M target attributes corresponding to the M annotated text contents in the text sample are used to query the N target nodes in the first graph structure corresponding to the names of the M target attributes, in order to determine whether there are any target nodes with connection relationships among the N target nodes. Here, M is a positive integer greater than N.

[0102] In another embodiment, after each user completes annotating the text content, the target node corresponding to the target attribute in the first graph structure can be queried for the name of the target attribute corresponding to the currently annotated text content, so as to determine whether there is a connection relationship between the currently queried target node and the target node queried based on the previously annotated text content.

[0103] When target nodes with connectivity are found, the names of the connectivity relationships between these target nodes are labeled as the connectivity relationships between the corresponding target attributes. The labeled connectivity relationships between target attributes can also be confirmed by the user.

[0104] For example, the text sample includes labeled first text content, second text content, third text content, and fourth text content, wherein the names of the target attributes corresponding to the first text content and the third text content are as follows: Figure 2 The name shown, the name of the target attribute corresponding to the second text content is as follows: Figure 2 The age shown, the name of the target attribute corresponding to the fourth text content is as follows: Figure 2 The height is shown. The target nodes corresponding to the names of each target attribute in the first graph structure are queried, and the target nodes with connection relationships are found to be: target nodes "Name" and "Age" with relationship 1, and target nodes "Name" and "Height" with relationship 2. Based on the queried connection relationships, connection relationships are marked between the target attribute corresponding to the first text content and the target attribute corresponding to the second text content, between the target attribute corresponding to the first text content and the target attribute corresponding to the fourth text content, between the target attribute corresponding to the third text content and the target attribute corresponding to the second text content, and between the target trait corresponding to the third text content and the target attribute corresponding to the fourth text content. Users can confirm the marked connection relationships between the target attributes corresponding to the first and second text content, and between the target attributes corresponding to the third and second text content, and delete the marked connection relationships between the target attributes corresponding to the first and fourth text content, and between the target trait corresponding to the third text content and the target attribute corresponding to the fourth text content, according to the actual situation.

[0105] The extraction model is trained using machine learning or other methods on the text samples labeled with the structure of the first graph. This training can be done in a supervised or unsupervised manner.

[0106] The extraction model can be a neural network model, such as a graph neural network model.

[0107] After the extraction model has been trained, it can be called to extract text from the obtained text to obtain structured text data.

[0108] As can be seen from the technical solutions of the above embodiments, the embodiments of this application obtain the relationships between the attributes and the attributes by pre-defining entities including several attributes stored in a first table structure. Then, the entities are transformed into a first graph structure for annotating text samples, so that the nodes in the first graph structure correspond to each attribute, and the connection relationship between each node corresponds to the relationship between each attribute. It can be seen that only entity definition is required in the annotation stage, without the need to define the relationship for several attributes, which reduces the steps of manually performing relationship definition and greatly simplifies the complexity of the definition process in the annotation stage.

[0109] Figure 3 This application illustrates another method for extracting structured text data provided by an embodiment of the present application, such as... Figure 3 As shown, the method includes the following steps.

[0110] S310. Obtain a predefined entity stored in a first table structure, wherein the first table structure includes several attributes belonging to the same entity.

[0111] S320. Convert the entity from the first table structure to the first graph structure; the first graph structure includes nodes corresponding to each of the several attributes and connection relationships between nodes determined based on the association relationships between the attributes.

[0112] S330. Output the first graph structure to annotate the text samples based on the first graph structure; wherein the annotated text samples are used to train the extraction model, and the extraction model is used to extract structured text data from the text to be extracted.

[0113] Steps S310-S330 can achieve the following: Figure 1 The method embodiments of steps S110-S130 shown will not be described again here to avoid repetition.

[0114] S340. Extract structured text data stored in the second graph structure from the text to be extracted using the completed extraction model.

[0115] After annotating the text samples based on the first graph structure, the trained extraction model extracts structured text data stored in the second graph structure from the text to be extracted. The second graph structure has the same nodes and connections between nodes as the first graph structure, and can include attribute values ​​corresponding to the attributes of each node extracted from the text to be extracted.

[0116] For example, as Figure 2 Taking the first graph structure shown as an example, the structured text data extracted from the text to be extracted is stored in the second graph structure, which can be like... Figure 4 As shown, two second diagram structures with the names Zhang San and Li Si are obtained respectively.

[0117] S350. The structured text data is converted from the second graph structure to the second table structure to obtain structured text data stored in the second table structure.

[0118] Since graph-based data analysis is complex and not a standard technique for data analysts, it requires a certain learning curve. To facilitate subsequent data analysis, the structured text data needs to be transformed from the second graph structure into a second table structure.

[0119] There are various ways to convert a second graph structure into a second table structure. For example, a pre-defined data mapping module can be used to find the nodes corresponding to each attribute in the second graph structure based on the relationship between the attributes, and then import the attribute values ​​of each node into the fields corresponding to each attribute in the second table structure.

[0120] The second table structure can take many forms. In one embodiment, the structured text data stored in the second table structure is a two-dimensional table, which includes information columns corresponding to each attribute among several attributes of the entity. The attribute values ​​corresponding to each attribute are imported from the extracted second graph structure into the corresponding information columns of the two-dimensional table to obtain structured text data; wherein, the second table structure corresponding to the entity is included.

[0121] For example, taking the second diagram structure as described in Table 4 above as an example, its corresponding two-dimensional table can be shown in Table 2 below.

[0122] Name age gender height Zhang San 18 male 180 Li Si 16 female 170

[0123] Table 2

[0124] The structured text data stored in the second table structure makes it easier for subsequent data analysts to use familiar data processing methods, such as Structured Query Language (SQL), to perform data analysis, thereby improving the efficiency of data analysis.

[0125] As can be seen from the above embodiments, this application embodiment converts the second graph structure extracted from the text to be extracted into a second table structure, thereby facilitating subsequent data analysis by data analysts and greatly improving the efficiency of data analysis.

[0126] As can be seen from the above embodiments, when a predefined entity existing in a first table structure is obtained, rules can be established based on preset association relationships to obtain the association relationships of each attribute from several attributes contained in the first table structure. This application embodiment uses one specific implementation as an example for illustration.

[0127] In one implementation, the first table structure stores several attributes, including a preset primary attribute and several secondary attributes, and there is an association relationship between the primary attribute and each of the secondary attributes.

[0128] It should be noted that each entity can define one or more primary attributes, and each primary attribute has relationships with all other attributes except for that primary attribute. For simplicity, the following examples will use the example of each entity defining only one primary attribute.

[0129] For example, taking the personnel information entity shown in Table 1 as an example, if the name is taken as the main attribute of the entity, the personnel information can be shown in Table 3 below.

[0130]

[0131] Table 3

[0132] The relationships between the primary attributes and the secondary attributes in Table 3 can be shown in Table 4 below.

[0133] Relationship Name Source Attributes Purpose attribute Name: Gender Name gender Name: Age Name age Name: Height Name height

[0134] Table 4

[0135] The names of each association can be obtained according to a preset name generation rule, which is to generate the name of the association based on the name of the subordinate attribute; or, the name of the association can be generated based on the combination of the name of the primary attribute and the name of the subordinate attribute.

[0136] As shown in Table 4, the combination of the names of the primary attribute and the secondary attribute can be generated in the form of "primary attribute: secondary attribute". The primary attribute is placed at the beginning of the name as the source attribute, and the secondary attribute is placed at the end of the name as the destination attribute, thus allowing the association direction to be determined by the name.

[0137] There are various ways to convert an entity, which includes a primary attribute and multiple secondary attributes, from a first table structure to a first graph structure. One implementation includes:

[0138] Each attribute of the entity in the first table structure is converted into a node in the first graph structure corresponding to each attribute; wherein the name of the node is the name of the corresponding attribute;

[0139] For each of the subordinate attributes, in response to the association relationship between the primary attribute and the subordinate attribute, it is determined that there is a connection relationship between the first node corresponding to the primary attribute and the second node corresponding to the subordinate attribute, and a name for the connection relationship is generated according to a preset name generation rule.

[0140] The step of generating the names of the connection relationships according to preset name generation rules includes:

[0141] Generate the name of the connection relationship based on the name of the attribute;

[0142] or,

[0143] The name of the connection relationship is generated based on the combination of the name of the primary attribute and the name of the secondary attribute.

[0144] Please see Figure 5 In the first diagram structure, the node corresponding to the primary attribute "name" is the first node, and the nodes corresponding to the secondary attributes "gender," "age," and "height" are the second nodes. Based on the association between the primary attributes and each secondary attribute in Table 4, corresponding connection relationships are generated between the first node and each second node, and based on the preset name generation rules: Name: Age, Name: Gender, Name: Height.

[0145] Text samples are labeled based on the first graph structure, and the labeled text samples are used to train the extraction model; then, the trained extraction model is used to extract structured text data stored in the second graph structure from the file to be extracted; the structured text data is transformed from the second graph structure into the second table structure.

[0146] In the case where the entity as described above includes a primary attribute and several secondary attributes, the second table structure includes fields corresponding to the several attributes. The step of converting the structured text data from the second graph structure to the second table structure includes:

[0147] Obtain the first attribute value of the first node corresponding to the main attribute in the second graph structure;

[0148] Store the first attribute value in the first field corresponding to the primary attribute in the second table structure; query the second attribute value of the second node that has a connection relationship with the first node in the second graph structure; wherein, the second attribute value is the attribute value of the secondary attribute corresponding to the second node;

[0149] The second attribute value is stored in the second field of the second table structure, which corresponds to the second attribute of the second node.

[0150] When the second table structure is a two-dimensional table, the two-dimensional table includes information columns corresponding to each attribute among several attributes of the entity, and the primary key information column of the two-dimensional table is the information column corresponding to the primary attribute of the entity. The step of converting the structured text data from the second graph structure to the second table structure includes:

[0151] Obtain the first attribute value of the first node corresponding to the main attribute in the second graph structure;

[0152] Store the first attribute value in the primary key information column corresponding to the main attribute in the two-dimensional table; query the second attribute value of the second node in the second graph structure that has a connection relationship with the first node;

[0153] The second attribute value is stored in the information column of the two-dimensional table corresponding to the subordinate attribute of the second node.

[0154] As can be seen from the technical solutions of the above embodiments, the embodiments of this application define several attributes of an entity as primary attributes and several secondary attributes respectively, and there is an association relationship between the primary attributes and each secondary attribute, thereby facilitating the conversion of the entity from a first table structure to a first graph structure, and the conversion of the extracted structured text data from a second graph structure to a second table structure, so as to improve the efficiency of data analysis.

[0155] Figure 6 Another method for extracting structured text data provided in this application includes the following steps.

[0156] S610. Obtain the text to be extracted.

[0157] S620. Based on the extraction model, structured text data stored in a second graph structure is extracted from the text to be extracted. The second graph structure corresponds to a first graph structure obtained by converting entities stored in a first table structure. The first table structure includes several attributes belonging to the same entity. The first graph structure includes nodes corresponding to each of the several attributes and connection relationships between nodes determined based on the association relationships between the attributes. The second graph structure includes attribute values ​​extracted from the text to be extracted that correspond to the attributes of each node in the first graph structure. The text samples used to train the extraction model are pre-annotated based on the first graph structure.

[0158] S630. Convert the structured text data from the second graph structure to the second table structure to obtain structured text data stored in the second table structure.

[0159] Steps S610-S630 can be achieved Figures 1-5 The method embodiments shown are used to achieve the same or similar technical effects, and repeated parts will not be described again here.

[0160] As can be seen from the technical solutions of the above embodiments, the embodiments of this application obtain the text to be extracted, extract the graph data corresponding to the graph structure obtained by the conversion of the user-defined entity from the text to be extracted based on the extraction model, and convert the graph data into structured text corresponding to the entity, thereby greatly simplifying the complexity of the definition process in the annotation stage, making it easier for data analysts to perform subsequent data analysis and greatly improving the efficiency of data analysis.

[0161] Corresponding to the embodiments of the aforementioned structured text data extraction method, this application also provides embodiments of a structured text data extraction apparatus.

[0162] Figure 7 A structured text data extraction device provided in this application embodiment, such as Figure 7 As shown, the structured text data extraction device includes: a definition module 701, a conversion module 702, and an extraction module 703.

[0163] The definition module 701 is used to obtain predefined entities stored in a first table structure, the first table structure including several attributes belonging to the same entity; the conversion module 702 is used to convert the entity from the first table structure into a first graph structure; the first graph structure includes nodes corresponding to each of the several attributes and connection relationships between nodes determined based on the association relationships between the attributes; the extraction module 703 is used to output the first graph structure to annotate text samples based on the first graph structure; wherein, the annotated text samples are used to train an extraction model, the extraction model being used to extract structured text data from the text to be extracted.

[0164] As can be seen from the technical solutions of the above embodiments, the embodiments of this application can realize the above-mentioned entity definition and relationship definition by pre-defined entities including several attributes stored in a first table structure, based on several attributes belonging to the entity in the first table structure and the association relationship between the attributes, and convert the first table structure into a first graph structure, so as to annotate the text samples based on the first graph structure, thereby greatly simplifying the complexity of the definition process in the annotation stage.

[0165] Based on the above embodiments, optionally, the extraction module is used for:

[0166] The trained extraction model extracts structured text data stored in a second graph structure from the text to be extracted.

[0167] The structured text data is converted from the second graph structure to the second table structure to obtain structured text data stored in the second table structure.

[0168] As can be seen from the above embodiments, this application embodiment converts the second graph structure extracted from the text to be extracted into a second table structure, thereby facilitating subsequent data analysis by data analysts and greatly improving the efficiency of data analysis.

[0169] Based on the above embodiments, optionally, the plurality of attributes stored in the first table structure include a preset primary attribute and a plurality of secondary attributes, and there is an association relationship between the primary attribute and each of the secondary attributes.

[0170] Optionally, the conversion module is used for:

[0171] Each attribute of the entity in the first table structure is converted into a node in the first graph structure corresponding to each attribute; wherein the name of the node is the name of the corresponding attribute;

[0172] For each of the subordinate attributes, in response to the association relationship between the primary attribute and the subordinate attribute, it is determined that there is a connection relationship between the first node corresponding to the primary attribute and the second node corresponding to the subordinate attribute, and a name for the connection relationship is generated according to a preset name generation rule.

[0173] Optionally, the conversion module is used for:

[0174] Generate the name of the connection relationship based on the name of the attribute;

[0175] or,

[0176] The name of the connection relationship is generated based on the combination of the name of the primary attribute and the name of the secondary attribute.

[0177] Optionally, each node in the second graph structure includes attribute values ​​corresponding to the attributes; the second table structure includes fields corresponding to the aforementioned attributes; the extraction module is used for:

[0178] Obtain the first attribute value of the first node corresponding to the main attribute in the second graph structure;

[0179] Store the first attribute value in the first field corresponding to the primary attribute in the second table structure; query the second attribute value of the second node that has a connection relationship with the first node in the second graph structure; wherein, the second attribute value is the attribute value of the secondary attribute corresponding to the second node;

[0180] The second attribute value is stored in the second field of the second table structure, which corresponds to the second attribute of the second node.

[0181] Optionally, the extraction module is used for:

[0182] In response to the user's annotation of several target attribute names for several text contents in the text sample, query the target node in the first graph structure that corresponds to the names of the several target attributes;

[0183] When target nodes with a connection relationship are found, the name of the connection relationship between the target nodes with the connection relationship is marked as the connection relationship between the corresponding target attributes.

[0184] Optionally, the structured text data stored in the second table structure is a two-dimensional table, the two-dimensional table includes information columns corresponding to each attribute among several attributes of the entity, and the primary key information column of the two-dimensional table is an information column corresponding to the primary attribute of the entity.

[0185] As can be seen from the technical solutions of the above embodiments, the embodiments of this application define several attributes of an entity as primary attributes and several secondary attributes respectively, and there is an association relationship between the primary attributes and each secondary attribute, thereby facilitating the conversion of the entity from a first table structure to a first graph structure, and the conversion of the extracted structured text data from a second graph structure to a second table structure, so as to improve the efficiency of data analysis.

[0186] Figure 8 Another structured text data extraction device provided in the embodiments of this application, such as Figure 8 As shown, the structured text data extraction device includes: a text input module 801, a text extraction module 802, and a text conversion module 803.

[0187] The text input module 801 is used to acquire the text to be extracted; the text extraction module 802 is used to extract structured text data stored in a second graph structure from the text to be extracted based on the extraction model, the second graph structure corresponding to a first graph structure obtained by converting entities stored in a first table structure in advance; wherein, the first table structure includes several attributes belonging to the same entity; the first graph structure includes nodes corresponding to each of the several attributes and connection relationships between nodes determined based on the association relationships between the attributes; the second graph structure includes attribute values ​​extracted from the text to be extracted that correspond to the attributes of each node in the first graph structure; the text samples used to train the extraction model are pre-annotated based on the first graph structure; the text conversion module 803 is used to convert the structured text data from the second graph structure to the second table structure to obtain structured text data stored in the second table structure.

[0188] As can be seen from the technical solutions of the above embodiments, the embodiments of this application obtain the text to be extracted, extract the second graph structure of the structured text data corresponding to the first graph structure of the user-defined entity from the text to be extracted based on the extraction model, and transform the second graph structure into the second table structure of the structured text data, thereby greatly simplifying the complexity of the definition process in the annotation stage, facilitating data analysts to perform subsequent data analysis, and greatly improving the efficiency of data analysis.

[0189] The embodiments of the structured text data extraction device of this application can be applied to computer devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of the computer device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 9 The diagram shown is a hardware structure diagram of a computer device containing the structured text data extraction device of this application. Except for... Figure 9 In addition to the processor, memory, network interface, and non-volatile memory shown, the computer device in which the device is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.

[0190] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0191] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0192] This application also provides a computer-readable storage medium storing a computer program thereon. When the program is executed by a processor, it implements the steps of the above-described structured text data extraction method and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0193] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0194] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0195] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0196] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0197] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0198] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0199] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0200] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of extracting structured text data, characterized by, The method comprises: acquiring a predefined entity stored in a first table structure, the first table structure comprising a plurality of attributes belonging to the same entity; converting the entity from the first table structure into a first graph structure, the first graph structure comprising nodes corresponding to each of the plurality of attributes and connection relationships between the nodes determined based on the association relationships between the attributes; outputting the first graph structure to label a text sample based on the first graph structure; wherein the labeled text sample is used to train an extraction model, the extraction model being used to extract structured text data stored in a second graph structure from a text to be extracted, convert the structured text data from the second graph structure into a second table structure, and obtain the structured text data stored in the second table structure.

2. The method of claim 1, wherein, The plurality of attributes stored in the first table structure comprises a preset primary attribute and a plurality of secondary attributes, and the primary attribute and each of the secondary attributes have an association relationship.

3. The method of claim 2, wherein, The conversion of the entity from the first table structure into the first graph structure comprises: converting each attribute of the entity in the first table structure into a node corresponding to the attribute in the first graph structure; wherein the name of the node is the name of the corresponding attribute; for each of the secondary attributes, in response to the association relationship between the primary attribute and the secondary attribute, determining that the first node corresponding to the primary attribute and the second node corresponding to the secondary attribute have a connection relationship, and generating the name of the connection relationship according to a preset name generation rule.

4. The method of claim 3, wherein, The generation of the name of the connection relationship according to the preset name generation rule comprises: generating the name of the connection relationship according to the name of the secondary attribute; or, generating the name of the connection relationship according to the combination of the name of the primary attribute and the name of the secondary attribute.

5. The method of claim 2, wherein, Each node in the second graph structure comprises an attribute value of a corresponding attribute: the second table structure comprises fields corresponding to the plurality of attributes respectively; The conversion of the structured text data from the second graph structure into the second table structure comprises: acquiring a first attribute value of a first node corresponding to the primary attribute in the second graph structure; storing the first attribute value to a first field corresponding to the primary attribute in the second table structure; querying a second attribute value of a second node having a connection relationship with the first node in the second graph structure; wherein the second attribute value is an attribute value of a secondary attribute corresponding to the second node; storing the second attribute value to a second field corresponding to the secondary attribute corresponding to the second node in the second table structure.

6. The method of claim 1, wherein, The labeling of the text sample based on the first graph structure comprises: in response to a user labeling a plurality of target attribute names corresponding to a plurality of text contents in a text sample, querying target nodes corresponding to the plurality of target attribute names in the first graph structure; in the case of querying target nodes having a connection relationship, labeling the name of the connection relationship between the target nodes having the connection relationship as the connection relationship between the corresponding target attributes.

7. The method of claim 1, wherein, The structured text data stored in the second table structure is a two-dimensional table, and the two-dimensional table includes information columns corresponding to respective attributes of the entity; and a primary key information column of the two-dimensional table is an information column corresponding to a primary attribute of the entity.

8. A method of structured text data extraction, characterized by, The method further includes: obtaining text to be extracted; extracting, based on an extraction model, structured text data stored in a second graph structure from the text to be extracted, the second graph structure corresponding to a first graph structure converted from a first table structure in which an entity is pre-defined; wherein the first table structure includes a plurality of attributes belonging to the same entity; the first graph structure includes nodes corresponding to respective attributes and connection relationships between the nodes determined based on association relationships between the respective attributes; the second graph structure includes attribute values of the respective attributes corresponding to the nodes in the first graph structure extracted from the text to be extracted; and text samples used to train the extraction model are pre-labeled based on the first graph structure; converting the structured text data from the second graph structure to a second table structure to obtain the structured text data stored in the second table structure.

9. A structured text data extraction apparatus characterized by comprising: The apparatus includes: a defining module configured to obtain an entity pre-defined and stored in a first table structure, the first table structure including a plurality of attributes belonging to the same entity; a converting module configured to convert the entity from the first table structure to a first graph structure, the first graph structure including nodes corresponding to respective attributes and connection relationships between the nodes determined based on association relationships between the respective attributes; an extracting module configured to output the first graph structure to label text samples based on the first graph structure; wherein the labeled text samples are used to train an extraction model, and the extraction model is used to extract structured text data stored in a second graph structure from text to be extracted, convert the structured text data from the second graph structure to a second table structure, and obtain the structured text data stored in the second table structure.

10. The apparatus of claim 9, wherein, The plurality of attributes stored in the first table structure include a preset primary attribute and a plurality of secondary attributes, and the primary attribute has an association relationship with each of the secondary attributes.

11. The apparatus of claim 10, wherein, The converting module is configured to: convert each attribute of the entity in the first table structure to a node corresponding to the attribute in the first graph structure; wherein a name of the node is a name of the corresponding attribute; for each of the secondary attributes, in response to the association relationship between the primary attribute and the secondary attribute, determine that a first node corresponding to the primary attribute and a second node corresponding to the secondary attribute have a connection relationship, and generate a name of the connection relationship according to a preset name generation rule.

12. The apparatus of claim 11, wherein, The converting module is configured to: generate the name of the connection relationship according to the name of the secondary attribute; or, generate the name of the connection relationship according to a combination of the name of the primary attribute and the name of the secondary attribute.

13. The apparatus of claim 10, wherein, Each node in the second graph structure includes an attribute value of a corresponding attribute; the second table structure includes fields corresponding to the plurality of attributes respectively; and the extracting module is configured to: obtaining a first attribute value of a first node corresponding to the main attribute in the second graph structure; storing the first attribute value to a first field corresponding to the main attribute in the second table structure; querying a second attribute value of a second node having a connection relationship with the first node in the second graph structure; the second attribute value is an attribute value of a slave attribute corresponding to the second node; storing the second attribute value to a second field corresponding to the slave attribute corresponding to the second node in the second table structure.

14. The apparatus of claim 9, wherein, The extraction module is configured to: in response to a user labeling a name of a plurality of target attributes corresponding to a plurality of text content in a text sample, querying a target node corresponding to the name of the plurality of target attributes in the first graph structure; in a case where the target nodes having a connection relationship are queried, labeling a name of a connection relationship between the target nodes having the connection relationship as a connection relationship between corresponding target attributes.

15. The apparatus of claim 9, wherein, The second table structure stores structured text data as a two-dimensional table, the two-dimensional table includes information columns corresponding to each attribute of a plurality of attributes of the entity, and a primary key information column of the two-dimensional table is an information column corresponding to a main attribute of the entity.

16. A structured text data extraction apparatus characterized by comprising: The apparatus comprises: a text input module configured to obtain a text to be extracted; a text extraction module configured to extract, based on an extraction model, structured text data stored in a second graph structure from the text to be extracted, the second graph structure corresponding to a first graph structure converted from a first table structure pre-defined and storing an entity; the first table structure includes a plurality of attributes belonging to the same entity; the first graph structure includes nodes corresponding to each attribute of the plurality of attributes and connection relationships between the nodes determined based on association relationships between the attributes; the second graph structure includes attribute values of attributes corresponding to each node in the first graph structure extracted from the text to be extracted; a text sample used for training the extraction model is pre-labeled based on the first graph structure; a text conversion module configured to convert the structured text data from the second graph structure to a second table structure to obtain structured text data stored in the second table structure.

17. A computer readable storage medium having stored thereon a computer program, characterized in that The program, when executed by a processor, implements the steps of the method of any one of claims 1-8.

18. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the program, implements the steps of the method of any one of claims 1-8. The processor, when executing the program, implements the steps of the method of any one of claims 1-8.

Images