A text processing method and electronic device
By segmenting text into fragments and encoding relative positions, entities and objects are identified, and triples are generated. This solves the problems of entity nesting and relationship overlap in knowledge graph construction and improves the accuracy of triple generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2023-02-28
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies suffer from entity nesting and relationship overlap during knowledge graph construction, leading to incomplete and inaccurate triple extraction.
By segmenting the text into fragments, identifying entities and objects, and using relative position encoding and pre-trained models to generate triples, entity nesting and relationship overlap are avoided.
It achieves entity extraction at the text fragment level, solves the entity nesting problem, and avoids interference between subject and object during relation extraction, thereby improving the accuracy of triple generation.
Smart Images
Figure CN115964516B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of knowledge graph technology, and in particular to a text processing method and an electronic device. Background Technology
[0002] Entities are the basic units of knowledge graphs and important information carriers in text. Entity extraction is an important and fundamental task in natural language processing, and a subtask of information extraction, aiming to extract predefined entity information from text data. Relation extraction, on the other hand, identifies the relationships between entities. Therefore, these two tasks are complementary and closely related, and a joint model is typically used to extract entities and relations, forming triples (s, p, o), where s, p, and o represent the entity subject, predicate or relation, and entity object, respectively.
[0003] In addition, entities may be nested, meaning one entity is completely contained within another longer entity; relationships may overlap, such as single entity overlap or entity pair overlap, which can also lead to the inability to extract triples completely and accurately. Summary of the Invention
[0004] This application provides a text processing method and electronic device to avoid entity nesting and relationship overlap when analyzing text to generate corresponding triples during the knowledge graph construction process.
[0005] This application provides a text processing method, including:
[0006] The text to be processed is obtained, and the text to be processed is divided into segments to obtain at least one segment; and at least one first segment is identified from the at least one segment, wherein the first segment is the entity body;
[0007] For each of the first segments, based on the remaining segments other than the first segment in the at least one segment, a second segment and a relation word corresponding to the first segment are determined, wherein the second segment is an entity object, and the relation word is used to represent the relationship between the first segment and the second segment;
[0008] For each first segment that has a corresponding second segment and relation word, generate a triple using the first segment, the corresponding second segment, and the relation word.
[0009] This method divides the text to be processed into segments, obtaining at least one segment. From these segments, at least one first segment is identified, where the first segment is the entity subject. For each first segment, based on the remaining segments in the at least one segment (excluding the first segment itself), a second segment and a relation word corresponding to that first segment are determined. For each first segment with a corresponding second segment and relation word, a triplet is generated using the first segment, its corresponding second segment, and the relation word. This achieves entity extraction on a segment-by-segment basis. Since the text segments are independent, the problem of entity nesting is effectively solved. Furthermore, because relations and objects are extracted based on their corresponding subjects, the situation where an entity is both a subject and an object does not interfere with each other. Therefore, the problems of entity nesting and relation overlap are avoided.
[0010] In some implementations, identifying the first segment from the at least one segment includes:
[0011] For each of the at least one segment:
[0012] Determine the vector of the segment;
[0013] Based on the vector of the segment, determine whether the segment is the first segment.
[0014] In some implementations, the vector of the segment includes the encoded vector of the first character and the encoded vector of the last character of the segment.
[0015] In some implementations, determining the second segment and relational term corresponding to each of the first segments based on the remaining segments other than the first segment in the at least one segment includes:
[0016] For each of the first segments:
[0017] Determine the relative positional encoding between the first segment and each of the remaining segments in the text to be processed;
[0018] Generate at least one hidden vector, wherein each hidden vector includes: a vector of the first segment, a vector of one remaining segment, and a relative positional encoding between the first segment and the remaining segment;
[0019] Using the at least one hidden vector, determine whether there is a relationship between the first segment and the remaining segments, take the remaining segments that are related to the first segment as the second segment, and determine the relation word corresponding to the relationship type between the second segment and the first segment.
[0020] In some implementations, the step of identifying at least one first segment from the at least one segment, and the step of determining, for each first segment, a second segment and relation word corresponding to the first segment based on the remaining segments other than the first segment in the at least one segment, are performed using a pre-trained word encoding model;
[0021] The method further includes:
[0022] Calculate a first loss function and a second loss function; the first loss function is the loss function corresponding to when the word encoding model identifies at least one first segment from the at least one segment; the second loss function is the loss function corresponding to when the word encoding model determines the second segment and relation word corresponding to each first segment based on the remaining segments other than the first segment in the at least one segment.
[0023] The total loss function of the word encoding model is calculated using the first loss function and the second loss function.
[0024] In some embodiments, the method further includes:
[0025] The parameters of the word encoding model are updated while minimizing the total loss function.
[0026] In some implementations, different segments contain different texts, and the positional relationship between different texts in each segment containing multiple texts is continuous.
[0027] Another embodiment of this application provides an electronic device including a memory and a processor, wherein the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory and execute any of the methods described above according to the obtained program.
[0028] Furthermore, according to embodiments, for example, a computer program product for a computer is provided, which includes software code portions that, when the product is run on the computer, perform the steps of the methods defined above. The computer program product may include a computer-readable medium on which the software code portions are stored. Furthermore, the computer program product may be directly loaded into the computer's internal memory and / or sent via a network through at least one of an upload process, a download process, and a push process.
[0029] Another embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for causing the computer to perform any of the methods described above. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 This is a schematic flowchart illustrating a text processing method provided in an embodiment of this application.
[0032] Figure 2 A flowchart illustrating the process of determining the object that has a relationship with the subject and determining the specific relational terms, as provided in the embodiments of this application;
[0033] Figure 3 A schematic diagram of the overall flow of a text processing method provided in an embodiment of this application;
[0034] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;
[0035] Figure 5 This is a schematic diagram of the structure of a text processing device provided in an embodiment of this application. Detailed Implementation
[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0037] This application provides a text processing method and electronic device to avoid entity nesting and relationship overlap when analyzing text to generate corresponding triples during the knowledge graph construction process.
[0038] The text processing methods and electronic devices in the embodiments of this application are based on the same concept. Since the methods and devices solve problems in similar ways, the implementation of the devices and methods can refer to each other, and repeated parts will not be described again.
[0039] The terms "first," "second," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0040] The following examples and embodiments are to be understood as illustrative only. While this specification may refer to "a," "an," or "some" examples or embodiments in several places, this does not mean that every such reference relates to the same example or embodiment, nor does it mean that the feature applies only to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. Furthermore, terms such as "comprising" and "including" should be understood not to limit the described embodiments to consisting only of those features mentioned; such examples and embodiments may also include features, structures, units, modules, etc., not specifically mentioned.
[0041] The various embodiments of this application will now be described in detail with reference to the accompanying drawings. It should be noted that the order in which the embodiments are presented in this application represents only a chronological order and does not represent the superiority or inferiority of the technical solutions provided by the embodiments.
[0042] Technical terminology used in the embodiments of this application:
[0043] Text fragments: The objects processed in this embodiment are sentence texts, which need to be segmented into fragments, i.e., enumerating all subsequences of a sentence of length n. A sentence of length n has a total of This design transforms the sequence labeling problem into an independent classification problem, which can address situations where one entity is completely contained within another entity.
[0044] Positional encoding: To make the model more sensitive to the length and span of entities and improve the accuracy of entity recognition, relative positional encoding is added to describe the relative distance between the positions of two vectors involved in the calculation during the attention process (i.e., the attention process, also known as the attention mechanism).
[0045] Multi-label classification: assigning multiple labels to a single sample. Unlike multi-class classification, in multi-label classification, each label is not mutually exclusive.
[0046] Nested entities: One entity is completely contained within another entity. For example, "a person riding an electric bike," where "electric bike" is an entity type: vehicle type; and "electric" is an entity type: energy type. These two entities are in a containment relationship in their descriptions and are therefore called nested entities.
[0047] Relationship overlap: When one or a pair of entities appear in multiple relations simultaneously, it is called relationship overlap. There are generally two types of relationship overlap:
[0048] EPO type: The subject and object are the same, and there are multiple relationships.
[0049] SEO Type: Only one entity is identical, but multiple relationships exist, such as "Zhang San, driving a BMW and running a red light." There is overlapping relationship: (Zhang San, driving, BMW), (BMW, running a red light). "BMW" appears as both the subject and object in two different relationships. That is, the same entity, "BMW," is driven by Zhang San and also ran a red light.
[0050] It should be noted that the relationship between different entities can be a word that appears in the sample (such as "driving" mentioned above), or it can be a word that does not appear in the sample but is generated by the system (such as "occurring" mentioned above).
[0051] The text processing method provided in this application embodiment includes, for example, the following:
[0052] Step 1: Perform fragment processing on the sample. That is, this step obtains the text to be processed (referred to as the sample) and divides the text into fragments to obtain at least one fragment.
[0053] In this model, a sample is a sentence. The sample is processed by segmentation, which divides the sentence sequence into all its subsequences. Each subsequence is a fragment of text, and each fragment is considered a candidate entity. Theoretically, every text fragment can potentially become an entity, and different text fragments are independent of each other. Assuming the sentence length is n, there are a total of... A text fragment.
[0054] Step 2: Use the BERT model pre-trained on a large-scale corpus to encode the text segments obtained in Step 1, representing them as vectors. That is, for each text segment, the vector corresponding to the text segment is obtained after processing by the BERT model.
[0055] Since the BERT model is based on character-level encoding for Chinese characters, in some implementations, the first and last characters of a text segment can be concatenated to represent that segment. Let the start and end positions of a text segment be (s, e), and the encoding vector of the i-th character in the BERT model be... Then the vector corresponding to the text fragment can be represented as .
[0056] Step 3: For each text fragment, determine whether it is an entity (s), that is, whether it is an entity of the subject type. In other words, this step identifies at least one first fragment from all the fragments in the sample, and the first fragment is the entity.
[0057] In some embodiments, the vector corresponding to each text fragment can be input into a preset classifier (multi-layer neural network model) for classification (which can be achieved through existing technology). The classifier can extract text fragments of category s (i.e., the text fragment is an entity).
[0058] Step 4: For each text fragment of category s (which can be abbreviated as s), extract the predicate (p) and its corresponding object (o) from all the remaining text fragments. In short, determine the po of s ("predicate" and "object" are short for "predicate and its corresponding object"). That is, determine the p and o corresponding to each s.
[0059] In some embodiments, step four is implemented using a preset word encoding model, and specific operations include, for example:
[0060] S41: Calculate the relative positional encoding between s and the remaining text segments. Let the starting position of s be... The starting position of the remaining text fragments is The relative distance is The relative distance is represented as a fixed-length vector using a trainable embedding matrix, which serves as the encoding of the relative position.
[0061] Here, the trainable embedding matrix is a set of randomly initialized vectors, which is part of the model parameters, and is represented as follows: , where n represents the number of vectors and d represents the vector dimension.
[0062] Relative distance is ,take out The vectors at the corresponding positions are used as the encodings at this relative distance. Taking "Zhang San who drove a BMW through a red light" as an example, and assuming that the first position starts from 0, for example, the starting position of "drove" and all segments starting with "drove" is 0, the starting position of "driving" and all segments starting with "driving" is 1, and so on. Then, for example:
[0063] Let s be the starting position of "Zhang San" in the sample "Zhang San who drove a BMW through a red light" , and the relative position relationship between the remaining segments and "Zhang San" is Specifically as follows:
[0064] For "drove", 9 – 0 = 9, take out the vector corresponding to 9 ;
[0065] For "driving", 9 – 0 = 9, take out the vector corresponding to 9 ;
[0066] For "driving a BMW", 9 – 0 = 9, take out the vector corresponding to 9 ;
[0067] ……
[0068] For "driving", 9 – 1 = 8, take out the vector corresponding to 8 ;
[0069] For "driving a BMW", 9 – 1 = 8, take out the vector corresponding to 8 ;
[0070] ……
[0071] For "BMW", 9 – 2 = 7, take out the vector corresponding to 7 ;
[0072] ……
[0073] Among them, because the starting positions of segments such as "drove", "driving", "driving a BMW",... in the sample "Zhang San who drove a BMW through a red light" are all 0 (based on the starting position of the first character "drove"), the relative distances from "Zhang San" are equal, and the corresponding relative position encodings are also the same.
[0074] S42: For s and a certain text segment i (any one of the above-mentioned text segments i), obtain a hidden vector for relation extraction, and this hidden vector is , where is the vector representation of the entity subject s, is the vector representation of any text segment other than s, This encodes the relative position of the entity s calculated in step S41 with any text segment other than s. The hidden vector is referred to as the "hidden layer vector," where "hidden layer" refers to the intermediate layer of a neural network, and the "hidden layer vector" is the calculated result of the intermediate layer. In this embodiment, it is obtained by concatenating multiple vectors. A vector of the first segment ( ) and a vector of the remaining segments ( ), and the relative distance vector between them ( ), generate a hidden vector ( As can be seen, one main vector can correspond to multiple hidden vectors.
[0075] S43: Perform multi-label classification on the hidden vector to obtain the relationship label between s and each of the remaining text segments. In other words, this step uses the hidden vector to determine whether there is a relationship between the first segment (subject) and the other segments (candidate objects). If a relationship exists, it means that the segment with the relationship is the entity object corresponding to the entity subject. Therefore, the other segments that have a relationship with the first segment are taken as the second segment (i.e., the object), and the relation word corresponding to the relationship type between the second segment and the first segment is determined.
[0076] In some embodiments, since there may be overlapping relationships, s and a text fragment may have multiple relationships. Therefore, it is necessary to use the sigmoid activation function to calculate the probability that the hidden vector belongs to each relationship. If the probability is greater than a preset threshold, then s and the text fragment are considered to belong to this relationship.
[0077] In some embodiments, different thresholds can be preset for different types of relationships.
[0078] For example, it is initially determined that the subject "electric vehicle" and the candidate object "running a red light" have an "occurrence" relationship. To improve accuracy, the probability of the subject "electric vehicle" and the candidate object "running a red light" having an "occurrence" relationship is further calculated. If the probability is greater than the preset threshold corresponding to the "occurrence" relationship type, it is considered that the subject "electric vehicle" and the candidate object "running a red light" have an "occurrence" relationship. Finally, "running a red light" is determined to be the object corresponding to the subject "electric vehicle", and the relation word between the two is "occurrence". Subsequently, a triple (electric vehicle, occurrence, running a red light) can be generated based on this.
[0079] S44: Since the word encoding model extracts entities in a joint manner first, and then extracts entities that can form a relationship with that entity, the construction of the loss function is divided into two parts: calculating the loss for two tasks, which are the first loss function values. Second loss function value :
[0080] ;
[0081] ;
[0082] The total loss function value is:
[0083]
[0084] in This represents the probability that text fragment i is entity s; This represents a real label, with a value of 0 or 1, where 0 represents a text fragment. Not the entity s, 1 represents a text fragment. It is the entity s.
[0085] Let R represent the probability that text fragments i and s form a relation r for entity subject s, where R is the set of all relations; This represents the real label, with a value of 0 or 1. 0 indicates that text fragments i and s do not form a relationship r, while 1 indicates that text fragments i and s form a relationship r.
[0086] S45: Minimize the loss function. The word encoding model training process uses backpropagation and a moving average algorithm based on the loss function value from each forward propagation to update the word encoding model's parameters and obtain the optimal word encoding model. In other words, this step updates the word encoding model's parameters while minimizing the total loss function.
[0087] For example, this step uses the "gradient descent method," a common parameter update method for training neural network models, as shown in the formula:
[0088]
[0089] This represents the preset learning rate, i.e., the preset constant. yes For model parameters The gradient. During parameter update, the loss value... It will decrease, that is, "minimize the loss function".
[0090] Step 5: Combine the extracted entity s with its corresponding relation p and entity o into a triple. The triple is represented as (s, p, o).
[0091] For example, extracting the entity and relation p from the sample "Zhang San, who ran a red light while riding an electric bike", we obtain the following three triples according to the steps above:
[0092] Triad 1: (Zhang San, driving, electric vehicle);
[0093] Triad 2: (Electric vehicle, incident, running a red light);
[0094] Triad three: (electric vehicle, energy type, electric).
[0095] In this example, there are cases of overlapping relationships and nested entities.
[0096] The relationships mentioned overlap, meaning that "electric vehicle" can be an entity subject in one relationship (see triplet two and triplet three above) and an entity object in another relationship (see triplet one above).
[0097] The entity nesting refers to the nesting relationship between "electric vehicle" and "electric" as two entities (see the triplet 3 above).
[0098] In summary, this application embodiment extracts entities on a text fragment basis (i.e., steps one to three). Since the text fragments are independent of each other, the problem of entity nesting can be effectively solved.
[0099] This application's embodiments are applicable to all scenarios with overlapping relationships, including EPO and SEO types. Since relationships and objects are extracted based on their corresponding subjects (i.e., step four), the situation where an entity is both a subject and an object is independent; and, wherein:
[0100] When extracting relations, the connection between the entity and each candidate object is considered. The entity and each of the remaining candidate objects are concatenated into vectors, and their relative positions are encoded (i.e., steps S41 and S42 above). Fully considering pairwise relationships is beneficial for relation classification.
[0101] Since the hidden vector composed of entities and any text fragments is classified using multi-label classification (i.e., step S43 above), even if the same pair of subjects and objects have different relationships, they can still be completely extracted.
[0102] The embodiments of this application can achieve joint extraction of entities and relations, and the loss function is constructed from two parts: entity extraction and relation extraction (i.e., In the process of minimizing the loss function, the model parameters are updated (i.e., steps S44 and S45 above). Therefore, the parameter matrices of the entity and relation parts are updated simultaneously, and there is no problem of error accumulation during the propagation process.
[0103] The following example uses the input sample "Zhang San, who runs a red light while riding an electric bike" as an example, along with the attached diagram to provide a specific illustration.
[0104] The specific process is as follows: Figure 1 As shown, it includes:
[0105] Step 1: Enumerate the segments of the input samples to obtain the following segments:
[0106] ride
[0107] Electric bike
[0108] riding electric bikes
[0109] riding an electric bike
[0110] ...
[0111] electricity
[0112] electric
[0113] electric vehicles
[0114] ...
[0115] open
[0116] Zhang San three
[0118] Step 2: Encode each segment. Specifically, the first and last parts of each segment correspond to a word vector. Obtain the vector for each segment, for example:
[0119] ride
[0120] Electric bike
[0121] riding electric bikes
[0122] riding an electric bike
[0123] ...
[0124] electric vehicles
[0125] ...
[0126] Zhang San
[0127] ...
[0128] Step 3: Input the above fragment vector into a preset classifier to determine whether it is an entity s, for example:
[0129] ride Non-physical entities
[0130] Electric bike Non-physical entities
[0131] riding electric bikes Non-physical entities
[0132] riding an electric bike Non-physical entities
[0133] ...
[0134] electric vehicles It is a physical entity
[0135] ...
[0136] Zhang San It is a physical entity
[0137] ...
[0138] Thus, two entities, "electric vehicle" and "Zhang San", were extracted.
[0139] Step 4: Obtain the POs of the two entities "electric vehicle" and "Zhang San".
[0140] Taking the physical entity "electric vehicle" as an example:
[0141] use We obtain vector representations of the entity "electric vehicle" and other segments.
[0142] Use a pre-defined relationship classifier to determine the relationships between the entity "electric vehicle" and other segments. See details in [link to documentation]. Figure 2 For example, "electric vehicle" and the text fragment "running a red light" have an "occurrence" relationship, and "electric vehicle" and the text fragment "electric" have an "energy type" relationship. Therefore, the following triplet is obtained through the entity "electric vehicle":
[0143] (Electric vehicle, incident, running a red light);
[0144] (Electric vehicle, energy type, electric).
[0145] Similarly, the following triples are obtained through the entity "Zhang San":
[0146] (Zhang San, driving, electric vehicle).
[0147] After obtaining the entity, the hidden vector is used for relation extraction, as follows: Figure 2 As shown, taking the entity "electric vehicle" as an example, the process for extracting its object and relationship includes:
[0148] use The segment vector representing "electric vehicle" ;
[0149] use The vector representing any other segment i, i = 1, 2, 3, ..., i.e. It could be:
[0150] ride
[0151] Electric bike
[0152] riding electric bikes
[0153] riding an electric bike
[0154] ...
[0155] Zhang San
[0156] ...
[0157] In other words, during the entity extraction stage, the text segment encoding is formed by concatenating the BERT vector of the first character and the BERT vector of the last character of the segment.
[0158] Let p represent the relative position vector, which represents the relative position of the entity body with any other segment i.
[0159] use (By concatenating the three) we obtain the fragments and the representations formed by the fragments. A relation classifier is then used to determine the relationship between the entity "electric vehicle" and other fragments. In the relation extraction stage, the goal is to extract objects with relation labels. Therefore, the hidden vector consists of the encoding of the text fragment as the subject, the encoding of the text fragment as the candidate object, and the encoding of their relative positions. This hybrid encoding method considers the interaction between two text fragments, which is beneficial for relation extraction. For example, "electric vehicle" and the text fragment "running a red light" form an "occurrence" relationship, and "electric vehicle" and the text fragment "electric" form an "energy type" relationship. Therefore, the following triplet corresponding to "electric vehicle" is obtained:
[0160] (Electric vehicle, incident, running a red light);
[0161] (Electric vehicle, energy type, electric).
[0162] In summary, in this embodiment, candidate entities are constructed on a fragment-by-fragment basis, and entity extraction is performed using a classifier. The entire process first extracts the subject s, then obtains the relation p and the object o. When extracting po, the subject and all other fragments (i.e., candidate objects) are considered. Furthermore, the extraction of the subject s and its po uses the same word encoding model, and the loss values of the two tasks of extracting the subject s and its po are calculated, thus achieving the joint extraction of the subject s and its po.
[0163] As can be seen, see Figure 3 This application provides a text processing method, including:
[0164] S101. Obtain the text to be processed, and divide the text to be processed into segments to obtain at least one segment; and identify at least one first segment from the at least one segment, wherein the first segment is an entity (e.g., the aforementioned entity s).
[0165] S102. For each of the first segments, based on the remaining segments other than the first segment in the at least one segment, determine the second segment and relation word (e.g., po as mentioned above) corresponding to the first segment, where the second segment is an entity object (e.g., object o as mentioned above), and the relation word is used to represent the relationship between the first segment and the second segment (e.g., relation p as mentioned above).
[0166] S103. For each first segment that has a corresponding second segment and relation word, generate a triple using the first segment, the corresponding second segment, and the relation word.
[0167] The triple, for example:
[0168] (Electric vehicle, incident, ran a red light)
[0169] In some implementations, identifying the first segment from the at least one segment includes:
[0170] For each of the at least one segment:
[0171] Determine the vector of the segment; for example, encode the segment using the BERT model to obtain the vector of the segment;
[0172] Based on the vector of the segment, determine whether the segment is the first segment. Specifically, existing techniques can be used for this determination, such as inputting the vector corresponding to each text segment into a preset classifier for classification, through which text segments of category s can be extracted.
[0173] In some implementations, the vector of the segment includes the encoded vector of the first character and the encoded vector of the last character of the segment. For example, the vector corresponding to the text segment can be represented as follows: .
[0174] In some implementations, determining the second segment and relational term corresponding to each of the first segments based on the remaining segments other than the first segment in the at least one segment includes:
[0175] For each of the first segments:
[0176] Determine the relative position encoding between the first segment and each of the remaining segments in the text to be processed; the relative position encoding is, for example, a vector corresponding to the relative distance between the starting position of the first segment and the starting positions of the remaining segments;
[0177] Generate at least one hidden vector, wherein each hidden vector includes: a vector of the first segment, a vector of one remaining segment, and a relative positional encoding between the first segment and the remaining segment;
[0178] Using at least one hidden vector, it is determined whether there is a relationship between the first segment and the other segments. The other segments that are related to the first segment are designated as the second segment (i.e., the segments with relationships are entity objects), and the relation words corresponding to the relationship type between the second segment and the first segment are determined. For example, "electric vehicle" and the text segment "running a red light" have an "occurrence" relationship, and "electric vehicle" and the text segment "electric" have an "energy type" relationship.
[0179] In some implementations, the step of identifying at least one first segment from the at least one segment, and the step of determining, for each first segment, a second segment and relation word corresponding to the first segment based on the remaining segments other than the first segment in the at least one segment, are performed using a pre-trained word encoding model;
[0180] The method further includes:
[0181] Calculate the first loss function and the second loss function; the first loss function is the loss function corresponding to the word encoding model when it identifies at least one first segment from the at least one segment (e.g., the one mentioned above). The second loss function is the loss function used by the word encoding model when, for each of the first segments, based on the remaining segments other than the first segment in the at least one segment, it determines the second segment and relation word corresponding to the first segment (e.g., the one described above). );
[0182] Using the first loss function and the second loss function, the total loss function of the word encoding model is calculated, for example, as described above. .
[0183] In some embodiments, the method further includes:
[0184] The parameters of the word encoding model are updated during the process of minimizing the total loss function, for example, using the formula described above. accomplish.
[0185] In some implementations, different segments contain different texts, and the positional relationship between different texts in each segment containing multiple texts is continuous.
[0186] The technical solution provided in this application employs different hybrid encoding methods at different stages. In the entity extraction stage, the text segment encoding is formed by concatenating the BERT vectors of the first and last characters of the segment. In the relation extraction stage, the goal is to extract objects with relation labels; therefore, the hidden vector consists of the encoding of the text segment as the subject, the encoding of the text segment as the candidate object, and the encoding of their relative positions. This hybrid encoding method considers the interaction between two text segments, which is beneficial for relation extraction.
[0187] The following describes the device or apparatus provided in the embodiments of this application, and the explanations or examples of the same or corresponding technical features as those described in the above methods will not be repeated hereafter.
[0188] See Figure 4 The electronic device provided in this application embodiment includes:
[0189] Processor 600 is used to read the program from memory 620 and execute the following procedures:
[0190] The text to be processed is obtained, and the text to be processed is divided into segments to obtain at least one segment; and at least one first segment is identified from the at least one segment, wherein the first segment is the entity body;
[0191] For each of the first segments, based on the remaining segments other than the first segment in the at least one segment, a second segment and a relation word corresponding to the first segment are determined, wherein the second segment is an entity object, and the relation word is used to represent the relationship between the first segment and the second segment;
[0192] For each first segment that has a corresponding second segment and relation word, generate a triple using the first segment, the corresponding second segment, and the relation word.
[0193] In some implementations, identifying the first segment from the at least one segment includes:
[0194] For each of the at least one segment:
[0195] Determine the vector of the segment;
[0196] Based on the vector of the segment, determine whether the segment is the first segment.
[0197] In some implementations, the vector of the segment includes the encoded vector of the first character and the encoded vector of the last character of the segment.
[0198] In some implementations, determining the second segment and relational term corresponding to each of the first segments based on the remaining segments other than the first segment in the at least one segment includes:
[0199] For each of the first segments:
[0200] Determine the relative positional encoding between the first segment and each of the remaining segments in the text to be processed;
[0201] Generate at least one hidden vector, wherein each hidden vector includes: a vector of the first segment, a vector of one remaining segment, and a relative positional encoding between the first segment and the remaining segment;
[0202] Using the at least one hidden vector, determine whether there is a relationship between the first segment and the remaining segments, take the remaining segments that are related to the first segment as the second segment, and determine the relation word corresponding to the relationship type between the second segment and the first segment.
[0203] In some implementations, the step of identifying at least one first segment from the at least one segment, and the step of determining, for each first segment, a second segment and relation word corresponding to the first segment based on the remaining segments other than the first segment in the at least one segment, are performed using a pre-trained word encoding model;
[0204] Processor 600 is also used to read programs from memory 620 and execute the following procedures:
[0205] Calculate a first loss function and a second loss function; the first loss function is the loss function corresponding to when the word encoding model identifies at least one first segment from the at least one segment; the second loss function is the loss function corresponding to when the word encoding model determines the second segment and relation word corresponding to each first segment based on the remaining segments other than the first segment in the at least one segment.
[0206] The total loss function of the word encoding model is calculated using the first loss function and the second loss function.
[0207] In some embodiments, the processor 600 is also configured to read a program from the memory 620 and perform the following processes:
[0208] The parameters of the word encoding model are updated while minimizing the total loss function.
[0209] In some implementations, different segments contain different texts, and the positional relationship between different texts in each segment containing multiple texts is continuous.
[0210] In some embodiments, the electronic device further includes a transceiver 610 for receiving and transmitting data under the control of the processor 600.
[0211] The electronic device can be a terminal device or a network-side device.
[0212] Among them, Figure 4 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 600) and memory (memory 620). The bus architecture can also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 610 can be multiple elements, including a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium.
[0213] In some embodiments, a user interface 630 is also included. The user interface 630 may be an interface that can connect to external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.
[0214] The processor 600 is responsible for managing the bus architecture and general processing, while the memory 620 can store the data used by the processor 600 when performing operations.
[0215] In some embodiments, the processor 600 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a CPLD (Complex Programmable Logic Device).
[0216] See Figure 5 The text processing apparatus provided in this application includes:
[0217] The first unit 11 is used to acquire the text to be processed, and to divide the text to be processed into segments to obtain at least one segment; and to identify at least one first segment from the at least one segment, wherein the first segment is an entity body;
[0218] The second unit 12 is used to determine, for each of the first segments, a second segment and a relation word corresponding to the first segment based on the remaining segments other than the first segment in the at least one segment, wherein the second segment is an entity object and the relation word is used to represent the relationship between the first segment and the second segment;
[0219] The third unit 13 is used to generate a triplet for each first segment that has a corresponding second segment and relation word, using the first segment, the corresponding second segment, and the relation word.
[0220] In some implementations, identifying the first segment from the at least one segment includes:
[0221] For each of the at least one segment:
[0222] Determine the vector of the segment;
[0223] Based on the vector of the segment, determine whether the segment is the first segment.
[0224] In some implementations, the vector of the segment includes the encoded vector of the first character and the encoded vector of the last character of the segment.
[0225] In some implementations, determining the second segment and relational term corresponding to each of the first segments based on the remaining segments other than the first segment in the at least one segment includes:
[0226] For each of the first segments:
[0227] Determine the relative positional encoding between the first segment and each of the remaining segments in the text to be processed;
[0228] Generate at least one hidden vector, wherein each hidden vector includes: a vector of the first segment, a vector of one remaining segment, and a relative positional encoding between the first segment and the remaining segment;
[0229] Using the at least one hidden vector, determine whether there is a relationship between the first segment and the remaining segments, take the remaining segments that are related to the first segment as the second segment, and determine the relation word corresponding to the relationship type between the second segment and the first segment.
[0230] In some implementations, the step of identifying at least one first segment from the at least one segment, and the step of determining, for each first segment, a second segment and relation word corresponding to the first segment based on the remaining segments other than the first segment in the at least one segment, are performed using a pre-trained word encoding model;
[0231] Unit 12 is also used for:
[0232] Calculate a first loss function and a second loss function; the first loss function is the loss function corresponding to when the word encoding model identifies at least one first segment from the at least one segment; the second loss function is the loss function corresponding to when the word encoding model determines the second segment and relation word corresponding to each first segment based on the remaining segments other than the first segment in the at least one segment.
[0233] The total loss function of the word encoding model is calculated using the first loss function and the second loss function.
[0234] In some embodiments, the second unit 12 is further used for:
[0235] The parameters of the word encoding model are updated while minimizing the total loss function.
[0236] In some implementations, different segments contain different texts, and the positional relationship between different texts in each segment containing multiple texts is continuous.
[0237] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.
[0238] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0239] This application provides a computing device, which may specifically be a desktop computer, portable computer, smartphone, tablet computer, personal digital assistant (PDA), etc. The computing device may include a central processing unit (CPU), memory, input / output devices, etc. Input devices may include a keyboard, mouse, touchscreen, etc., and output devices may include display devices such as liquid crystal displays (LCDs) and cathode ray tubes (CRTs).
[0240] The memory may include read-only memory (ROM) and random access memory (RAM), and provides the processor with program instructions and data stored in the memory. In the embodiments of this application, the memory may be used to store the program of any of the methods provided in the embodiments of this application.
[0241] The processor executes any of the methods described in the embodiments of this application according to the program instructions stored in the memory.
[0242] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the methods described in the above embodiments. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.
[0243] This application provides a computer-readable storage medium for storing computer program instructions used in the apparatus provided in the above-described embodiments, including a program for performing any of the methods provided in the above-described embodiments. The computer-readable storage medium may be a non-transitory computer-readable medium.
[0244] The computer-readable storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).
[0245] It should be understood that:
[0246] The access technology used by entities in a communication network to transmit traffic can be any suitable current or future technology, such as WLAN (Wireless Local Access Network), WiMAX (Microwave Access Global Interoperability), LTE, LTE-A, 5G, Bluetooth, infrared, etc.; in addition, embodiments may also apply wired technologies, such as IP-based access technologies, such as wired networks or fixed lines.
[0247] An embodiment suitable for implementation as software code or as part thereof and for operation using a processor or processing function is independent of the software code and can be specified using any known or future-developed programming language, such as high-level programming languages such as Objective-C, C, C++, C#, Java, Python, Javascript, other scripting languages, etc., or low-level programming languages such as machine language or assembler.
[0248] The implementation of the embodiments is hardware-independent and can be implemented using any known or future-developed hardware technology or any combination thereof, such as microprocessors or CPUs (central processing units), MOS (metal-oxide-semiconductor), CMOS (complementary MOS), BiMOS (bipolar MOS), BiCMOS (bipolar CMOS), ECL (emitter-coupled logic), and / or TTL (transistor-transistor logic).
[0249] The embodiments may be implemented as individual devices, apparatuses, units, components or functions, or in a distributed manner. For example, one or more processors or processing functions may be used or shared in the process, or one or more processing segments or processing portions may be used and shared in the process, wherein one or more physical processors may be used to implement one or more processing portions dedicated to a particular process as described.
[0250] The device can be implemented by a semiconductor chip, a chipset, or a (hardware) module that includes such a chip or chipset.
[0251] The implementation can also be implemented as any combination of hardware and software, such as ASIC (Application-Specific IC (Integrated Circuit)) components, FPGA (Field Programmable Gate Array) or CPLD (Complex Programmable Logic Device) components or DSP (Digital Signal Processor) components.
[0252] The embodiments can also be implemented as computer program products, including a computer-usable medium in which computer-readable program code is embodied, the computer-usable program code being adapted to perform the processes described in the embodiments, wherein the computer-usable medium may be a non-transitory medium.
[0253] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0254] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0255] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0256] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0257] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A text processing method characterized by, The method includes: The text to be processed is obtained, and the text to be processed is divided into segments to obtain at least one segment; and at least one first segment is identified from the at least one segment, wherein the first segment is the entity body; For each of the first segments, based on the remaining segments other than the first segment in the at least one segment, a second segment and a relation word corresponding to the first segment are determined, wherein the second segment is an entity object, and the relation word is used to represent the relationship between the first segment and the second segment; For each first segment that has a corresponding second segment and relation word, generate a triple using the first segment, the corresponding second segment, and the relation word; Wherein, for each of the first segments, determining the second segment and relational words corresponding to the first segment based on the remaining segments other than the first segment in the at least one segment includes: For each of the first segments: Determine the relative positional encoding between the first segment and each of the remaining segments in the text to be processed; Generate at least one hidden vector, wherein each hidden vector includes: a vector of the first segment, a vector of one remaining segment, and a relative positional encoding between the first segment and the remaining segment; Perform multi-label classification on the hidden vector to obtain the relationship label between the first segment and each of the remaining text segments. Determine whether there is a relationship between the first segment and the remaining segments. Take the remaining segments that are related to the first segment as the second segment and determine the relation word corresponding to the relationship type between the second segment and the first segment.
2. The method of claim 1, wherein, Identifying the first segment from the at least one segment includes: For each of the at least one segment: Determine the vector of the segment; Based on the vector of the segment, determine whether the segment is the first segment.
3. The method of claim 2, wherein, The vector of the segment includes the encoding vector of the first character and the encoding vector of the last character of the segment.
4. The method of claim 1, wherein, The step of identifying at least one first segment from the at least one segment, and the step of determining, for each first segment, a second segment and a relation word corresponding to the first segment based on the remaining segments other than the first segment in the at least one segment, are performed using a pre-trained word encoding model; The method further includes: Calculate a first loss function and a second loss function; the first loss function is the loss function corresponding to when the word encoding model identifies at least one first segment from the at least one segment; the second loss function is the loss function corresponding to when the word encoding model determines the second segment and relation word corresponding to each first segment based on the remaining segments other than the first segment in the at least one segment. The total loss function of the word encoding model is calculated using the first loss function and the second loss function.
5. The method of claim 4, wherein, The method further includes: The parameters of the word encoding model are updated while minimizing the total loss function.
6. The method according to claim 1, characterized in that, Different segments contain different texts, and the positional relationship between different texts in each segment containing multiple texts is continuous.
7. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method according to any one of claims 1 to 6.
8. A computer program product for use in a computer, characterized in that, Includes a software code portion that, when the product is run on the computer, is used to perform the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing the computer to perform the method according to any one of claims 1 to 6.