A knowledge extraction method and system based on data enhancement and priority constraint, a terminal and a storage medium

Abstract

The application discloses a kind of knowledge extraction method, system, terminal and storage medium based on data enhancement and priority constraint, the method includes: obtaining initial text data, and the initial text data is carried out data enhancement processing, and obtains enhanced text data;Determine pre-training language model and non-autoregressive model, through pre-training language model text coding processing is carried out to enhanced text data, and through non-autoregressive model model training is carried out, and obtains knowledge extraction model;Current text data is obtained, and the current text data is carried out triple knowledge extraction processing by the knowledge extraction model, and obtains target triple knowledge extraction result.The application is processed to initial text data by data enhancement, effectively alleviate the problem of data sparsity and the diversity of triple knowledge relationship instance is insufficient, and through enhanced text data, non-autoregressive model is trained, and the knowledge extraction precision of knowledge extraction model is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a knowledge extraction method, system, terminal, and computer-readable storage medium based on data augmentation and priority constraints. Background Technology

[0002] Knowledge extraction is the foundation for building knowledge graphs, requiring the extraction of necessary triples from various types of data. Since knowledge extraction involves large datasets, it typically requires automated extraction using algorithmic models, and the accuracy of these models determines the effectiveness of the extraction.

[0003] However, due to the sparsity of data and the lack of diversity of triple knowledge relationship instances in existing technologies, the knowledge extraction accuracy of knowledge extraction models is insufficient and cannot meet users' needs for knowledge extraction. Summary of the Invention

[0004] The main objective of this invention is to provide a knowledge extraction method, system, terminal, and computer-readable storage medium based on data augmentation and priority constraints. This invention aims to address the problem that in the prior art, due to data sparsity and insufficient diversity of triple knowledge relationship instances, the knowledge extraction accuracy of the knowledge extraction model is insufficient and cannot meet the user's knowledge extraction needs.

[0005] To achieve the above objectives, the present invention provides a knowledge extraction method based on data augmentation and priority constraints, the method comprising the following steps:

[0006] Obtain initial text data and perform data augmentation processing on the initial text data to obtain augmented text data;

[0007] A pre-trained language model and a non-autoregressive model are determined. The enhanced text data is processed by text encoding using the pre-trained language model, and the model is trained using the non-autoregressive model to obtain a knowledge extraction model.

[0008] Obtain the current text data, and perform triple knowledge extraction processing on the current text data through the knowledge extraction model to obtain the target triple knowledge extraction result.

[0009] Optionally, the knowledge extraction method based on data augmentation and priority constraints, wherein obtaining initial text data and performing data augmentation processing on the initial text data to obtain augmented text data specifically includes:

[0010] Obtain triples from the initial text data and determine the relational properties of the triples.

[0011] Based on the properties of the relationship, the triplet data is augmented to obtain initial augmented text data;

[0012] The entity self-reference enhancement strategy is used to generate self-reference triples corresponding to the triple data, and the enhanced text data is obtained based on the self-reference triples and the initial enhanced text data.

[0013] Optionally, in the knowledge extraction method based on data augmentation and priority constraints, the relation properties include symmetric relations, undirected relations, and asymmetric relations.

[0014] The step of performing data augmentation processing on the triplet data based on the relational properties to obtain initial augmented text data specifically includes:

[0015] If the relation property is the symmetric relation or the undirected relation, then obtain the symmetric triplet of the triplet data;

[0016] If the relation property is the asymmetric relation, then obtain the invertible triplet of the triplet data;

[0017] The symmetric triples and the invertible triples are added to the triples of the text data to obtain the initial enhanced text data.

[0018] Optionally, the knowledge extraction method based on data augmentation and priority constraints, wherein determining a pre-trained language model and a non-autoregressive model, performing text encoding processing on the augmented text data using the pre-trained language model, and training the model using the non-autoregressive model to obtain a knowledge extraction model, specifically includes:

[0019] A pre-trained language model is determined, the enhanced text data is input into the pre-trained language model, and the enhanced text data is processed by the encoder in the pre-trained language model to obtain context embedding data.

[0020] A non-autoregressive model is determined, and dynamic attention gating is obtained by performing context feature extraction and weight dynamic adjustment processing based on the context embedding data through the non-autoregressive decoder in the non-autoregressive model;

[0021] The loss is calculated by using the fully connected layer in the non-autoregressive model based on the dynamic attention gating to obtain the subject loss and the object loss. The subject loss and the object loss are then weighted, optimized, and normalized to obtain the target loss and the initial knowledge extraction model.

[0022] The initial knowledge extraction model is iteratively trained based on the target loss and the enhanced text data. When the number of iterations reaches a preset iteration threshold, the training is completed, and the knowledge extraction model is obtained.

[0023] Optionally, the knowledge extraction method based on data augmentation and priority constraints, wherein determining the non-autoregressive model, and obtaining dynamic attention gating by performing context feature extraction and dynamic weight adjustment processing based on the context embedding data using the non-autoregressive decoder in the non-autoregressive model, specifically includes:

[0024] The context embedding data is input into the non-autoregressive decoder in the non-autoregressive model to obtain a preset number of query embedding data, and the dependency relationship between triples in the enhanced text data is calculated based on the query embedding data.

[0025] Contextual features are calculated based on the contextual embedding data and the query embedding data. The dependencies between triples in the enhanced text data and the contextual features are dynamically weighted to obtain dynamic attention gating.

[0026] Optionally, the knowledge extraction method based on data augmentation and priority constraints, wherein the step of calculating the subject loss and object loss through the fully connected layer in the non-autoregressive model according to the dynamic attention gating, specifically includes:

[0027] The dynamic attention gating is input into the fully connected layer of the non-autoregressive model, and the dynamic attention gating is mapped to the relation category space through the fully connected layer to obtain the relation prediction probability.

[0028] Calculate the predicted probability of the subject's starting position, the predicted probability of the subject's ending position, the predicted probability of the object's starting position, and the predicted probability of the object's ending position based on the predicted probability of the relationship.

[0029] The subject loss is calculated based on the predicted probability of the subject's starting position and the predicted probability of the subject's ending position, and the object loss is calculated based on the predicted probability of the object's starting position and the predicted probability of the object's ending position.

[0030] Optionally, the knowledge extraction method based on data augmentation and priority constraints, wherein the weighted optimization and normalization of the subject loss and the object loss to obtain the target loss and the initial knowledge extraction model specifically includes:

[0031] The Lagrange multiplier method with priority constraints is used to perform weighted optimization on the subject loss and the object loss to obtain the optimal loss for the subject position and the optimal loss for the object position.

[0032] A rescaling strategy is used to normalize the optimal loss at the subject position, the optimal loss at the object position, the subject loss, and the object loss to obtain the target loss. The initial training of the non-autoregressive model is then completed, resulting in the initial knowledge extraction model.

[0033] Furthermore, to achieve the above objectives, the present invention also provides a knowledge extraction system based on data augmentation and priority constraints, wherein the knowledge extraction system based on data augmentation and priority constraints includes:

[0034] The data augmentation processing module is used to acquire initial text data and perform data augmentation processing on the initial text data to obtain augmented text data;

[0035] The model training module is used to determine a pre-trained language model and a non-autoregressive model, perform text encoding processing on the enhanced text data using the pre-trained language model, and train the model using the non-autoregressive model to obtain a knowledge extraction model.

[0036] The triplet knowledge extraction module is used to acquire the current text data, and to perform triplet knowledge extraction processing on the current text data through the knowledge extraction model to obtain the target triplet knowledge extraction result.

[0037] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a knowledge extraction program based on data augmentation and priority constraints stored in the memory and executable on the processor, wherein when the knowledge extraction program based on data augmentation and priority constraints is executed by the processor, it implements the steps of the knowledge extraction method based on data augmentation and priority constraints as described above.

[0038] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a knowledge extraction program based on data augmentation and priority constraints, and the knowledge extraction program based on data augmentation and priority constraints, when executed by a processor, implements the steps of the knowledge extraction method based on data augmentation and priority constraints as described above.

[0039] In this invention, initial text data is acquired and augmented to obtain augmented text data. A pre-trained language model and a non-autoregressive model are determined. The augmented text data is then encoded using the pre-trained language model, and the non-autoregressive model is used for model training to obtain a knowledge extraction model. Current text data is acquired, and the knowledge extraction model is used to extract triplet knowledge from the current text data to obtain the target triplet knowledge extraction result. This invention effectively alleviates the problems of data sparsity and insufficient diversity of triplet data relationship instances by augmenting the initial text data. Simultaneously, by using the augmented text data to train the non-autoregressive model, the knowledge extraction accuracy of the model can be effectively improved. The resulting knowledge extraction model can accurately extract triplet data from the current text data. Attached Figure Description

[0040] Figure 1 This is a flowchart of a preferred embodiment of the knowledge extraction method based on data augmentation and priority constraints of the present invention;

[0041] Figure 2 This is a schematic diagram of the overall process of a preferred embodiment of the knowledge extraction method based on data augmentation and priority constraints of the present invention;

[0042] Figure 3 This is a structural diagram of a preferred embodiment of the knowledge extraction system based on data augmentation and priority constraints of the present invention;

[0043] Figure 4 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0045] Knowledge extraction is the foundation for building knowledge graphs, requiring the extraction of necessary triples from various types of data. Knowledge extraction involves large datasets and generally requires automated extraction through algorithmic models; the accuracy of the model determines the effectiveness of knowledge extraction. Entity-relation joint extraction refers to directly extracting triples (subject, relation, object) from text data. These triples are often distributed across different sentences, requiring the model algorithm to understand and process long texts. While there are many existing methods for entity-relation joint extraction, several problems remain: 1. Insufficient data sparsity and relation instance diversity: Existing datasets contain a limited number of entity-relation instances, failing to fully reflect the diversity of relation instances and limiting the model's generalization ability; 2. Entity-relation optimization conflicts: In the entity-relation joint extraction framework, there is a conflict between the optimization objectives of entity recognition and relation classification. Low-priority tasks (such as relation classification) can interfere with high-priority tasks (such as entity recognition), thus reducing overall performance.

[0046] To address the aforementioned issues, this invention proposes an improved joint entity-relation extraction method. This method alleviates data sparsity by employing symmetry and entity self-reference data augmentation techniques, utilizes priority constraints for joint optimization, and dynamically adjusts optimization weights using Lagrange multipliers to mitigate optimization conflicts between entity and relation identification, thereby improving the accuracy of joint entity-relation extraction.

[0047] The knowledge extraction method based on data augmentation and priority constraints described in the preferred embodiment of the present invention, such as... Figure 1 As shown, the knowledge extraction method based on data augmentation and priority constraints includes the following steps:

[0048] Step S10: Obtain initial text data and perform data augmentation processing on the initial text data to obtain augmented text data.

[0049] This invention relates to a knowledge extraction task, the purpose of which is to extract all triples (subject, relation, object) from text data. First, this invention divides the triples in the text data into three categories based on the nature and characteristics of the relations.

[0050] Specifically, the triplet data in the initial text data is obtained, and the relational properties of the triplet data are obtained; the relational properties include symmetric relations, undirected relations, and asymmetric relations.

[0051] like Figure 2As shown, after obtaining the initial text data, the present invention first needs to perform data augmentation processing on the initial text data. The data augmentation process in the present invention includes data augmentation based on relational properties and data augmentation based on entity self-reference. Among them, the data augmentation process based on relational properties is as follows: According to the property characteristics of the relation, the initial text data is divided into three categories: 1. Symmetric relation: The relation of the triple data satisfies symmetry, that is, if (e s ,r,e o The statement is true (where e is the abbreviation for entity, r is the abbreviation for relation, S is the abbreviation for subject, O is the abbreviation for object, e is the abbreviation for object). s e is the main entity in the entity. o (e is an object in an entity), then (e) o ,r,e s 1. **True Relations:** Relations r ∈ R have no directionality (where R is the set of all relations), e.g., "genre" and "sister city". For example, relations r ∈ R have a definite directionality, e.g., "follows" and "followed by". "True" means that, for example, in the triple (Guangzhou, adjacent, Guangdong), the two cities are indeed adjacent, so the new triple (Guangdong, adjacent, Guangzhou) is also true. This indicates that relation r satisfies symmetry, and therefore, the new triple can be added to the original triple data. 2. **Undirected Relations:** Relation r ∈ R has no directionality (where R is the set of all relations), e.g., "member of". 3. **Asymmetric Relations:** Relation r ∈ R has a definite directionality, e.g., "follows" and "followed by".

[0052] If the relation property is a symmetric relation or a directionless relation, then obtain the symmetric triplet of the triplet data; if the relation property is an asymmetric relation, then obtain the invertible triplet of the triplet data; add the symmetric triplet and the invertible triplet to the triplet data of the text data to obtain the initial enhanced text data.

[0053] To address the three types of relationships mentioned above, this invention proposes the following strategy: 1. If the triple (e s ,r,e o If the relation r is a symmetric relation or a non-directional relation, then a new triple (e) is added. o ,r,e s 2. If the triple (e) s ,r,e o If relation r is asymmetric and an invertible relation r′ exists in the preset relation set, then add a triple (e). o ,r′,e sAmong them, reversible relations: for example, the triple (Xiaoming, father, Xiaowang), and if the set of relations also has a reversible relation son that is the father relation, then the new triple (Xiaowang, son, Xiaoming) is also correct. This means that relation r satisfies the reversible relation, so new triples can be added to the original triple data.

[0054] The entity self-reference enhancement strategy is used to generate self-reference triples corresponding to the triple data, and the enhanced text data is obtained based on the self-reference triples and the initial enhanced text data.

[0055] Data augmentation based on entity self-reference: Entities may have some implicit semantic association with themselves. To capture this characteristic, this invention designs an entity self-reference augmentation strategy. Specifically, for each entity e... i ∈E (E is an abbreviation for entity set), generate the corresponding self-referential triple (e i ,NR,e i ), where “NR” stands for No Relation, indicating that there is no explicit semantic relationship between the entity and itself (e.g., Guangdong, no relation (NR), Guangdong), but there is implicit contextual relevance (the entity self-reference method generates this type of triple to enhance entity recognition and the diversity of relation recognition).

[0056] Step S20: Determine the pre-trained language model and the non-autoregressive model, perform text encoding processing on the enhanced text data using the pre-trained language model, and train the model using the non-autoregressive model to obtain the knowledge extraction model.

[0057] The pre-trained language model in this invention preferably adopts a model like BERT (Bidirectional Encoder Representations from Transformers, a deep learning model based on Transformers).

[0058] Specifically, a pre-trained language model is determined, the enhanced text data is input into the pre-trained language model, and the enhanced text data is processed by the encoder in the pre-trained language model to obtain context embedding data.

[0059] like Figure 2 As shown, after data augmentation of the text data, the augmented text data is input into a pre-trained language model, and the pre-trained language model performs text encoding processing on the augmented text data.

[0060] For a document D of length l (i.e., the enhanced text data in this invention), it is represented as: Where, xt This represents the word at position t. This invention uses a Pretrained Language Model (PLM) as the encoder to obtain the document's context embedding H and attention weights A.

[0061] H, A = PLM(D);

[0062] Where H∈R l×d , A∈R h×l×l d represents the encoding dimension of PLM, and h represents the number of attention heads.

[0063] like Figure 2 As shown, the context embedding data is input into the non-autoregressive decoder in the non-autoregressive model to obtain a preset number of query embedding data, and the dependency relationship between triples in the enhanced text data is calculated based on the query embedding data; context features are calculated based on the context embedding data and the query embedding data, and the dependency relationship between triples in the enhanced text data and the context features are dynamically adjusted in weight to obtain dynamic attention gating.

[0064] The non-autoregressive decoder in this invention consists of several identical Transformer blocks, each Transformer containing sub-modules such as multi-head self-attention, multi-head cross-attention, and feedforward network.

[0065] The non-autoregressive decoder processes the context embedding data as follows:

[0066] 1. Query Embedding: The non-autoregressive decoder first generates a fixed number of query embeddings Q∈R. N×d N is the number of decoding layers, and each query embedding corresponds to a potential triple. These query embeddings are generated using the following formula:

[0067] Q = Proj(Concat(Q0, H));

[0068] Where H represents the encoder output (i.e., the context embedding data in this invention), Concat(·) is the concatenation operation, and Proj(·) is the linear transformation of the conditional projection. Q0 is an initialization matrix, initialized with a Gaussian distribution that follows a variance of 0 and a mean of 1.

[0069] 2. Decoder Layer: The decoder consists of N stacked decoding layers. Each decoding layer contains the following modules: a. Self-attention module: used to capture the dependencies between triples, its calculation formula is:

[0070]

[0071] Among them, A self The output of the self-attention module represents the dependencies between triples. d represents the learnable parameters of the self-attention module. k is the scaling factor for attention, and T is the matrix transpose. b. Cross-attention module: used to integrate contextual information from the input document; its calculation formula is:

[0072]

[0073] Among them, A cross The output of the cross-attention module represents contextual information. c. Dynamic attention gating mechanism: used to dynamically adjust the weights of cross-attention and self-attention to enhance the ability to capture key information. Its gating calculation method is as follows:

[0074] A output =σ(W gate )·A cross +(1-σ(W gate ))·A self ;

[0075] Where σ is the Sigmoid activation function, W gate These are learnable parameters. The decoder output A output Used for relation classification and entity location prediction using triples.

[0076] The dynamic attention gating is input into the fully connected layer of the non-autoregressive model, and the dynamic attention gating is mapped to the relation category space through the fully connected layer to obtain the relation prediction probability. Based on the relation prediction probability, the prediction probability of the subject's starting position, the prediction probability of the subject's ending position, the prediction probability of the object's starting position, and the prediction probability of the object's ending position are calculated. The subject loss is calculated based on the subject's starting position prediction probability and the subject's ending position prediction probability, and the object loss is calculated based on the object's starting position prediction probability and the object's ending position prediction probability.

[0077] Furthermore, this invention classifies and predicts the location of enhanced text data based on the obtained dynamic attention gating: 1. Relationship classification: Mapped to the relation category space through a fully connected layer:

[0078] P R =Softmax(QW R );

[0079] in, C represents the probability of predicting the relationship. r W represents the number of relation categories. R1. Learnable parameters. 2. Entity recognition: Predicting the start and end positions of the subject and object using a positional scoring function.

[0080]

[0081] in, These represent the probability predictions for the subject's starting position, subject's ending position, object's starting position, and object's ending position, respectively. and This represents the learnable parameters.

[0082] Furthermore, the loss calculation in this invention includes classification loss calculation and joint optimization with priority constraints. The classification loss processing is as follows: For the classification task of entities and relations, cross-entropy loss is used to measure the accuracy of classification.

[0083] The objective of relation classification is to predict the relation category in a triple, and its loss is defined as:

[0084]

[0085] Among them, L r Let L be the cross-entropy loss between entities and relations, where R represents the total set of relations, containing multiple types of relations, and r represents a type of relation in the set R. The goal of entity location classification is to predict the locations of subjects and objects; the subject loss is L. s With object loss L o They are defined as follows:

[0086]

[0087] In this context, S represents the subject, O represents the object, start indicates the starting position, and end indicates the position. Since both S and O are entities, predicting the entity's starting position and ending position requires identifying the entity.

[0088] The Lagrange multiplier method is used to perform weighted optimization on the subject loss and the object loss to obtain the optimal loss for the subject position and the optimal loss for the object position. A rescaling strategy is then used to normalize the optimal loss for the subject position, the optimal loss for the object position, the subject loss, and the object loss to obtain the target loss. The initial training of the non-autoregressive model is then completed, and the initial knowledge extraction model is obtained.

[0089] The process of joint optimization and priority constraint in this invention is as follows: To balance entity location classification and relation classification tasks, this invention employs a priority constraint optimization method based on the Lagrange multiplier method. Entity location classification and object location classification are designated as high-priority objectives, while relation classification is designated as a low-priority objective. By introducing Lagrange multipliers, the losses of each task are weighted and optimized, thereby ensuring the inviolability of the priority objectives. The optimization objective is defined as:

[0090]

[0091] in, and These are the optimal losses for the subject's position and the object's position, respectively, λ s With λ o It is a Lagrange operator used to constrain the optimization of high-priority objectives.

[0092] To maintain the stability of high-priority objectives, this invention dynamically updates the Lagrange operator in each round of training:

[0093]

[0094] Where α is the learning rate, λ max It is the upper bound of the Lagrange operator, and clip is used to prevent training instability caused by excessive updates.

[0095] To further improve the stability of the training process, this invention employs a rescaling strategy to normalize the loss function, resulting in the following final loss function form:

[0096]

[0097] This rescaling method effectively prevents gradient explosion caused by the rapid growth of Lagrange multipliers, thus making the optimization process smoother. Through this method, the present invention maximizes the performance of the relationship classification task while ensuring that the performance of the subject and object position classification task is not degraded.

[0098] The initial knowledge extraction model is iteratively trained based on the target loss and the enhanced text data. When the number of iterations reaches a preset iteration threshold, the training is completed, and the knowledge extraction model is obtained.

[0099] After obtaining the initial knowledge extraction model, it is necessary to iteratively train the initial knowledge extraction model to correct the model parameters. The specific implementation process is as follows: using text augmentation data, the text augmentation data is repeatedly input into the updated initial knowledge extraction model as training data. Each time, the calculated loss is used to correct the model parameters to achieve model training, and finally a trained model (i.e., the knowledge extraction model in this invention) is obtained.

[0100] Step S30: Obtain the current text data, and perform triple knowledge extraction processing on the current text data through the knowledge extraction model to obtain the target triple knowledge extraction result.

[0101] When it is necessary to extract triples from the current text data, the current text data is input into the trained knowledge extraction model. The knowledge extraction model performs triple knowledge extraction processing on the current text data, and the triples extracted from the previous text data are obtained in the operation result of the knowledge extraction model.

[0102] The beneficial effects of this invention: This invention proposes a document-level entity relation joint extraction method, which aims to solve the challenges faced by the prior art, such as data sparsity, relation instance diversity and entity relation optimization conflict, by introducing a data augmentation strategy based on relation properties and a priority constraint optimization method based on the Lagrange multiplier method. The knowledge extraction method proposed in this invention can effectively improve the accuracy of entity relation joint extraction.

[0103] Furthermore, such as Figure 3 As shown, based on the above-described knowledge extraction method based on data augmentation and priority constraints, this invention also provides a knowledge extraction system based on data augmentation and priority constraints, wherein the knowledge extraction system based on data augmentation and priority constraints includes:

[0104] The data augmentation processing module 51 is used to acquire initial text data and perform data augmentation processing on the initial text data to obtain augmented text data;

[0105] Model training module 52 is used to determine a pre-trained language model and a non-autoregressive model, perform text encoding processing on the enhanced text data through the pre-trained language model, and train the model through the non-autoregressive model to obtain a knowledge extraction model.

[0106] The triplet knowledge extraction module 53 is used to obtain the current text data, and to perform triplet knowledge extraction processing on the current text data through the knowledge extraction model to obtain the target triplet knowledge extraction result.

[0107] Furthermore, such as Figure 4 As shown, based on the above-mentioned knowledge extraction method and system based on data augmentation and priority constraints, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 4 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0108] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a knowledge extraction program 40 based on data augmentation and priority constraints, which can be executed by the processor 10 to implement the knowledge extraction method based on data augmentation and priority constraints described in this application.

[0109] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the knowledge extraction method based on data augmentation and priority constraints.

[0110] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The terminals communicate with each other via a system bus.

[0111] In one embodiment, when the processor 10 executes the knowledge extraction program 40 based on data augmentation and priority constraints in the memory 20, it implements the steps of the knowledge extraction method based on data augmentation and priority constraints as described above.

[0112] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a knowledge extraction program based on data augmentation and priority constraints, and the knowledge extraction program based on data augmentation and priority constraints, when executed by a processor, implements the steps of the knowledge extraction method based on data augmentation and priority constraints as described above.

[0113] In summary, this invention provides a knowledge extraction method, system, and terminal based on data augmentation and priority constraints. The method includes: acquiring initial text data and performing data augmentation on the initial text data to obtain augmented text data; determining a pre-trained language model and a non-autoregressive model, performing text encoding on the augmented text data using the pre-trained language model, and training the model using the non-autoregressive model to obtain a knowledge extraction model; acquiring current text data, and performing triplet knowledge extraction on the current text data using the knowledge extraction model to obtain a target triplet knowledge extraction result. This invention effectively alleviates the problems of data sparsity and insufficient diversity of triplet data relationship instances by performing data augmentation on the initial text data. Simultaneously, by using the augmented text data after data augmentation to train the non-autoregressive model, the knowledge extraction accuracy of the model can be effectively improved. The knowledge extraction model obtained after training can accurately extract triplet data from the current text data.

[0114] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0115] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0116] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A knowledge extraction method based on data augmentation and priority constraints, characterized in that, The knowledge extraction method based on data augmentation and priority constraints includes: Obtain initial text data and perform data augmentation processing on the initial text data to obtain augmented text data; A pre-trained language model and a non-autoregressive model are determined. The enhanced text data is processed by text encoding using the pre-trained language model, and the model is trained using the non-autoregressive model to obtain a knowledge extraction model. The process of determining a pre-trained language model and a non-autoregressive model, performing text encoding on the enhanced text data using the pre-trained language model, and training the model using the non-autoregressive model to obtain a knowledge extraction model specifically includes: A pre-trained language model is determined, the enhanced text data is input into the pre-trained language model, and the enhanced text data is processed by the encoder in the pre-trained language model to obtain context embedding data. A non-autoregressive model is determined, and dynamic attention gating is obtained by performing context feature extraction and weight dynamic adjustment processing based on the context embedding data through the non-autoregressive decoder in the non-autoregressive model; The loss is calculated by using the fully connected layer in the non-autoregressive model based on the dynamic attention gating to obtain the subject loss and the object loss. The subject loss and the object loss are then weighted, optimized, and normalized to obtain the target loss and the initial knowledge extraction model. The initial knowledge extraction model is iteratively trained based on the target loss and the enhanced text data. When the number of iterations reaches a preset iteration threshold, the training is completed, and the knowledge extraction model is obtained. The loss calculation, performed through the fully connected layer in the non-autoregressive model using the dynamic attention gating, yields the subject loss and the object loss. Specifically, this includes: The dynamic attention gating is input into the fully connected layer of the non-autoregressive model, and the dynamic attention gating is mapped to the relation category space through the fully connected layer to obtain the relation prediction probability. Calculate the predicted probability of the subject's starting position, the predicted probability of the subject's ending position, the predicted probability of the object's starting position, and the predicted probability of the object's ending position based on the predicted probability of the relationship. The subject loss is calculated based on the predicted probability of the subject's starting position and the predicted probability of the subject's ending position, and the object loss is calculated based on the predicted probability of the object's starting position and the predicted probability of the object's ending position. Obtain the current text data, and perform triple knowledge extraction processing on the current text data through the knowledge extraction model to obtain the target triple knowledge extraction result.

2. The knowledge extraction method based on data augmentation and priority constraints according to claim 1, characterized in that, The process of obtaining initial text data and performing data augmentation on the initial text data to obtain augmented text data specifically includes: Obtain triples from the initial text data and determine the relational properties of the triples. Based on the properties of the relationship, the triplet data is augmented to obtain initial augmented text data; The entity self-reference enhancement strategy is used to generate self-reference triples corresponding to the triple data, and the enhanced text data is obtained based on the self-reference triples and the initial enhanced text data.

3. The knowledge extraction method based on data augmentation and priority constraints according to claim 2, characterized in that, The properties of the relationship include symmetric relationships, undirected relationships, and asymmetric relationships; The step of performing data augmentation processing on the triplet data based on the relational properties to obtain initial augmented text data specifically includes: If the relation property is the symmetric relation or the undirected relation, then obtain the symmetric triplet of the triplet data; If the relation property is the asymmetric relation, then obtain the invertible triplet of the triplet data; The symmetric triples and the invertible triples are added to the triples of the text data to obtain the initial enhanced text data.

4. The knowledge extraction method based on data augmentation and priority constraints according to claim 1, characterized in that, The step of obtaining dynamic attention gating by performing context feature extraction and dynamic weight adjustment based on the context embedding data using the non-autoregressive decoder in the non-autoregressive model specifically includes: The context embedding data is input into the non-autoregressive decoder in the non-autoregressive model to obtain a preset number of query embedding data, and the dependency relationship between triples in the enhanced text data is calculated based on the query embedding data. Contextual features are calculated based on the contextual embedding data and the query embedding data. The dependencies between triples in the enhanced text data and the contextual features are dynamically weighted to obtain dynamic attention gating.

5. The knowledge extraction method based on data augmentation and priority constraints according to claim 1, characterized in that, The weighted optimization and normalization of the subject loss and the object loss to obtain the target loss and the initial knowledge extraction model specifically includes: The Lagrange multiplier method with priority constraints is used to perform weighted optimization on the subject loss and the object loss to obtain the optimal loss for the subject position and the optimal loss for the object position. A rescaling strategy is used to normalize the optimal loss at the subject position, the optimal loss at the object position, the subject loss, and the object loss to obtain the target loss. The initial training of the non-autoregressive model is then completed, resulting in the initial knowledge extraction model.

6. A knowledge extraction system based on data augmentation and priority constraints, characterized in that, The knowledge extraction system based on data augmentation and priority constraints is used to implement the knowledge extraction method based on data augmentation and priority constraints as described in any one of claims 1-5, wherein the knowledge extraction system based on data augmentation and priority constraints includes: The data augmentation processing module is used to acquire initial text data and perform data augmentation processing on the initial text data to obtain augmented text data; The model training module is used to determine a pre-trained language model and a non-autoregressive model, perform text encoding processing on the enhanced text data using the pre-trained language model, and train the model using the non-autoregressive model to obtain a knowledge extraction model. The triplet knowledge extraction module is used to acquire the current text data, and to perform triplet knowledge extraction processing on the current text data through the knowledge extraction model to obtain the target triplet knowledge extraction result.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a knowledge extraction program based on data augmentation and priority constraints stored in the memory and executable on the processor. When the knowledge extraction program based on data augmentation and priority constraints is executed by the processor, it implements the steps of the knowledge extraction method based on data augmentation and priority constraints as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a knowledge extraction program based on data augmentation and priority constraints, which, when executed by a processor, implements the steps of the knowledge extraction method based on data augmentation and priority constraints as described in any one of claims 1-5.

Images