Text sentence processing method, device, computer equipment and storage medium

Abstract

The present application relates to a text sentence processing method, apparatus, computer device, and storage medium, comprising: obtaining sample text sentences containing entity pairs and relationship labels for the entity pairs; extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels, and performing positive and negative sampling processing to obtain a training set; inputting the training set into a relationship extraction model to be trained to generate a loss value including a contrast loss value; the contrast loss value is used to characterize the difference between the similarity of the sentences in the positive sentence pairs and the similarity of the sentences in the negative sentence pairs; adjusting the parameters of the relationship extraction model based on the loss value, and returning to the step of extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels to perform iterative training until a training stop condition is met to obtain a relationship extraction model; the relationship extraction model is used to identify entity relationships between entity pairs in the text sentences. The present method can effectively improve the accuracy of entity relationship extraction.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text statement processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the rapid development of computer technology, natural language processing (NLP) technology has gradually matured. Relation extraction is a key NLP task, aiming to extract structured information from large amounts of raw text for use in question-answering systems, dialogue systems, recommendation systems, search engines, and more. With the advancement of neural network models, a growing number of relationship extraction methods based on feature engineering have emerged.

[0003] Current methods for extracting relationships often rely on feature engineering. Most relationship extraction models use entity word vectors as the primary features for relationship extraction, leading to a certain degree of overfitting to entity names during relationship extraction. This can easily lead to error accumulation during feature construction, hindering system performance and resulting in low accuracy in entity relationship extraction for natural language processing tasks. Summary of the Invention

[0004] Based on this, it is necessary to provide a text sentence processing method, device, computer equipment and storage medium that can effectively improve the accuracy of entity relationship extraction in natural language processing tasks to address the above technical problems.

[0005] A text sentence processing method, the method comprising:

[0006] Obtaining a sample text sentence; the sample text sentence includes entity pairs and relationship labels of the entity pairs;

[0007] Extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performing positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set;

[0008] Inputting the training set into the relation extraction model to be trained to generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to represent the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair;

[0009] Adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met, thereby obtaining a relationship extraction model; the relationship extraction model is used to identify entity relationships between entity pairs in text sentences.

[0010] A text sentence processing device, comprising:

[0011] A sample acquisition module is used to acquire sample text sentences; the sample text sentences contain entity pairs and relationship labels of the entity pairs;

[0012] a sampling processing module, configured to extract positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and perform positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set;

[0013] a relation extraction module, configured to input the training set into a relation extraction model to be trained and generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to represent the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair;

[0014] A parameter adjustment module is used to adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, so as to perform iterative training until the training stop condition is met to obtain a relationship extraction model.

[0015] In one embodiment, the entity pair includes a head entity and a tail entity; the sample acquisition module is further used to obtain an initial sample text sentence; according to a preset probability value, a head entity or a tail entity is selected from the entity pair of the initial sample text sentence; the selected head entity or tail entity is replaced with a preset character to obtain a sample text sentence after entity masking.

[0016] In one embodiment, after adjusting the parameters of the relationship extraction model according to the loss value and before returning to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, the sample acquisition module is further used to select a head entity or a tail entity from the entity pair of the sample text sentence according to a preset probability value; and replace the selected head entity or tail entity with a preset character to obtain a sample text sentence after entity masking.

[0017] In one embodiment, the sampling processing module is further used to generate a sample sentence package based on the sample text sentences with the same relationship labels; combine any two sample text sentences in the sample sentence package to obtain a positive sentence pair; and combine any two sample text sentences with different relationship labels to obtain a negative sentence pair.

[0018] In one embodiment, the sampling processing module is further used to sample the positive example sentence pairs and the negative example sentence pairs according to a preset positive-negative example sampling ratio; and generate a training set based on the sampled positive example sentence pairs and negative example sentence pairs.

[0019] In one embodiment, the relationship extraction module is further used to input the training set into the relationship extraction model to be trained, extract the semantic relationship features of each of the sample text sentences in the training set; determine the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair based on the semantic relationship features of each of the sample text sentences; determine the contrast loss value based on the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair, and generate the loss value based on the contrast loss value.

[0020] In one embodiment, the relationship extraction module is also used to extract the contextual semantic features and entity association features of each of the sample text sentences in the training set through the relationship extraction model; perform feature splicing on the contextual semantic features and the entity association features to obtain spliced ​​joint features; and obtain the semantic relationship features of each of the sample text sentences based on the spliced ​​joint features.

[0021] In one embodiment, the positive example sentence pair and the negative example sentence pair respectively have corresponding sample labels determined according to the relationship labels; the relationship extraction module is further used to generate the contrast loss value based on the difference between the similarity between the sentences in the positive example sentence pair and the corresponding sample labels, and the difference between the similarity between the sentences in the negative example sentence pair and the corresponding sample labels.

[0022] In one embodiment, the loss value includes a first loss value and a second loss value; masking is performed on the text in each of the sample text sentences in the training set; text context features of each text in each of the sample text sentences after masking are extracted; mask prediction is performed on each of the sample text sentences based on the text context features, and a corresponding cross entropy loss is determined based on the mask prediction result; and the second loss value is determined based on the cross entropy loss.

[0023] In one embodiment, the device also includes a text processing module for obtaining a text sentence to be processed; identifying entity pairs in the text sentence to be processed; and inputting the text sentence to be processed and the entity pair into the relationship extraction model to obtain an entity relationship extraction result of the entity pair in the text sentence to be processed.

[0024] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0025] Obtaining a sample text sentence; the sample text sentence includes entity pairs and relationship labels of the entity pairs;

[0026] Extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performing positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set;

[0027] Inputting the training set into the relation extraction model to be trained to generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to represent the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair;

[0028] Adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met, thereby obtaining a relationship extraction model; the relationship extraction model is used to identify entity relationships between entity pairs in text sentences.

[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0030] Obtaining a sample text sentence; the sample text sentence includes entity pairs and relationship labels of the entity pairs;

[0031] Extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performing positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set;

[0032] Inputting the training set into the relation extraction model to be trained to generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to represent the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair;

[0033] Adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain the relationship extraction model.

[0034] The above-mentioned text sentence processing method, device, computer equipment and storage medium obtain sample text sentences containing entity pairs and relationship labels of entity pairs, extract positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and perform positive and negative sample sampling processing on the positive sentence pairs and negative sentence pairs, so as to obtain an effective positive and negative training set. By inputting the training set into the relationship extraction model to be trained, a loss value including a comparative loss value is generated. The comparative loss value represents the difference between the similarity of the sentences in the positive sentence pairs and the similarity of the sentences in the negative sentence pairs. Therefore, by adjusting the parameters of the relationship extraction model according to the loss value, it is possible to effectively perform comparative learning based on the similarity of the positive and negative sample sentence pairs. And return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met, and a trained relationship extraction model is obtained. By sampling positive and negative sample sentence pairs and adjusting parameters according to the difference in similarity between the positive and negative sample sentence pairs for comparative learning, the model can better learn the features related to the two entities, thereby effectively improving the relationship extraction accuracy of the relationship extraction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is an application environment diagram of a text sentence processing method in one embodiment;

[0036] Figure 2 Schematic diagram of a flow chart of a text sentence processing method in one embodiment;

[0037] Figure 3 Schematic diagram of a flow chart of a text sentence processing method in another embodiment;

[0038] Figure 4 1 is a flowchart of a text sentence processing method in another embodiment;

[0039] Figure 5 Schematic diagram of a flow chart of a text sentence processing method in a specific embodiment;

[0040] Figure 6 A schematic diagram of training a relationship extraction model in a specific embodiment;

[0041] Figure 7 A comparison diagram of the relationship extraction effects obtained by testing on a relationship extraction task dataset using supervised learning in one embodiment;

[0042] Figure 8 A comparison diagram of the relationship extraction effects obtained by testing on a relationship extraction task dataset using few-shot learning in one embodiment;

[0043] Figure 9is a structural block diagram of a text sentence processing device in one embodiment;

[0044] Figure 10 is a structural block diagram of a text sentence processing device in another embodiment;

[0045] Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] The target object detection method provided in this application can be applied to a computer device. The computer device can be a terminal or a server. It is understood that the target object detection method provided in this application can be applied to a terminal or a server, or to a system including a terminal and a server, and implemented through the interaction between the terminal and the server.

[0048] The text sentence processing method provided in this application can be applied to Figure 1 In the application environment shown. The terminal 102 communicates with the server 104 through the network. The terminal 102 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 102 and the server 104 can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0049] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network that provides these resources is called a "cloud." As a provider of cloud computing infrastructure, a cloud computing resource pool (referred to as a cloud platform, commonly referred to as an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed within this resource pool for external clients to choose from. This cloud computing resource pool primarily includes computing devices (virtualized machines, including operating systems), storage devices, and network devices. Based on logical functional divisions, the PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. SaaS can also be deployed directly on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS is a variety of business software, such as web portals and image processing software.

[0050] Specifically, after the terminal 102 obtains the sample text sentences containing entity pairs and relationship labels of the entity pairs from the server 104, it extracts positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performs positive and negative sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set. By inputting the training set into the relationship extraction model to be trained, a loss value including a contrast loss value is generated. The contrast loss value represents the difference between the similarity of the sentences in the positive sentence pairs and the similarity of the sentences in the negative sentence pairs. And return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met, and a trained relationship extraction model is obtained.

[0051] It can be understood that the text sentence processing method in each embodiment of the present application adopts the machine learning technology in artificial intelligence technology to train a relationship extraction model that can accurately extract entity pair relationships. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0052] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by formulas. It can be understood that the relationship extraction model trained in some embodiments of the present application is trained using machine learning technology. By training the relationship extraction model based on this machine learning technology, a relationship extraction model with more accurate relationship extraction can be trained.

[0053] In one embodiment, Figure 2 As shown, a text sentence processing method is provided, and the method is applied to a computer device as an example for explanation. The computer device can be Figure 1 The terminal or server in the embodiment includes the following steps:

[0054] S202, obtaining a sample text sentence; the sample text sentence includes entity pairs and relationship labels of the entity pairs.

[0055] A sample is a subset of individuals actually observed or investigated in a study, while the entire research population is called a population. A sample text sentence is a collective term for the elements to be examined that are extracted from the population. Sample text sentences refer to the training sample data used to pre-train the relation extraction model. Sample text sentences also include entity pairs and the relationship labels between the entity pairs.

[0056] It is understood that the sample text sentences can be obtained from the Internet, input from a local input device, or read from a memory. The sample text sentences include pre-identified entity pairs. The relationship labels of the entity pairs in the sample text sentences can be obtained through manual annotation or automatically generated based on remote supervision, that is, automatic annotation based on the assumption of remote supervision.

[0057] The distant supervision algorithm operates on the following assumption: for a triple (consisting of a pair of entities and a relationship) in an existing knowledge graph, it is assumed that any sentence in the sample document library containing this pair of entities reflects this relationship to some extent. Based on this assumption, the distant supervision algorithm can label the relationship labels for sentences in the sample document library based on a pre-annotated knowledge graph, effectively automatically labeling the samples.

[0058] It is understood that the sample text sentence can be a sentence. The relationship tag records the position of the labeled entity pairs in the sample text sentence and the relationship between the labeled entity pairs. For example, the location information of the entity in the sample text sentence can be annotated in the form of an entity identifier, such as the sample text sentence can be " <e1>Tsinghua University <e1>Located in <e2>Beijing <e2>",So" <e1> ”、" <e2>” represent the corresponding entity identifiers, that is, "Tsinghua University" can be labeled as entity 1, and "Beijing" can be labeled as entity 2. The relationship label between the labeled entity pairs in the sample sentence pair can be represented by a vector or a relationship type label. For example, if the relationship between the entity pairs is type 1, it can be represented by a vector, for example, except for the position where the relationship type is set to 1, all other positions are set to 0.

[0059] An entity refers to something objectively existing in reality that is distinguishable and independent. Examples include names of people, places, commodities, and names. Examples include "China," "Beijing," and "The Great Wall." An entity pair is a pair of entities. Entities are the most basic elements in a knowledge graph, and different entities have different relationships. A relationship is a connection of some nature between things, while an entity relationship represents an abstract connection between two or more entities. For example, entity relationships can be described using triples, a data structure that represents a triple table.

[0060] Relation extraction is the process of identifying relationships between entities by learning the semantic connections between multiple entities in a text. The input for relation extraction is a paragraph or sentence of text, and the output is typically a triple: <entity 1, relationship, entity 2>. For example, given the input text "Tsinghua University is a university in Haidian District, Beijing," after relation extraction, the output triple is <Tsinghua University, location, Haidian District, Beijing>. This could also be represented as "location (Tsinghua University, Haidian District, Beijing)." Of course, in some cases, two entities may have no relationship, which can be represented as <entity 1, NA, entity 2>.

[0061] Specifically, the relationship extraction model to be trained can be a pre-built neural network model, for example, a neural network model based on BERT (Bidirectional Encoder Representations from Transformer), CNN (Convolutional Neural Network), DNN (Deep Neural Network), LSTM (Long Short-Term Memory Neural Network), etc., which is not limited here.

[0062] In the process of pre-training the relationship extraction model, the computer device first obtains sample text sentence pairs, which contain pre-identified entity pairs and relationship labels of the entity pairs, so as to train the relationship extraction model using the sample text sentence pairs.

[0063] S204 , extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performing positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set.

[0064] The process of selecting samples is called sampling, and the sampling method varies depending on the object. A positive sentence pair is a sentence pair consisting of two sample text sentences with the same relation label; a negative sentence pair is a sentence pair consisting of two sample text sentences with different relation labels.

[0065] Specifically, after obtaining sample text sentences and the relationship labels for each entity pair, the computer device combines sample text sentences with the same relationship labels to obtain text sentence pairs with the same relationship. These sample sentence pairs with the same relationship are used as positive sentence pairs. Sample text sentence pairs with different relationship labels are further combined to form text sentence pairs with different relationships. These sample sentence pairs with different relationships are then used as negative sentence pairs. Positive and negative sentence pairs are thus extracted from the sample text sentences based on the relationship labels.

[0066] It's understandable that the number of negative pairs among the positive and negative pairs obtained from the sample text sentences far exceeds the number of positive pairs. Using only downsampling might result in ineffective utilization of negative pairs. Therefore, positive and negative sample sampling can be performed in each round of training to generate the training data for each round.

[0067] After obtaining positive and negative sentence pairs from the sample text sentences, the computer device further performs positive and negative example sampling processing on the positive and negative sentence pairs. For example, the sampling processing can be performed according to a preset method, such as sampling according to a preset positive-negative ratio, so as to obtain training samples covering both positive and negative sentence pairs and generate the training set for this round. By performing positive and negative example sampling processing, the utilization rate of negative example samples can be effectively improved, thereby effectively obtaining a training set containing positive and negative example samples, which is conducive to better learning the entity relationship information of the positive and negative sentence pairs during the process of training the relationship extraction model.

[0068] S206, input the training set into the relationship extraction model to be trained to generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to characterize the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair.

[0069] The initial relation extraction model refers to the relation extraction model before training, before model parameters are adjusted. A relation extraction model can include a loss function. A loss function maps the values ​​of a random event or its related random variables to a non-negative real number to represent the "difference" or "loss" of the random event. In machine learning, loss functions are used to estimate model parameters. They are often used as learning criteria in conjunction with optimization problems, i.e., minimizing the loss function to solve and evaluate the model.

[0070] It can be understood that the loss value can be obtained based on the loss function, and the loss value represents the difference between the result predicted by the relation extraction model and the actual sample label.

[0071] The computer device performs positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set, and then inputs the training set obtained this time into the relationship extraction model to be trained to perform relationship extraction processing.

[0072] Specifically, the encoding layer in the relation extraction model encodes each sample text sentence in the training set to obtain word vector representations, sentence vector representations, and contextual semantic information representations for each sentence. These extracted feature vectors are then fused to obtain the final vector representation for each sample text sentence. A linear mapping layer in the relation extraction model then uses the final vector representation of each sample text sentence to obtain the probability distribution of each sample text sentence across all relation categories and the corresponding loss value, which is then used to train the relation extraction model.

[0073] After obtaining the final vector representation of each sample text sentence through the encoding layer of the relation extraction model, the similarity extraction layer of the relation extraction model determines the similarity of the sentences in each positive sentence pair and the similarity of the sentences in each negative sentence pair. The contrastive loss value can then be calculated based on the difference between the similarity of the sentences in the positive sentence pairs and the similarity of the sentences in the negative sentence pairs.

[0074] Since the contrastive loss value represents the difference between the similarity of sentences in a positive sentence pair and the similarity of sentences in a negative sentence pair, by adjusting the parameters of the relation extraction model based on the loss value, it is possible to effectively perform contrastive learning based on the similarity of positive and negative sample sentence pairs.

[0075] Contrastive learning learns representations by maximizing the consistency between different semantic extensions of the same data sample through a contrastive loss in the latent space. Contrastive self-supervised learning constructs representations by learning encodings that make two things similar or different. This allows the relation extraction model to continuously learn similar entity relationships during training, improving the accuracy of semantic relationship predictions. This effectively improves the relationship extraction accuracy of the relation extraction model.

[0076] S208, adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain the relationship extraction model; the relationship extraction model is used to identify the entity relationship of entity pairs in the text sentences.

[0077] It is understood that during the training of the relation extraction model, multiple rounds of iterative training are required. Each round of iterative training gradually converges the model by adjusting the model parameters of the initial relation extraction model in that round, thereby obtaining the final relation extraction model. The current round is the current round of model training.

[0078] The training stop condition refers to the end condition of the model training. For example, the training stop condition can be reaching a preset number of iterations, or the relationship extraction performance index of the relationship extraction model after adjusting the parameters reaches a preset index.

[0079] The computer device uses the relation extraction model to perform relation extraction on the sample text sentences in the training set, obtains the similarity between the sample text sentences and the entity relationship extraction results, and then adjusts the parameters of the relation extraction model based on the obtained loss value. The computer device further returns to the step of extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels, thereby iteratively training the relation extraction model.

[0080] Specifically, the computer device may perform positive and negative example sampling on the positive and negative sentence pairs, and the resulting training set may be used as the training set for the current round of training. Relationship extraction processing is performed on the training set for this round using the relationship extraction model, and after obtaining a loss value, the parameters of the relationship extraction model for this round are adjusted in a direction that reduces the loss difference based on the obtained loss value.

[0081] If the training stop condition is not met in the current round, the process returns to the step of extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels to enter the next round. The computer device then resamples the positive and negative sentence pairs to obtain the training set for the next round. This training set is then input into the relationship extraction model for relationship extraction processing, thereby continuing iterative training. When the training stop condition is met, iterative training stops, resulting in a trained relationship extraction model.

[0082] It can be understood that the trained relationship extraction model is a machine learning model that has the ability to extract entity relationships from text sentences. It can be directly applied to perform relationship extraction processing on text sentences to identify the entity relationships between entity pairs in text sentences.

[0083] In one embodiment, the difference between the predicted relationship extraction result and the relationship label can be measured using a loss function. A function such as cross entropy or mean squared error can be selected as the loss function. The computer device can terminate training when the value of the loss function is less than a preset value, thereby obtaining a relationship extraction model that can be used to accurately and effectively extract entity relationships from text.

[0084] It can be understood that the above-mentioned relationship extraction model can be applied to relationship extraction tasks in multiple scenarios. For example, the relationship extraction model can be trained by supervised learning, semi-supervised learning, open learning (unsupervised learning), few-shot learning training, etc., without limitation here.

[0085] In one embodiment, a supervised model training method can be used. Under supervised learning, the data input to the model is training data, and each set of training data has a clear identifier, that is, each set of sample text sentences includes entity pairs and relationship labels of entity pairs. In the process of training the relationship extraction model using supervised learning, each prediction result is compared with the relationship label of the data in the training set to obtain the corresponding difference, and the parameters of the relationship extraction model are continuously adjusted according to the difference, until the prediction result of the direct relationship extraction model reaches an expected accuracy. Among them, the relationship extraction model can adopt logistic regression (Logistic Regression), back propagation neural network (BackPropagationNeural Network), etc., which are not limited here.

[0086] Taking the back propagation algorithm used in the relation extraction model as an example, in the process of iterative training of the relation extraction model, based on the back propagation algorithm, the parameters are updated in the direction of gradient descent, the weights and biases are adjusted to minimize the overall error, and the parameters of the relation extraction model are gradually adjusted to iteratively train the relation extraction model.

[0087] In another embodiment, a semi-supervised model training method can be used. In semi-supervised learning, a large amount of unlabeled data can be used, and labeled data can be used for pattern recognition. By using semi-supervised learning, the resource consumption of processing data can be effectively reduced, and at the same time, relatively high accuracy can be achieved. In the semi-supervised training method, the computer device can only use sample text sentences that partially contain entity pair relationship labels and sample text sentences that partially do not contain entity pair relationship labels as training sets, and input them into the relationship extraction model for prediction and training. In this learning method, part of the input data is labeled and part is not labeled. In this learning and training method, the model first needs to learn the intrinsic structure of the data in order to reasonably organize the data for prediction. By first attempting to model the unlabeled data and then predicting the labeled data on this basis, the computer device can then adjust the parameters of the relationship extraction model according to the loss value of the semi-supervised learning and iteratively train the relationship extraction model.

[0088] In the above-mentioned text sentence processing method, after a computer device obtains sample text sentences containing entity pairs and relationship labels for the entity pairs, it extracts positive and negative sentence pairs from the sample text sentences based on the relationship labels and performs positive and negative sample sampling on the positive and negative sentence pairs, thereby obtaining an effective positive and negative training set. The training set is then input into the relationship extraction model to be trained, generating a loss value including a contrastive loss. The contrastive loss value represents the difference between the similarity between the sentences in the positive and negative sentence pairs. By adjusting the parameters of the relationship extraction model based on the loss value, effective contrastive learning can be performed based on the similarity between the positive and negative sample sentence pairs. The method then returns to the step of extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels to perform iterative training until a training stop condition is met, thereby obtaining a trained relationship extraction model. By sampling positive and negative sample sentence pairs and adjusting the parameters based on the difference in similarity between the positive and negative sample sentence pairs for contrastive learning, the model can better learn the features related to the two entities, thereby effectively improving the relationship extraction accuracy of the relationship extraction model.

[0089] In one embodiment, Figure 3 As shown, a flowchart of another text sentence processing method is provided, including the following steps:

[0090] S302, obtaining a sample text sentence; the initial sample text sentence contains entity pairs and relationship labels of the entity pairs, and the entity pairs include a head entity and a tail entity.

[0091] S304: Select a head entity or a tail entity from the entity pairs of the initial sample text sentence according to a preset probability value.

[0092] S306: Replace the selected head entity or tail entity with a preset character to obtain a sample text sentence after entity masking.

[0093] S308 , extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performing positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set.

[0094] S310, input the training set into the relation extraction model to be trained to generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to characterize the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair.

[0095] S312, adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain the relationship extraction model.

[0096] The entity pairs in each sample text sentence include a head entity and a tail entity. The head entity and the tail entity can be determined according to the position of the entity in the sample text sentence. For example, the head entity can be the first entity in the sample text sentence, and the tail entity can be the second entity in the same sample text sentence.

[0097] It can be understood that entity masking refers to masking some entities in the sample text sentences of the training samples. Specifically, some entities in the sample text sentences can be replaced in a mask mark manner to mask some entities.

[0098] Before obtaining the sample text sentence, the computer device may also obtain the original initial sample text sentence. The initial sample text sentence represents the initial sample data that has not been preprocessed. The computer device may then perform entity masking on the initial sample text sentence according to a preset probability value. The masking of the head entity and the tail entity is independent of each other. For example, in a sample text sentence, no entity may be masked, one entity may be masked, or all entities may be masked.

[0099] Specifically, the computer device selects a head entity or a tail entity from the entity pairs of the initial sample text sentence according to a preset probability value. For example, a portion of the head entity or the tail entity can be randomly selected from the initial sample text sentence for masking according to a probability of 0.7. The computer device then replaces the selected head entity or tail entity with a preset character, for example, replacing the entity word with a special symbol. In one embodiment, the head entity and the tail entity can also each correspond to a special symbol, and this applies to all sentences.

[0100] By replacing the selected head entity or tail entity to be masked with a preset character, the entity is masked, thereby obtaining a sample text sentence after the entity is masked, and then further processing the sample text sentence after the entity is masked.

[0101] In this embodiment, by pre-processing the initial sample text sentences with entity masking, masking the head or tail entity with a certain probability, it is possible to effectively prevent the relationship extraction model from overfitting to the entity names. During the model training process, not only can more contextual information be utilized for learning, but it can also prevent large gaps in downstream tasks.

[0102] In one embodiment, after adjusting the parameters of the relation extraction model according to the loss value, and before returning to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relation labels, a step of performing entity masking processing on the sample text sentences is further performed. Figure 4 As shown, a flowchart of another text sentence processing method is provided, which specifically includes the following steps:

[0103] S402, obtaining a sample text sentence; the sample text sentence includes entity pairs and relationship labels of the entity pairs.

[0104] S404: Select a head entity or a tail entity from the entity pairs of the sample text sentence according to a preset probability value.

[0105] S406: Replace the selected head entity or tail entity with a preset character to obtain a sample text sentence after entity masking.

[0106] S408, extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performing positive and negative sample sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set;

[0107] S410, inputting the training set into the relation extraction model to be trained to generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to represent the difference between the similarity of the sentences in the positive example sentence pair and the similarity of the sentences in the negative example sentence pair;

[0108] S412, adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain the relationship extraction model.

[0109] According to a preset probability value, a head entity or a tail entity is selected from the entity pairs of the sample text sentence; the selected head entity or tail entity is replaced with a preset character to obtain a sample text sentence after entity masking.

[0110] It is understood that in the process of training the relation extraction model, a dynamic entity masking method can also be adopted. That is, in each round of iterative training of the relation extraction model, entity masking is performed on the sample text sentences.

[0111] Specifically, after obtaining a sample text sentence, the computer device can select the head entity or the tail entity from the entity pair of the sample text sentence according to a preset probability value, and replace the selected head entity or tail entity with a preset character to obtain the sample text sentence after entity masking. The computer device then extracts positive sentence pairs and negative sentence pairs from the sample text sentence based on the relationship label, and performs positive and negative example sampling on the positive sentence pairs and the negative sentence pairs to obtain a training set. The training set is then input into the relationship extraction model to be trained, a loss value is generated, and the parameters of the relationship extraction model are adjusted based on the loss value.

[0112] Next, after adjusting the parameters of the relationship extraction model based on the loss value, and before returning to the step of extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels, the computer device further performs entity masking processing on the sample text sentences, thereby re-performing entity masking processing on the sample text sentence pairs to obtain updated entity-masked sample text sentences. The computer device then continues to perform the step of extracting positive and negative sentence pairs from the entity-masked sample text sentences based on the relationship labels, thereby achieving dynamic entity masking processing on the sample text sentences during the process of training the relationship extraction model.

[0113] In this embodiment, by adopting the dynamic entity masking method, the data for each training can be randomly subjected to entity masking processing, thereby enabling the relationship extraction model to make greater use of the contextual information of the sample text sentences after the masked entities for learning, thereby effectively improving the relationship extraction accuracy of the relationship extraction model.

[0114] In one embodiment, positive sentence pairs and negative sentence pairs are extracted from sample text sentences based on relationship tags, including: generating a sample sentence package based on sample text sentences with the same relationship tags; combining any two sample text sentences in the sample sentence package to obtain a positive sentence pair; and combining any two sample text sentences with different relationship tags to obtain a negative sentence pair.

[0115] Among them, the relation labels can be the entity relations between entity pairs in sample text sentences pre-labeled under the distant supervision assumption.

[0116] After obtaining sample text sentences, the computer device then extracts positive and negative sentence pairs from the sample text sentences based on the relationship tags. Specifically, the computer device may first obtain sample text sentences with consistent relationship tags and generate a sample sentence package from these sample text sentences. Therefore, all sample text sentences in this generated sample sentence package have the same relationship tags. The computer device then combines any two sentences in the same package to generate sentence pairs, and the resulting sentence pairs are considered positive sentence pairs.

[0117] At the same time, the computer device combines any two sample text sentences with different relationship labels in the sample sentence package to obtain multiple sentence pairs with different relationship labels. These sentence pairs with different relationship labels are negative sentence pairs.

[0118] In this embodiment, by using sample text sentences according to the relationship labels to generate positive sentence pairs and negative sentence pairs, all sentence pairs with consistent relationships are positive examples, that is, the semantic metrics of these sentence pairs are similar, and sentence pairs with inconsistent relationships are negative examples, that is, the semantic metrics of these sentence pairs are different. For example, for sentences with the same relationship labels, the positive sentence pairs obtained by forming a sentence pair in pairs should have a relationship representation that is close under the dot product metric. For sentences with different relationship labels, the negative sentence pairs obtained by forming a sentence pair in pairs should have a relationship representation that is far away under the dot product metric. By performing comparative learning and training on positive and negative samples, similar semantic features and different semantic features can be better learned, thereby effectively improving the semantic information extraction accuracy of the relationship extraction model.

[0119] In one embodiment, positive and negative example sampling processing is performed on positive and negative example sentence pairs to obtain a training set, including: sampling and extracting positive and negative example sentence pairs according to a preset positive and negative example sampling ratio; and generating a training set based on the sampled positive and negative example sentence pairs.

[0120] It can be understood that by performing positive and negative sampling on the sample text sentences, it can be ensured that both positive sentence pairs and negative sentence pairs will be sampled during the sampling process.

[0121] After extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels, the computer device further samples the positive and negative sentence pairs according to a preset positive-negative sampling ratio. For example, in the positive and negative sampling extraction of the current batch, assume that a batch has 2N sentences, with sentences with the same relationship forming the positive examples and sentences with different relationships forming the negative examples. Then, in the batches of this round of training, the positive-negative sampling ratio of positive and negative examples can be set to 1:2N. This can effectively improve the utilization rate of negative sample examples and effectively ensure that both positive and negative sentences are included in the sampling process.

[0122] In this embodiment, positive sentence pairs and negative sentence pairs are sampled and extracted according to a preset positive-negative sampling ratio, so that both positive and negative examples can be sampled, which can effectively improve the utilization rate of negative samples. Therefore, the training set obtained by positive and negative example sampling can be better used for model training, so that the performance of semantic relationship information extraction of the relationship extraction model is better.

[0123] In one embodiment, a training set is input into a relation extraction model to be trained to generate a loss value, including: inputting the training set into the relation extraction model to be trained to extract semantic relation features of each sample text sentence in the training set; determining the similarity of sentences in a positive sentence pair and the similarity of sentences in a negative sentence pair based on the semantic relation features of each sample text sentence; determining a comparative loss value based on the similarity of sentences in the positive sentence pair and the similarity of sentences in the negative sentence pair, and generating a loss value based on the comparative loss value.

[0124] The relationship extraction model may include an encoding layer, a similarity extraction layer, and a relationship extraction layer. The encoding layer is used to extract information such as lexical features, syntactic features, contextual semantic features, and semantic relationship features of the sample text sentences. For example, the encoding layer may be a neural network structure based on a BERT encoder, wherein the encoding layer may include an encoder (Transformer Encoder) and a decoder (Transformer Decoder). For example, the encoding layer may include several layers of Transformers, such as a 12-layer Transformer.

[0125] The similarity extraction layer can be a network layer based on a self-attention mechanism, used to calculate the similarity between sample text sentences based on the semantic relationship features of each sample text sentence. For example, a network structure such as an attention network or an MLP (Multi-Layer Perceptron, fully connected neural network) can be used. The relationship extraction layer can use a classifier to classify entity relationships based on the semantic relationship features of each sample text sentence, thereby obtaining relationship prediction results for each sample text sentence.

[0126] Specifically, in each round of model training, the computer device obtains a training set by performing positive and negative example sampling processing on positive sentence pairs and negative sentence pairs, and then inputs the obtained training set into the relationship extraction model to be trained.

[0127] First, the encoding layer of the relation extraction model extracts semantic features from each sample text sentence in the training set. This layer extracts contextual semantic features and entity association features between entity pairs for each sample text sentence. The encoding layer then uses these extracted contextual semantic features and entity association features to derive a final semantic relationship vector representation for each sample text sentence. The final semantic relationship vector representation represents the semantic relationship features of the sample text sentences extracted by the encoding layer of the relation extraction model.

[0128] Furthermore, after obtaining the semantic relationship features of each sample text sentence through the encoding layer of the relation extraction model, the similarity between the sample text sentences in each positive sentence pair and the similarity between the sample text sentences in each negative sentence pair are calculated through the similarity extraction layer of the relation extraction model based on the semantic relationship features of each sample text sentence. In this way, the similarity of the sentences in the positive sentence pairs and the similarity of the sentences in the negative sentence pairs predicted by the relation extraction model are obtained. For example, the similarity of sentences can be obtained by calculating the vector dot product, vector cosine similarity between sentences, or by introducing additional neural network calculations, etc., which are not limited here.

[0129] The computer device then determines a contrast loss value based on the similarity between the sentences in the positive sentence pair and the similarity between the sentences in the negative sentence pair. The contrast loss value can be determined based on the difference between the similarity between the sentences in the positive sentence pair and the similarity between the sentences in the negative sentence pair. Specifically, the positive sentence pair and the negative sentence pair can also include actual annotated sample labels. The sample labels of the positive sentence pair and the negative sentence pair can be determined based on the relationship labels of the sample text sentences, or based on preset labels corresponding to the positive and negative sample examples.

[0130] The difference between the similarity of sentences in a positive sentence pair and the similarity of sentences in a negative sentence pair can be determined based on the similarity of sentences in the positive sentence pair and the similarity of sentences in the negative sentence pair predicted by the relation extraction model, and the difference between the sample labels of the positive sentence pair and the negative sentence pair, thereby obtaining the comparative loss values ​​corresponding to the positive sentence pair and the negative sentence pair. A loss value for the relation extraction model is then generated based on the comparative loss value, and the parameters of the relation extraction model are adjusted based on the loss value, thereby effectively performing comparative learning based on the similarity of positive and negative sample sentence pairs. This allows the relation extraction model to continuously learn similar entity relationships during training to improve the accuracy of semantic relationship prediction, thereby effectively improving the relationship extraction accuracy of the relation extraction model.

[0131] It enables the neural network-based relation extraction model to better learn the contextual information of text sentences, thereby effectively improving the robustness of the relation extraction model and the accuracy of relation extraction.

[0132] In one embodiment, the semantic relationship features of each sample text sentence in the training set are extracted, including: extracting the contextual semantic features and entity association features of each sample text sentence in the training set through a relationship extraction model; performing feature splicing on the contextual semantic features and entity association features to obtain spliced ​​joint features; and obtaining the semantic relationship features of each sample text sentence based on the spliced ​​joint features.

[0133] Specifically, the computer device inputs the training set obtained by sampling positive and negative examples of positive sentence pairs and negative sentence pairs into the relation extraction model to be trained. The encoding layer of the relation extraction model first extracts semantic features of each sample text sentence in the training set. The encoding layer of the relation extraction model can first extract the word vector features of each word in each sample text sentence, and then extract the sentence vector features of the sample text sentence based on the word vector features, as well as the position information of each word, and further extract the grammatical features of the sample text sentence, etc., so that deep feature extraction can be further performed based on these features to obtain the contextual semantic features of each sample text sentence and the entity association features between entity pairs.

[0134] The encoding layer of the relation extraction model then concatenates the obtained contextual semantic features and entity association features, thereby obtaining the final semantic relationship vector representation of each sample text sentence based on the concatenated joint features. The final semantic relationship vector representation is the semantic relationship feature of the sample text sentence extracted by the encoding layer of the relation extraction model.

[0135] Since the sample text sentences contain identified entity pairs, they can also include the location information of the entity pairs. Therefore, after the sample text sentences are input into the relation extraction model, the encoding layer of the relation extraction model can directly extract the word vector representation corresponding to each entity based on the location information of the labeled entities, which is conducive to quickly and accurately characterizing the semantic relationships of the sample text sentences.

[0136] In one embodiment, the sample text sentence contains the entity pairs that have been identified and the location information of the entities. Furthermore, the sample text sentence may also include the vectors of the entity pairs that have been extracted and the word vectors or term vectors of each character or word in the sample text sentence. After the computer device performs positive and negative sampling on the sample text sentence to obtain a training set, and inputs the training set into the relation extraction model to be trained, the relation extraction model can further extract the grammatical features, contextual semantic features, and entity association features between entity pairs of the sample text sentence directly based on the location information and entity vectors of the entity pairs of each sample text sentence, as well as the word vectors or term vectors of each character or word in the sample text sentence, so as to quickly and accurately obtain the semantic relationship representation of each sample text sentence, and at the same time effectively reduce the amount of calculation in the model training process, and effectively improve the training efficiency and training accuracy of the relation extraction model.

[0137] In one embodiment, the positive sentence pair and the negative sentence pair respectively have corresponding sample labels determined according to the relationship labels; the contrast loss value is determined according to the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair, including: generating the contrast loss value according to the difference between the similarity of the sentences in the positive sentence pair and the corresponding sample labels, and the difference between the similarity of the sentences in the negative sentence pair and the corresponding sample labels.

[0138] The computer device extracts positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels. In the process of obtaining a training set including positive sentence pairs and negative sentence pairs, the computer device can also generate sample labels corresponding to the positive and negative samples in the training set.

[0139] It can be understood that the relationship labels of the entity pairs in the positive example sentence pairs are consistent, while the relationship labels of the entity pairs in the negative example sentence pairs are inconsistent. The computer device can further determine the sample labels of the positive example sentence pairs based on the relationship labels, and determine the sample labels of the negative example sentence pairs based on the relationship labels.

[0140] The sample labels corresponding to the positive sentence pair and the negative sentence pair can be determined as the actual sample labels of the positive and negative samples. Thus, the positive sentence pair and the negative sentence pair each have a corresponding sample label determined according to the relationship label. For example, the sample label of the positive sentence pair can be uniformly determined as 1, and the sample label of the negative sentence pair can be uniformly determined as 0, so that the actual labels (1 or 0) of the positive and negative samples can be obtained.

[0141] The computer device inputs the training set into the relation extraction model to be trained, and performs relation extraction processing on the sample text sentences in the training set through the relation extraction model. After obtaining the similarity between each sample text sentence and the entity relationship extraction result, the comparative loss value is generated according to the difference between the similarity of the sentences in the positive sentence pair and the corresponding sample label, as well as the difference between the similarity of the sentences in the negative sentence pair and the corresponding sample label.

[0142] Specifically, the computer device will compare the predicted similarity of the sentences in the positive sentence pair through the relational extraction model with the actual sample label of the positive sentence pair. For example, the positive difference can be obtained by calculating the predicted difference between the similarity prediction value of the positive sentence pair and the sample label (such as the positive example is 1). And the predicted similarity of the sentences in the negative sentence pair is compared with the actual sample label of the negative sentence pair. For example, the negative difference can be obtained by calculating the predicted difference between the similarity prediction value of the negative sentence pair and the sample label (such as the negative example is 0). Then, a comparative loss value is generated based on the positive difference and the negative difference.

[0143] The computer device further adjusts the parameters of the relationship extraction model according to the obtained contrast loss value, and returns to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, thereby iteratively training the relationship extraction model.

[0144] In this embodiment, by calculating the similarity of sentences in the positive sentence pairs and the similarity of sentences in the negative sentence pairs, and the differences between them and the corresponding sample labels, the contrast loss value is obtained, and the parameters of the relationship extraction model are adjusted according to the contrast loss value, thereby effectively realizing contrast learning based on the similarity of positive and negative sample sentence pairs.

[0145] In one embodiment, adjusting the parameters of the relationship extraction model according to the loss value includes: updating the parameters of the relationship extraction model according to the contrast loss value so that the relationship extraction model increases the similarity between positive sample sentence pairs and reduces the similarity between negative sample sentence pairs during the iterative training process.

[0146] The contrast loss value may be determined by a contrast loss function of a relation extraction model. For example, the contrast loss function may be a NT_XentLoss (the normalized temperature-scaled cross-entropy loss) function.

[0147] It can be understood that under the assumption of distant supervision, if the head and tail entities have a certain relationship in the knowledge graph, then all sentences containing these two entities will express this relationship. Generally speaking, if two sentences have the same entity relationship labels annotated under distant supervision, then the two sentences can be considered to have a high degree of similarity, and therefore their relationship representation should be close under the click metric. Conversely, if the entity relationship labels annotated under distant supervision are inconsistent, then the two sentences can be considered to have no similarity, and therefore their relationship representation should be far apart under the click metric.

[0148] After the computer device uses the relation extraction model to determine the similarity between each sample text sentence, it calculates the difference between the sentence similarity in the positive sentence pair and the sentence similarity in the negative sentence pair and the corresponding sample labels to obtain a contrastive loss value. The computer device then updates the parameters of the relation extraction model based on the contrastive loss value.

[0149] Specifically, the computer device updates the parameters of the relation extraction model based on the contrastive loss value, so that the similarity between positive sample sentence pairs is continuously increased and the similarity between negative sample sentence pairs is continuously reduced during the iterative training of the relation extraction model. This enables the relation extraction model to learn the similar features between sentences with the same relationship and distinguish them from other samples.

[0150] For example, a binary cross entropy loss function can be used to train the predicted similarity and the actual label (0 or 1). During the iterative training process, the relationship extraction model will tend to make the predicted output closer to the actual sample label. For example, for positive example sentences, the closer the similarity of the predicted output is to the actual sample label 1, the smaller the loss function L; the closer the predicted output is to 0, the larger L. For negative example sentences, the closer the similarity of the predicted output is to the actual sample label 0, the smaller the loss function L; the closer the predicted function is to 1, the larger L. By continuously converging the loss function, the change trend of the loss function is made to meet the actual requirements, so as to train the required relationship extraction model.

[0151] In one embodiment, L CP Represents the contrast loss function, and the formula of the contrast loss function can be as follows:

[0152]

[0153] Among them, x A , x B They represent the vector representation of sentence A and sentence B after encoding. The relationship between sentence A and sentence B is the same, but the relationship between sentence A and the rest of the sentences in the training set of the current round is inconsistent. i represents the index of each sentence in the training set with different relationships. Vector x representing sentence A A The transpose of This method first calculates the dot product of the vectors of the two sentences to determine their similarity. The relationship extraction model then performs entity relationship prediction, outputting the entity relationships for each pair of entities in each sample text sentence. This increases the inter-class variance of the classifiers in the relationship extraction model during model training, thereby improving the accuracy of the model in identifying entity relationships.

[0154] In one embodiment, the loss value includes a first loss value and a second loss value; the first loss value is a contrast loss value; the method further includes: masking the text in each sample text sentence in the training set; extracting the text context features of each text in each sample text sentence after masking; performing mask prediction on each sample text sentence based on the text context features, and determining the corresponding cross entropy loss based on the mask prediction result; and determining the second loss value based on the cross entropy loss.

[0155] The target loss function of the relation extraction model may include a first loss function and a second loss function. The first loss function may be a contrast loss function, and the second loss function may be a language model loss function. For example, the first loss function may be L CP (Contrastive Loss, contrast loss function), the second loss function can be L MLM (MaskedLanguage Model Loss, masked language model loss function).

[0156] After the computer device inputs the training set into the relation extraction model to be trained, the semantic relation features of each sample text sentence are obtained through the encoding layer of the relation extraction model. Furthermore, the similarity extraction layer of the relation extraction model calculates the similarity between the sample text sentences in each positive sentence pair, as well as the similarity between the sample text sentences in each negative sentence pair, based on the semantic relation features of each sample text sentence. Furthermore, the relation extraction layer of the relation extraction model performs entity relationship classification based on the semantic relation features of each sample text sentence. Specifically, a linear mapping layer in the relation extraction model is used to obtain the probability distribution of each sample text sentence over all relation sets, thereby obtaining the relation prediction results for each sample text sentence.

[0157] During the training of the relation extraction model, the relation extraction model also masks the text in each sample text sentence in the training set. For example, an MLM (masked language model) can be used to perform mask prediction tasks. Specifically, the computer device can use a random masking method to mask part of the text in the sample text sentence. Specifically, preset characters can be used to replace the masked part of the text, that is, randomly masking part of the input (token) of the sample text sentence. For example, any text or entity in the sample text sentence can be randomly masked.

[0158] The masked sample text sentences are then input into the encoding layer of the relational extraction model for semantic feature extraction. The encoding layer of the relational extraction model can extract the word vectors and position vectors corresponding to each word or text in the sample text sentences. Based on this information, the contextual semantic information corresponding to each text can be obtained, and the contextual semantic features and entity association features corresponding to each sample text sentence can be further extracted.

[0159] After extracting the contextual semantic information of the text in each sample text sentence after masking through the relation extraction model, each sample text sentence is masked and predicted based on the contextual semantic information of the text. Then, the cross entropy loss can be calculated based on the mask prediction results to obtain the cross entropy loss corresponding to each text in the sample text sentence.

[0160] The computer device then determines a second loss value based on the cross entropy loss, or directly uses the obtained cross entropy loss as the second loss value. The second loss value can represent the contextual semantic loss corresponding to the text in the sample text sentence.

[0161] It is understood that the loss value can be determined by combining the first loss value and the second loss value. For example, the loss value of the relationship extraction model can be the sum of the first loss value and the second loss value.

[0162] The computer device then adjusts the parameters of the relationship extraction model based on the total loss value including the first loss value and the second loss value, and returns to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain the relationship extraction model.

[0163] In one embodiment, the objective loss function of the relation extraction model can be expressed as:

[0164] L=L CP +L MLM

[0165] Among them, L CP is the first loss function, L MLM The target loss function of the relation extraction model is constructed by combining the first loss function and the second loss function. The first loss value is obtained by the corresponding first loss function, and the second loss value is obtained by the corresponding second loss function. By introducing L MLM The loss function can effectively prevent catastrophic forgetting during model training and can help improve the semantic understanding ability of the relationship extraction model.

[0166] In this embodiment, a first loss value representing the difference between the similarity of sentences in a positive sentence pair and the similarity of sentences in a negative sentence pair, and a second loss value representing the contextual semantic loss of the text corresponding to the sample text sentence are calculated, and the parameters of the relationship extraction model are adjusted based on the first and second loss values. This effectively enables the relationship extraction model to perform comparative learning based on the similarity of positive and negative sample sentence pairs, and at the same time effectively improves the relationship extraction model's ability to understand semantics, allowing the model to gradually converge, thereby effectively improving the relationship extraction accuracy of the relationship extraction model.

[0167] In one embodiment, Figure 5 As shown, a flowchart of a specific text sentence processing method is provided, which includes the following steps:

[0168] S502, obtaining an initial sample text sentence; the initial sample text sentence contains entity pairs and relationship labels of the entity pairs.

[0169] S504 , selecting a head entity or a tail entity from the entity pairs of the initial sample text sentence according to a preset probability value; replacing the selected head entity or tail entity with a preset character to obtain a sample text sentence after entity masking.

[0170] S506 , generating a sample sentence package based on the sample text sentences with the same relationship tags; combining any two sample text sentences in the sample sentence package to obtain a positive sentence pair.

[0171] S508: Combine any two sample text sentences with different relationship labels to obtain a negative sentence pair.

[0172] S510, sampling and extracting positive sentence pairs and negative sentence pairs according to a preset positive-negative sampling ratio; generating a training set based on the sampled positive sentence pairs and negative sentence pairs; the positive sentence pairs and negative sentence pairs respectively have corresponding sample labels determined according to the relationship labels.

[0173] S512: Input the training set into the relation extraction model to be trained, and extract the contextual semantic features and entity association features of each sample text sentence in the training set through the relation extraction model.

[0174] S514: Concatenate the context semantic features and the entity association features, and obtain the semantic relationship features of each sample text sentence based on the concatenated joint features.

[0175] S516 , determining the similarity of sentences in the positive sentence pair and the similarity of sentences in the negative sentence pair based on the semantic relationship features of each sample text sentence.

[0176] S518 , generating a first loss value according to the difference between the similarity of the sentences in the positive sentence pair and the corresponding sample label, and the difference between the similarity of the sentences in the negative sentence pair and the corresponding sample label.

[0177] S520, masking the text in each sample text sentence in the training set; extracting the text context features of each text in each sample text sentence after masking; performing mask prediction on each sample text sentence based on the text context features, and determining the corresponding cross entropy loss based on the mask prediction result; determining a second loss value based on the cross entropy loss.

[0178] S522, generating a loss value according to the first loss value and the second loss value; the first loss value is a comparative loss value; the comparative loss value is used to characterize the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair.

[0179] S524, update the parameters of the relationship extraction model according to the loss value, so that the relationship extraction model increases the similarity between positive sample sentence pairs and reduces the similarity between negative sample sentence pairs during the iterative training process; and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain the relationship extraction model.

[0180] like Figure 6 Figure 1 is a schematic diagram of training a relationship extraction model in a specific embodiment. For example, sample text sentences include "①SpaceX was founded in 2002 by Elon Musk", "②Gates, as the co-founder of Microsoft", "③Sundar Pichai is the CEO of Alphabet Inc", "④Cook joined Apple in March 1998", etc. The sample text sentences include entity pairs and relationship labels of entity pairs. Entity pairs may also include corresponding location information identifiers and entity vector representations. For example, the entity pairs in the first sample text sentence ① are "SpaceX" and "ElonMusk", and the corresponding entity vectors are "Q193701" and "Q317521" respectively. The relationship label of the entity pair is "founded by".

[0181] After the computer device inputs these sample text sentences into the relation extraction model to be trained, the semantic relationship features of each sample text sentence are extracted respectively by the encoder in the relation extraction model. The semantic relationship features can be the final vector representation of each sample text sentence extracted by the encoder. Among them, the encoder of the relation extraction model can be a BERT encoder. After the encoder outputs the semantic relationship features of each sample text sentence, it further calculates the similarity between each sample text sentence through the similarity extraction layer, and outputs the entity relationship prediction results of each sample text sentence. Figure 6 The similarity between each sample text sentence predicted by the relation extraction model shows that the first sentence ① and the second sentence ② have a high degree of similarity, while the first sentence ① and the third sentence ③ and the fourth sentence ④ have no similarity. The computer device then generates a contrastive loss value based on the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair. The parameters of the relation extraction model are adjusted based on the loss value including the contrastive loss value, and iterative training is performed.

[0182] In this embodiment, after subjecting sample text sentences to entity masking, positive and negative sentence pairs are extracted from the sample text sentences based on the relationship labels, and positive and negative sample sampling is performed on the positive and negative sentence pairs, thereby obtaining an effective positive and negative training set. By inputting the training set into the relationship extraction model to be trained, a loss value, including a contrastive loss value, is generated. The contrastive loss value represents the difference between the similarity of the sentences in the positive sentence pairs and the similarity of the sentences in the negative sentence pairs. By adjusting the parameters of the relationship extraction model based on the loss value, effective contrastive learning can be performed based on the similarity of the positive and negative sample sentence pairs. The step of extracting positive and negative sentence pairs from the sample text sentences based on the relationship labels is then returned to iterative training until the training stop condition is met, resulting in a trained relationship extraction model. By performing contrastive learning based on positive and negative sample samples and adjusting parameters based on the difference in similarity between the positive and negative sample sentence pairs, the model can better learn the features related to the two entities, thereby effectively improving the relationship extraction accuracy of the relationship extraction model.

[0183] In a specific embodiment of experimental testing, a dataset is obtained from a relation extraction dataset as a sample text sentence of the relation extraction model for testing. The supervised learning method and the few-shot learning method are used to test the relation extraction task dataset. Figure 7 The figure shows a comparison chart of the relationship extraction effects obtained by testing on the relationship extraction task dataset using supervised learning. Specifically, in the relationship extraction task using supervised learning, test data are obtained from the TACRED, SemEval, Wiki80, and ChenProt datasets as sample text sentences, and effect tests are conducted on 1%, 10%, and 100% prior samples, respectively, and the corresponding relationship extraction results are output through the pre-trained relationship extraction model. At the same time, traditional relationship extraction tasks are also used, such as those based on the BERT model and the MTB model to obtain the effects on the corresponding relationship extraction task datasets. Reference Figure 7 , showing the relationship extraction effects obtained by testing on the corresponding relationship extraction task datasets using the BERT-based model (i.e. the model corresponding to BERT in the figure), the MTB model (i.e. the model corresponding to MTB in the figure), and the relationship extraction model obtained by the above-mentioned text sentence processing method (i.e. the model corresponding to CP in the figure).

[0184] like Figure 8 The following is a comparison chart of the relationship extraction effect obtained by testing the relationship extraction task dataset using few-shot learning. Figure 8 , showing the use of the BERT-based model (i.e., the model corresponding to BERT in the figure), the MTB model (i.e., the model corresponding to MTB in the figure), and the relationship extraction model obtained by the above-mentioned text sentence processing method (i.e., the model corresponding to CP in the figure). Test data were obtained from the FewRel1.0 and FewRel2.0 DonmainAdaptation datasets as sample text sentences, and effect experiments were conducted in the "5-way 1-shot", "5-way 5-shot", "10-way 1-shot", and "10-way 5-shot" sample batches, and the relationship extraction effects were obtained by testing on the corresponding relationship extraction task datasets.

[0185] Among them, C+M means the complete sentence input; OnlyC means only the context is input, masking the entity; OnlyM means only the entity is input without using the context. Figure 7 、 Figure 8 As can be seen from the figure, the relationship extraction model obtained by the above text sentence processing method is better than the existing model in most relationship extraction tasks. When only context information is used, the relationship extraction model obtained by the above text sentence processing method has a relatively greater improvement. From this, it can be seen that the relationship extraction model obtained by the above text sentence processing method (i.e., the CP model in the figure) can better utilize context information and extract relationships more accurately, thereby effectively improving the relationship extraction accuracy of the relationship extraction model.

[0186] In one embodiment, after obtaining the relationship extraction model, it also includes: obtaining the text sentence to be processed; identifying the entity pairs in the text sentence to be processed; inputting the text sentence to be processed and the entity pairs into the relationship extraction model to obtain the entity relationship extraction results of the entity pairs in the text sentence to be processed.

[0187] It can be understood that the text sentences to be processed are text sentences that have not been task-labeled and pre-processed. After iteratively training the relation extraction model using sample text sentences, a trained relation extraction model with high relation extraction accuracy is obtained. The pre-trained relation extraction model can then be directly used to process the text sentences to handle the relation extraction task of text sentences.

[0188] Specifically, after obtaining the text sentence to be processed, the computer device first identifies entity pairs in the text sentence to be processed. The computer device may identify entity pairs in the text sentence to be processed according to a preset algorithm or through a preset language processing model, again without limitation.

[0189] The computer device further inputs the text sentence to be processed and the identified entity pairs into the relationship extraction model, extracts the contextual semantic features and entity association features of the text sentence to be processed through the relationship extraction model, and performs feature splicing on the contextual semantic features and entity association features, obtains the semantic relationship features of the text sentence to be processed based on the spliced ​​joint features, and then predicts the entity relationship extraction results of the entity pairs in the text sentence to be processed based on the semantic relationship features.

[0190] In one embodiment, the relationship extraction model can also be a sub-model of the language processing model. When the computer device processes the text sentence to be processed, it can first input the text sentence to be processed into a pre-trained language processing model, and use the language processing model to identify entity pairs in the text sentence to be processed. Then, the text sentence to be processed and the entity pairs are input into the relationship extraction model, and the relationship extraction model performs entity relationship identification on the text sentence to be processed, and outputs the entity relationship extraction results of the entity pairs in the text sentence to be processed.

[0191] In this embodiment, since the relationship extraction model is a pre-trained neural network model with high relationship extraction accuracy, it can accurately and effectively perform relationship extraction processing on the text sentences to be processed, thereby obtaining entity relationship extraction results with high accuracy.

[0192] This application also provides an application scenario that can be applied to a natural language question-answering system, and the application scenario applies the above-mentioned text sentence processing method. Specifically, the application of the text sentence processing method in this application scenario is as follows:

[0193] In a natural language question-answering processing system, a user device obtains a user's instructions. For example, the user device can obtain the user input text entered by the user and initiate an interaction request to the question-answering system. The question-answering system then performs natural language processing on the user input text, such as first performing word segmentation and named entity recognition on the user input text to obtain entity information in the user input text. The user input text and the identified entity information are further input into a pre-trained relation extraction model, and the entity relationship extraction results of the entities in the user input text are output through the relation extraction model. The question-answering system can further obtain corresponding response information based on the entity relationship extraction results of the identified user input text and the corresponding semantic information, and return it to the user device and output it. This can effectively improve the semantic understanding ability of the question-answering system in the question-answering interaction, thereby effectively improving the interaction accuracy of the question-answering system.

[0194] It should be understood that although Figure 2-5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2-5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0195] In one embodiment, Figure 9 As shown, a text sentence processing device 900 is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: a sample acquisition module 902, a sampling processing module 904, a relationship extraction module 906 and a parameter adjustment module 908, wherein:

[0196] The sample acquisition module 902 is used to acquire sample text sentences; the sample text sentences include entity pairs and relationship labels between entity pairs;

[0197] Sampling processing module 904, configured to extract positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and perform positive and negative sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set;

[0198] The relation extraction module 906 is used to input the training set into the relation extraction model to be trained and generate a loss value; the loss value includes a contrast loss value; the contrast loss value is used to represent the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair;

[0199] The parameter adjustment module 908 is used to adjust the parameters of the relationship extraction model according to the loss value, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain the relationship extraction model; the relationship extraction model is used to identify the entity relationship of entity pairs in the text sentences.

[0200] In one embodiment, the entity pair includes a head entity and a tail entity; the sample acquisition module 902 is also used to obtain an initial sample text sentence; according to a preset probability value, the head entity or the tail entity is selected from the entity pair of the initial sample text sentence; the selected head entity or the tail entity is replaced with a preset character to obtain a sample text sentence after entity masking.

[0201] In one embodiment, the sample acquisition module 902 is further used to select a head entity or a tail entity from the entity pair of the sample text sentence according to a preset probability value; replace the selected head entity or tail entity with a preset character to obtain a sample text sentence after entity masking.

[0202] In one embodiment, the sampling processing module 904 is further used to generate a sample sentence package based on sample text sentences with the same relationship tags; combine any two sample text sentences in the sample sentence package to obtain a positive sentence pair; and combine any two sample text sentences with different relationship tags to obtain a negative sentence pair.

[0203] In one embodiment, after adjusting the parameters of the relationship extraction model according to the loss value, and before returning to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, the sampling processing module 904 is also used to sample the positive sentence pairs and negative sentence pairs according to a preset positive-negative sampling ratio; and generate a training set based on the sampled positive sentence pairs and negative sentence pairs.

[0204] In one embodiment, the relationship extraction module 906 is also used to input the training set into the relationship extraction model to be trained, extract the semantic relationship features of each sample text sentence in the training set; determine the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair based on the semantic relationship features of each sample text sentence; determine the contrast loss value based on the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair, and generate a loss value based on the contrast loss value.

[0205] In one embodiment, the relationship extraction module 906 is also used to extract the contextual semantic features and entity association features of each sample text sentence in the training set through a relationship extraction model; perform feature splicing on the contextual semantic features and entity association features to obtain spliced ​​joint features; and obtain the semantic relationship features of each sample text sentence based on the spliced ​​joint features.

[0206] In one embodiment, the positive example sentence pairs and the negative example sentence pairs respectively have corresponding sample labels determined according to the relationship labels; the relationship extraction module 906 is also used to generate a contrast loss value based on the difference between the similarity between the sentences in the positive example sentence pairs and the corresponding sample labels, and the difference between the similarity between the sentences in the negative example sentence pairs and the corresponding sample labels.

[0207] In one embodiment, the parameter adjustment module 908 is also used to update the parameters of the relationship extraction model according to the contrast loss value, so that during the iterative training process of the relationship extraction model, the similarity between the positive sample sentence pairs is increased, and the similarity between the negative sample sentence pairs is reduced.

[0208] In one embodiment, the loss value includes a first loss value and a second loss value; the first loss value is a contrast loss value; the parameter adjustment module 908 is also used to mask the text in each sample text sentence in the training set; extract the text context features of each text in each sample text sentence after masking; perform mask prediction on each sample text sentence based on the text context features, and determine the corresponding cross entropy loss based on the mask prediction result; determine the second loss value based on the cross entropy loss.

[0209] In one embodiment, Figure 10 As shown, the above-mentioned text sentence processing device 900 also includes a text processing module 910, which is used to obtain the text sentence to be processed; identify the entity pairs in the text sentence to be processed; input the text sentence to be processed and the entity pair into the relationship extraction model to obtain the entity relationship extraction result of the entity pair in the text sentence to be processed.

[0210] For the specific definition of the text sentence processing device, please refer to the definition of the text sentence processing method above, which will not be repeated here. The various modules in the above-mentioned text sentence processing device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0211] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as sample text sentences, positive sentence pairs and negative sentence pairs. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a text sentence processing method is implemented.

[0212] Those skilled in the art will understand that Figure 10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0213] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0214] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0215] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0216] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0217] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0218] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims. < / e1>

Claims

1. A text sentence processing method, characterized in that: The method comprises: Obtaining a sample text sentence; the sample text sentence contains entity pairs and relationship labels of the entity pairs; some entities selected according to preset probability values ​​in the sample text sentence are masked; Extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and performing positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set; The training set is input into the relation extraction model to be trained to generate a loss value; the loss value includes a first loss value and a second loss value, the first loss value is a contrast loss value; the contrast loss value is used to characterize the difference between the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair; the second loss value is determined according to the cross-entropy loss; the cross-entropy loss is determined according to the corresponding mask prediction result; the mask prediction result is obtained by masking the text in each of the sample text sentences in the training set, extracting the text context features of each text in each of the masked sample text sentences, and then performing mask prediction on each of the sample text sentences according to the text context features; Adjust the parameters of the relationship extraction model according to the loss value, re-mask some entities in the sample text sentence pairs according to the preset probability values, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain a relationship extraction model; the relationship extraction model is used to identify entity relationships between entity pairs in text sentences.

2. The method according to claim 1, characterized in that The entity pair includes a head entity and a tail entity; and obtaining a sample text sentence includes: Get the initial sample text sentence; Selecting a head entity or a tail entity from the entity pairs of the initial sample text sentence according to a preset probability value; The selected head entity or tail entity is replaced with a preset character to obtain a sample text sentence after entity masking.

3. The method according to claim 1, characterized in that The re-masking of some entities in the sample text sentence pair according to the preset probability value includes: Selecting a head entity or a tail entity from the entity pairs of the sample text sentence according to a preset probability value; The selected head entity or tail entity is replaced with a preset character to obtain a sample text sentence after entity masking.

4. The method according to claim 1, wherein The extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels includes: generating a sample sentence package based on the sample text sentences with the same relationship tags; Combining any two sample text sentences in the sample sentence package to obtain a positive sentence pair; Any two sample text sentences with different relationship labels are combined to obtain a negative sentence pair.

5. The method according to claim 1, wherein The performing positive and negative example sampling processing on the positive example sentence pairs and the negative example sentence pairs to obtain a training set includes: Sampling the positive example sentence pairs and the negative example sentence pairs according to a preset positive and negative example sampling ratio; Generate a training set based on the sampled positive and negative sentence pairs.

6. The method according to claim 1, characterized in that Inputting the training set into the relation extraction model to be trained to generate a loss value includes: Inputting the training set into the relation extraction model to be trained, and extracting the semantic relation features of each sample text sentence in the training set; Determining the similarity of the sentences in the positive example sentence pair and the similarity of the sentences in the negative example sentence pair based on the semantic relationship features of the sample text sentences; A contrast loss value is determined according to the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair, and the loss value is generated according to the contrast loss value.

7. The method according to claim 6, characterized in that The extracting of semantic relationship features of each of the sample text sentences in the training set includes: Extracting contextual semantic features and entity association features of each of the sample text sentences in the training set through the relationship extraction model; Performing feature splicing on the context semantic feature and the entity association feature to obtain a spliced ​​joint feature; According to the spliced ​​joint features, the semantic relationship features of each of the sample text sentences are obtained.

8. The method according to claim 6, characterized in that The positive example sentence pair and the negative example sentence pair respectively have corresponding sample labels determined according to the relationship label; The determining of the contrast loss value according to the similarity of the sentences in the positive sentence pair and the similarity of the sentences in the negative sentence pair includes: The contrast loss value is generated according to the difference between the similarity of the sentences in the positive sentence pair and the corresponding sample label, and the difference between the similarity of the sentences in the negative sentence pair and the corresponding sample label.

9. The method according to claim 1, characterized in that The adjusting the parameters of the relationship extraction model according to the loss value includes: The parameters of the relation extraction model are updated according to the contrast loss value, so that the relation extraction model increases the similarity between the positive example sentence pairs and reduces the similarity between the negative example sentence pairs during the iterative training process.

10. The method according to any one of claims 1 to 9, characterized in that After obtaining the relationship extraction model, the method further includes: Get the text sentence to be processed; Identifying entity pairs in the text sentence to be processed; The text sentence to be processed and the entity pair are input into the relationship extraction model to obtain the entity relationship extraction result of the entity pair in the text sentence to be processed.

11. A text sentence processing device, characterized in that: The device comprises: A sample acquisition module is used to acquire a sample text sentence; the sample text sentence includes entity pairs and relationship labels of the entity pairs; some entities selected according to preset probability values ​​in the sample text sentence are masked; a sampling processing module, configured to extract positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels, and perform positive and negative example sampling processing on the positive sentence pairs and the negative sentence pairs to obtain a training set; A relation extraction module, configured to input the training set into a relation extraction model to be trained and generate a loss value; the loss value includes a first loss value and a second loss value, the first loss value being a contrast loss value; the contrast loss value being configured to characterize the difference between the similarity between the sentences in the positive sentence pair and the similarity between the sentences in the negative sentence pair; the second loss value being determined based on a cross-entropy loss; the cross-entropy loss being determined based on a corresponding mask prediction result; the mask prediction result being obtained by masking the text in each of the sample text sentences in the training set, extracting text context features of each of the masked sample text sentences, and then performing mask prediction on each of the sample text sentences based on the text context features; A parameter adjustment module is used to adjust the parameters of the relationship extraction model according to the loss value, re-mask some entities in the sample text sentence pairs according to preset probability values, and return to the step of extracting positive sentence pairs and negative sentence pairs from the sample text sentences according to the relationship labels to perform iterative training until the training stop condition is met to obtain a relationship extraction model; the relationship extraction model is used to identify entity relationships between entity pairs in text sentences.

12. The device according to claim 11, characterized in that The entity pair includes a head entity and a tail entity; the sample acquisition module is also used to obtain an initial sample text sentence; according to a preset probability value, the head entity or the tail entity is selected from the entity pair of the initial sample text sentence; the selected head entity or the tail entity is replaced with a preset character to obtain a sample text sentence after entity masking.

13. The device according to claim 11, characterized in that The parameter adjustment module is further configured to select a head entity or a tail entity from the entity pairs of the sample text sentence according to a preset probability value; and replace the selected head entity or tail entity with a preset character to obtain a sample text sentence after entity masking.

14. The device according to claim 11, characterized in that The sampling processing module is further configured to generate a sample sentence package based on the sample text sentences with the same relational labels; and to combine any two sample text sentences in the sample sentence package to obtain a positive sentence pair; Any two sample text sentences with different relationship labels are combined to obtain a negative sentence pair.

15. The device according to claim 11, characterized in that The sampling processing module is further configured to sample and extract the positive example sentence pairs and the negative example sentence pairs according to a preset positive-negative example sampling ratio; and generate a training set based on the sampled positive example sentence pairs and negative example sentence pairs.

16. The device according to claim 11, characterized in that The relationship extraction module is also used to input the training set into the relationship extraction model to be trained, extract the semantic relationship features of each sample text sentence in the training set; determine the similarity of the sentences in the positive example sentence pair and the similarity of the sentences in the negative example sentence pair based on the semantic relationship features of each sample text sentence; determine the contrast loss value based on the similarity of the sentences in the positive example sentence pair and the similarity of the sentences in the negative example sentence pair, and generate the loss value based on the contrast loss value.

17. The device according to claim 16, characterized in that The relationship extraction module is also used to extract the contextual semantic features and entity association features of each sample text sentence in the training set through the relationship extraction model; perform feature splicing on the contextual semantic features and the entity association features to obtain spliced ​​joint features; and obtain the semantic relationship features of each sample text sentence based on the spliced ​​joint features.

18. The device according to claim 16, characterized in that The positive example sentence pair and the negative example sentence pair respectively have corresponding sample labels determined according to the relationship labels; the relationship extraction module is also used to generate the contrast loss value based on the difference between the similarity between the sentences in the positive example sentence pair and the corresponding sample labels, and the difference between the similarity between the sentences in the negative example sentence pair and the corresponding sample labels.

19. The device according to claim 11, characterized in that The parameter adjustment module is also used to update the parameters of the relationship extraction model according to the contrast loss value, so that the relationship extraction model increases the similarity between the positive example sentence pairs and reduces the similarity between the negative example sentence pairs during the iterative training process.

20. The device according to any one of claims 11 to 19, characterized in that The device also includes a text processing module for obtaining a text sentence to be processed; identifying entity pairs in the text sentence to be processed; and inputting the text sentence to be processed and the entity pairs into the relationship extraction model to obtain an entity relationship extraction result of the entity pairs in the text sentence to be processed.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

22. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

23. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images