Training method, classification method, device, equipment, and medium for relational classification model

Abstract

The present disclosure relates to the field of natural language processing technology, and specifically to a method, apparatus, device, medium, and product for training a relationship classification model. The method comprises: obtaining a training sample set; inputting original text into the relationship classification model to obtain original prediction score information; obtaining a first probability score based on threshold class information and first relationship information, and obtaining a second probability score based on the threshold class information and the first relationship information; obtaining a first prediction loss based on the first probability score and the second probability score; obtaining a target prediction loss based on at least the first prediction loss, and training the relationship classification model based on the target prediction loss. This solution enhances the distinguishability between positive class relationships and threshold classes, and between negative class relationships and threshold classes, thereby ensuring that the trained relationship classification model can accurately identify different relationships and improve the user experience.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular to a training method, classification method, apparatus, device, and medium for a relational classification model. Background Art

[0002] Relation extraction is an important task in information extraction. It can extract structured knowledge used to describe entity relationships from unstructured natural language text, thereby constructing and expanding various knowledge graphs to store rich world knowledge.

[0003] In related technologies, relationship extraction can be performed based on a relationship classification model. Therefore, before performing relationship extraction, the relationship classification model needs to be trained so that it can learn the relationships between entity pairs. Typically, relationship extraction tasks only identify relationships between entity pairs in a sentence. However, with the in-depth practice of relationship extraction technology in actual business, the reasoning scope of traditional relationship extraction has been expanded to the document level. That is, the extraction of relationships requires comprehensive reasoning across multiple sentences, multiple paragraphs, and multiple hops of entities to ensure that the trained relationship classification model can learn a wide variety of relationships between entity pairs, which greatly increases the difficulty of relationship extraction.

[0004] The inventors of the present disclosure have discovered that in related technologies, when training a relationship classification model, the trained relationship classification model is often unable to accurately identify different relationships, thereby damaging the user experience. Summary of the Invention

[0005] In order to solve the problems in the related art, the embodiments of the present disclosure provide a training method, classification method, apparatus, device, and medium for a relationship classification model.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for training a relationship classification model, comprising:

[0007] Acquire a training sample set, where the training sample set includes original text and entity text information, where the entity text information is used to indicate at least one preset entity;

[0008] Inputting the original text into the relationship classification model to obtain original prediction score information, the original prediction score information including threshold class information and at least one relationship information, the relationship information is used to indicate the probability that the corresponding entity pair in the original text belongs to the corresponding relationship, and the threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship;

[0009] Obtaining a first probability score that the target entity pair belongs to the first relationship based on the threshold class information and first relationship information corresponding to the first relationship, and obtaining a second probability score that the target entity pair belongs to the second relationship based on the threshold class information and second relationship information corresponding to the second relationship, the first relationship belongs to a positive class relationship of the target entity pair, the second relationship belongs to a negative class relationship of the target entity pair, and the entities in the target entity pair belong to preset entities;

[0010] Obtain a first prediction loss according to the first probability score and the second probability score;

[0011] A target prediction loss is obtained based on at least the first prediction loss, and a relation classification model is trained based on the target prediction loss.

[0012] In one implementation of the present disclosure, the method further includes:

[0013] Calculating a first information entropy between the first relationship information and the threshold class information according to the first probability score, calculating a second information entropy between the second relationship information and the threshold class information according to the second probability score, and obtaining a second prediction loss according to the first information entropy and the second information entropy;

[0014] Obtain the target prediction loss based on at least the first prediction loss, including:

[0015] A target prediction loss is obtained based on at least the first prediction loss and the second prediction loss.

[0016] In one implementation of the present disclosure, the raw prediction score information includes a raw prediction score vector;

[0017] Obtaining a first probability score of the target entity pair belonging to the first relationship according to the threshold class information and the first relationship information corresponding to the first relationship includes:

[0018] Obtain a first probability score according to the component corresponding to the first relationship in the original prediction score vector, namely the first relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information;

[0019] Obtaining a second probability score of the target entity belonging to the second relationship according to the threshold class information and the second relationship information corresponding to the second relationship includes:

[0020] The second probability score is obtained according to the component corresponding to the second relationship in the original prediction score vector, namely the second relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information.

[0021] In one implementation of the present disclosure,

[0022] Calculating a second information entropy of the second relationship information and the threshold class information according to the second probability score includes:

[0023] Obtaining a second prediction loss based on the first information entropy, the second information entropy, the first temperature adjustment factor, and the second temperature adjustment factor. In one implementation of the present disclosure, obtaining a target prediction loss based on at least the first prediction loss and the second prediction loss includes:

[0024] A third predicted loss is obtained based on the first predicted loss and the second predicted loss, and the target predicted loss is obtained at least based on the third predicted loss.

[0025] In one implementation of the present disclosure, the original text is input into the relational classification model to obtain original prediction score information, including:

[0026] Input the original text into the relation classification model to obtain the original prediction score information and the final representation vector of each entity pair in the original text;

[0027] The method also includes:

[0028] Obtaining a fourth prediction loss based on the number of relational entity pairs in the target text that have at least one identical relationship with the target entity pair, the final representation vector of the relational entity pair, the final representation vector of the target entity pair, the final representation vector of entity pairs in the target text that are different from the target entity pair, and a third temperature adjustment factor;

[0029] Obtaining a fifth prediction loss according to the final representation vector of the target entity pair, the final representation vector of an entity pair in the target text that is different from the target entity pair, and the third temperature adjustment factor;

[0030] Obtaining a sixth predicted loss based on the fourth predicted loss and the fifth predicted loss;

[0031] The obtaining the target prediction loss at least according to the third prediction loss includes:

[0032] A seventh predicted loss is obtained based on the third predicted loss and the sixth predicted loss, and the target predicted loss is obtained based at least on the seventh predicted loss.

[0033] In one implementation of the present disclosure, the method further includes:

[0034] Obtaining a third probability score based on a component of the original prediction score vector corresponding to a random negative class relationship and a component of the original prediction score vector corresponding to a threshold class, wherein the random negative class relationship belongs to a subset randomly selected from the negative class relationships of NA samples, and the NA samples are entity pairs without a predefined relationship;

[0035] Obtaining a third information entropy according to the third probability score, and obtaining an eighth predicted loss according to the third information entropy, the third probability score, and the second temperature adjustment coefficient;

[0036] The obtaining the target predicted loss at least according to the first predicted loss and the second predicted loss includes:

[0037] Obtain a ninth predicted loss based on the eighth predicted loss, the first predicted loss, and the second predicted loss;

[0038] The tenth predicted loss is obtained based on the ninth predicted loss and the sixth predicted loss, and the target predicted loss is obtained based on the tenth predicted loss.

[0039] In a second aspect, an embodiment of the present disclosure provides a relationship classification method, the method comprising:

[0040] Get the text to be processed,

[0041] Acquire a relationship classification model, where the relationship classification model is trained according to the method described in any one of the first aspect or any one of the implementations of the first aspect;

[0042] The text to be processed is input into the relational classification model to obtain target prediction score information output by the relational classification model. The target prediction score information includes target threshold class information and at least one target relationship information. The target relationship information is used to indicate the probability that the corresponding entity in the text to be processed belongs to the corresponding relationship. The target threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship.

[0043] In a third aspect, an embodiment of the present disclosure provides a training device for a relational classification model, comprising:

[0044] A sample set acquisition module is configured to acquire a training sample set, the training sample set including original text and entity text information, the entity text information is used to indicate at least one preset entity;

[0045] a prediction score acquisition module configured to input raw text into the relationship classification model to obtain raw prediction score information, the raw prediction score information including threshold class information and at least one relationship information, the relationship information being used to indicate the probability that a corresponding entity pair in the raw text belongs to a corresponding relationship, and the threshold class information being used to indicate a lower limit of the probability of determining that the entity pair belongs to the relationship;

[0046] a probability score acquisition module, configured to acquire a first probability score that the target entity pair belongs to the first relationship based on the threshold class information and first relationship information corresponding to the first relationship, and to acquire a second probability score that the target entity pair belongs to the second relationship based on the threshold class information and second relationship information corresponding to the second relationship, wherein the first relationship belongs to a positive class relationship of the target entity pair, the second relationship belongs to a negative class relationship of the target entity pair, and the entities in the target entity pair belong to preset entities;

[0047] A predicted loss acquisition module, configured to acquire a first predicted loss according to the first probability score and the second probability score;

[0048] The model training module is configured to obtain a target prediction loss based on at least the first prediction loss, and train a relation classification model based on the target prediction loss.

[0049] In a fourth aspect, an embodiment of the present disclosure provides a relationship classification device, the device comprising:

[0050] The module for obtaining text to be processed is configured to obtain text to be processed.

[0051] A classification model acquisition module is configured to acquire a relationship classification model, where the relationship classification model is trained according to the method of the first aspect or any one of the implementations of the first aspect;

[0052] The relationship classification module is configured to input the text to be processed into the relationship classification model to obtain the target prediction score information output by the relationship classification model. The target prediction score information includes target threshold class information and at least one target relationship information. The target relationship information is used to indicate the probability that the corresponding entity in the text to be processed belongs to the corresponding relationship. The target threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship.

[0053] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement the method steps as described in any one of the first aspect or any implementation manner of the first aspect.

[0054] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored, wherein when the computer instructions are executed by a processor, the method steps as described in any one of the first aspect or any one of the implementations of the first aspect are implemented.

[0055] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method steps as described in any one of the first aspect or any one of the implementations of the first aspect are implemented.

[0056] In the technical solution provided by the present disclosure, a training sample set is obtained, raw text is input into a relation classification model to obtain raw prediction score information, a first probability score of the target entity pair belonging to the first relation is obtained based on threshold class information and first relation information corresponding to the first relation, a second probability score of the target entity pair belonging to the second relation is obtained based on the threshold class information and second relation information corresponding to the second relation, a first prediction loss is obtained based on the first probability score and the second probability score, a target prediction loss is obtained based on at least the first prediction loss, and the relation classification model is trained based on the target prediction loss. In the above solution, when calculating the first probability score of the target entity pair belonging to the first relation in its positive class relations, the first probability score is not calculated based on the probability of the target entity pair belonging to each relation in all positive class relations and the probability of the target entity pair belonging to the threshold class. Instead, the first probability score is calculated based only on the threshold class information and the first relation information corresponding to the first relation. Therefore, even if the probability of the target entity pair belonging to the first relation increases, the probability of the target entity pair belonging to other positive class relations of the target entity pair will not be reduced, nor will a forced ranking be introduced between the probabilities of the target entity pair belonging to different positive class relations, thereby not reducing the distinguishability between the target entity pair belonging to the positive class relation and the target entity pair belonging to the threshold class. Similarly, when calculating the second probability score of the target entity pair belonging to the second relationship in its negative class relationship, the above scheme is not based on the probability of the target entity pair belonging to each relationship in all negative class relationships and the probability of the target entity pair belonging to the threshold class, but only obtains the second probability score based on the threshold class information and the second relationship information corresponding to the second relationship. Therefore, even if the probability of the target entity pair belonging to the second relationship increases, it will not reduce the probability of the target entity pair belonging to other negative class relationships of the target entity pair, nor will it introduce forced sorting between the probabilities of the target entity pair belonging to different negative class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the negative class relationship and the target entity pair belonging to the threshold class. In summary, in the above scheme, when training the relationship classification model, forced sorting between the probabilities of the entity pair belonging to different positive class relationships or forced sorting between the probabilities of the entity pair belonging to different negative class relationships will not be introduced, thereby enhancing the distinguishability of the entity pair belonging to the positive class relationship and the entity pair belonging to the threshold class, as well as the distinguishability of the entity pair belonging to the negative class relationship and the entity pair belonging to the threshold class, thereby ensuring that the trained relationship classification model can accurately identify different relationships and improve the user experience.

[0057] In the technical solution provided in the present disclosure, by calculating the first information entropy of the first relationship information and the threshold class information according to the first probability score, calculating the second information entropy of the second relationship information and the threshold class information according to the second probability score, and obtaining the second prediction loss according to the first information entropy and the second information entropy, at least obtaining the target prediction loss according to the first prediction loss and the second prediction loss, it can be ensured that when the relationship classification model is trained based on the target prediction loss, the information entropy corresponding to each relationship is encouraged to be as small as possible when compared with the threshold class, thereby ensuring that the difference between the probability corresponding to the relationship and the probability corresponding to the threshold class is as large as possible, so that the trained relationship classification model can optimize the difference between the probability of each relationship and the probability corresponding to the threshold class in a more balanced manner, further ensuring that the trained relationship classification model can accurately identify different relationships.

[0058] In the technical solution disclosed herein, the first probability score is obtained by comparing the first relationship with the threshold class pair by pair, and the second probability score is obtained by comparing the second relationship with the threshold class pair by pair. Therefore, the first prediction loss is calculated based on the first probability score corresponding to each relationship in the positive class relationship of the target entity pair and the second probability score corresponding to each relationship in the negative class relationship of the target entity pair. This ensures that when the relationship classification model is trained based on at least the first prediction loss, the learning of the relationship classification model can focus on optimizing the comparison between each relationship class and the threshold class, thereby helping to increase the discrimination between the probability corresponding to the relationship class and the probability corresponding to the threshold class.

[0059] In the technical solution disclosed in the present invention, by introducing the first information entropy corresponding to each relationship in the positive class relationship of the target entity pair and the second information entropy corresponding to each relationship in the negative class relationship of the target entity pair when calculating the second prediction loss, it can be ensured that when the relationship classification model is trained based on at least the target prediction loss obtained according to the first prediction loss and the second prediction loss, it can be encouraged that when the probability of each positive class relationship of the target entity pair and the probability of each negative class relationship are compared with the probability corresponding to the threshold class, the smaller the information entropy of the probability distribution, the better, that is, the greater the difference between the above-mentioned compared probabilities, the better, thereby helping the relationship classification model to balance and optimize the difference between the probability corresponding to each entity relationship and the probability corresponding to the threshold class, and further ensure that the trained relationship classification model can accurately identify different relationships.

[0060] In the technical solution disclosed herein, by calculating the sum of the first prediction loss and the second prediction loss of each entity pair in the original text to obtain the third prediction loss, it can be ensured that when the relationship classification model is trained based on at least the target prediction loss obtained based on the third prediction loss, the learning of the relationship classification model can focus on optimizing the comparison between each relationship class and the threshold class, thereby helping to increase the distinction between the probability corresponding to the relationship class and the probability corresponding to the threshold class. At the same time, it can also enable the relationship classification model to balance the difference between the probability corresponding to each entity relationship and the probability corresponding to the threshold class, thereby ensuring that the trained relationship classification model can accurately identify different relationships.

[0061] In the technical solution disclosed in the present invention, considering that in the document-level relationship extraction dataset, the number of entity pairs corresponding to each type of relationship presents a typical long-tail distribution, wherein the number of entity pairs corresponding to the relationships at the tail accounts for a very small proportion of the number of entity pairs, it is difficult to find two entity pairs with the same relationship label at the tail in the same batch of entity pairs. When faced with such a situation, in order to ensure that the trained relationship classification model can accurately identify different relationships at the tail in the long-tail distribution, by introducing the final representation vector of each entity pair, it can be ensured that when the relationship classification model is trained based on at least the target prediction loss obtained according to the seventh prediction loss, the relationship classification model can learn that the representation vectors of entity pairs belonging to the same relationship gradually approach, while the representation vectors of entity pairs belonging to different relationships gradually move away, thereby helping to enhance the ability of the trained relationship classification model to distinguish different entity pairs.

[0062] In the technical solution disclosed herein, considering that in a document-level relationship extraction dataset, many samples with NA labels, i.e., the relationships of NA samples, are not undefined relationships, but are caused by omissions in the labeling work, such document-level relationship extraction datasets may contain a lot of label noise. If the NA samples are still regarded as correctly labeled samples, it will cause serious misleading to the training of the relationship classification model, so that the trained relationship classification model cannot accurately identify different relationships. In the above solution, by obtaining the third probability score of the entity pair in a subset randomly extracted from the relationship label set of the negative class relationship of the NA sample, and calculating the third information entropy based on the third probability score, calculating the eighth prediction loss based on the third information entropy and the third probability score, calculating the ninth prediction loss based on the eighth prediction loss, the first prediction loss and the second prediction loss, and calculating the tenth prediction loss based on the ninth prediction, it can be ensured that when the relationship classification model is trained based on the target prediction loss obtained according to the tenth prediction loss, some samples in the NA sample will not be incorrectly labeled, and the error penalty is originally a correctly labeled sample, thereby enhancing the resistance of the trained relationship classification model to the noise generated by the incorrect labeling, and helping the trained relationship classification model to accurately identify different relationships.

[0063] In the technical solution provided by the present disclosure, a relation classification model is obtained by obtaining a text to be processed, and the text to be processed is input into the relation classification model to obtain target prediction score information output by the relation classification model. In the process of training the relation classification model used in the above solution, since when calculating the first probability score of the target entity pair belonging to the first relationship in its positive class relationship, it is not based on the probability of the target entity pair belonging to each relationship in all positive class relationships and the probability of the target entity pair belonging to the threshold class, but only based on the threshold class information and the first relationship information corresponding to the first relationship to obtain the first probability score, even if the probability of the target entity pair belonging to the first relationship increases, that is, the probability of the target entity pair belonging to other positive class relationships of the target entity pair will not be reduced, and no forced sorting between the probabilities of the target entity pair belonging to different positive class relationships will be introduced, thereby not reducing the distinguishability of the target entity pair belonging to the positive class relationship and the target entity pair belonging to the threshold class. Similarly, in the process of training the relation classification model used in the above scheme, when calculating the second probability score of the target entity pair belonging to the second relationship in its negative class relationship, it is not based on the probability of the target entity pair belonging to each relationship in all negative class relationships and the probability of the target entity pair belonging to the threshold class, but only based on the threshold class information and the second relationship information corresponding to the second relationship to obtain the second probability score. Therefore, even if the probability of the target entity pair belonging to the second relationship increases, it will not reduce the probability of the target entity pair belonging to other negative class relationships of the target entity pair, nor will it introduce a forced sorting between the probabilities of the target entity pair belonging to different negative class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the negative class relationship and the target entity pair belonging to the threshold class. To summarize, during the training of the relation classification model used in the above scheme, no forced sorting between the probabilities of entity pairs belonging to different positive class relations or the forced sorting between the probabilities of entity pairs belonging to different negative class relations will be introduced, thereby enhancing the distinguishability of entity pairs belonging to positive class relations and entity pairs belonging to threshold classes, as well as the distinguishability of entity pairs belonging to negative class relations and entity pairs belonging to threshold classes, thereby ensuring that different relations in the text to be processed are accurately identified based on the target prediction score information output by the trained relation classification model, thereby improving the user experience.

[0064] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0066] Figure 1 A flowchart of a method for training a relational classification model according to an embodiment of the present disclosure is shown.

[0067] Figure 2 A flowchart of a relationship classification method according to an embodiment of the present disclosure is shown.

[0068] Figure 3 A structural block diagram of a training apparatus for a relational classification model according to an embodiment of the present disclosure is shown.

[0069] Figure 4 A structural block diagram of a relationship classification device according to an embodiment of the present disclosure is shown.

[0070] Figure 5 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0071] Figure 6 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0072] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0073] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof exist or are added.

[0074] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0075] In this disclosure, if it involves operations of obtaining user information or user data or displaying user information or user data to others, such operations are all authorized and confirmed by the user, or actively selected by the user.

[0076] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0077] Relation extraction is a key task in information extraction. It extracts structured knowledge describing relationships between entities from unstructured natural language text, thereby building and expanding a variety of knowledge graphs to store a wealth of world knowledge. As a core component of traditional information extraction tasks, relationship extraction technology has been widely used to support various key business systems and scenarios, such as intelligent customer service question-and-answer systems, automated text comprehension, and knowledge graph construction.

[0078] For example, relationship extraction can be performed on the sentence "The Oriental Pearl Radio and Television Tower is a landmark building in Shanghai" to extract the fact triple "(Oriental Pearl Radio and Television Tower, located in, Shanghai)", where "Oriental Pearl Radio and Television Tower" and "Shanghai" are the head entity and the tail entity respectively, and the relationship between them is "located in".

[0079] In related technologies, relationship extraction can be performed based on a relationship classification model. Therefore, before performing relationship extraction, the relationship classification model needs to be trained so that it can learn the relationship between entity pairs.

[0080] Typically, traditional sentence-level relationship extraction tasks only identify relationships between entity pairs within a sentence. However, as relationship extraction technology becomes more widely used in real-world applications, traditional relationship extraction methods are no longer able to meet the needs of more complex application scenarios. To address this challenge, document-level relationship extraction expands the reasoning scope of traditional relationship extraction to the document level. This means that relationship extraction requires comprehensive reasoning across multiple sentences, paragraphs, and entities across multiple hops. This ensures that the trained relationship classification model can learn a wide variety of relationships between entity pairs (where a pair of entities can correspond to one or more relationships), significantly increasing the difficulty of relationship extraction.

[0081] It should be noted that for an entity pair consisting of a pair of entities, if the relationship labels of the entity pair are in the predefined relationship label set, the entity pair can be called a positive sample, otherwise the entity pair is called a negative sample. The entity pairs in the negative sample can be understood as having no relationship (no relation), that is, NA relationship. For each sample, that is, an entity pair, we call all the relations corresponding to the entity pair the positive relations of the entity pair, and all relations other than all the relations corresponding to the entity pair are called negative relations of the entity pair.

[0082] In one technical solution, relationship extraction can be performed based on an adaptive thresholding loss function (ATLOP) model. In the process of training the ATLOP model, the probabilities of all positive class relationships and threshold classes can be calculated based on a normalized exponential function (softmax function), that is, based on the probability that a certain entity pair belongs to each relationship in all positive class relationships and the probability that the entity pair belongs to the threshold class, the probability that the entity pair belongs to a specific positive class relationship is calculated, and the corresponding prediction loss is obtained based on the obtained probability, so as to further train the ATLOP model. Among them, the probability that the entity pair belongs to the threshold class can be understood as determining the lower limit of the probability that the entity pair belongs to any relationship.

[0083] The inventors of this application discovered that in this scheme, since the probability of an entity pair belonging to a specific positive class relationship is calculated based on the probability of the entity pair belonging to each relationship in all positive class relationships and the probability of the entity pair belonging to the threshold class, the sum of the normalized probability of the entity pair belonging to each relationship in all positive class relationships and the probability of the entity pair belonging to the threshold class is limited to 1. As a result, when the probability of the entity pair belonging to a certain positive class relationship is increased, the probability of the entity pair belonging to other positive class relationships will inevitably be reduced. Therefore, the above scheme introduces a forced sorting between the probabilities of entity pairs belonging to different positive class relationships, and further reduces the distinguishability of entity pairs belonging to positive class relationships and entity pairs belonging to threshold classes, so that the trained ATLOP model is often unable to accurately identify different relationships, thereby damaging the user experience.

[0084] Taking into account the shortcomings of the above schemes, the inventors of the present disclosure proposed a new scheme: by obtaining a training sample set, inputting the original text into the relationship classification model to obtain the original prediction score information, obtaining the first probability score of the target entity belonging to the first relationship based on the threshold class information and the first relationship information corresponding to the first relationship, and obtaining the second probability score of the target entity belonging to the second relationship based on the threshold class information and the second relationship information corresponding to the second relationship, obtaining the first prediction loss based on the first probability score and the second probability score, obtaining the target prediction loss at least based on the first prediction loss, and training the relationship classification model based on the target prediction loss. Among them, when calculating the first probability score of the target entity pair belonging to the first relationship in its positive class relationship, the above scheme is not based on the probability of the target entity pair belonging to each relationship in all positive class relationships and the probability of the target entity pair belonging to the threshold class, but only obtains the first probability score based on the threshold class information and the first relationship information corresponding to the first relationship. Therefore, even if the probability of the target entity pair belonging to the first relationship increases, it will not reduce the probability of the target entity pair belonging to other positive class relationships of the target entity pair, nor will it introduce a forced sorting between the probabilities of the target entity pair belonging to different positive class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the positive class relationship and the target entity pair belonging to the threshold class. Similarly, when calculating the second probability score of the target entity pair belonging to the second relationship in its negative class relationship, the above scheme is not based on the probability of the target entity pair belonging to each relationship in all negative class relationships and the probability of the target entity pair belonging to the threshold class, but only obtains the second probability score based on the threshold class information and the second relationship information corresponding to the second relationship. Therefore, even if the probability of the target entity pair belonging to the second relationship increases, it will not reduce the probability of the target entity pair belonging to other negative class relationships of the target entity pair, nor will it introduce forced sorting between the probabilities of the target entity pair belonging to different negative class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the negative class relationship and the target entity pair belonging to the threshold class. In summary, in the above scheme, when training the relationship classification model, forced sorting between the probabilities of the entity pair belonging to different positive class relationships or forced sorting between the probabilities of the entity pair belonging to different negative class relationships will not be introduced, thereby enhancing the distinguishability of the entity pair belonging to the positive class relationship and the entity pair belonging to the threshold class, as well as the distinguishability of the entity pair belonging to the negative class relationship and the entity pair belonging to the threshold class, thereby ensuring that the trained relationship classification model can accurately identify different relationships and improve the user experience.

[0085] Figure 1 A flowchart illustrating a method for training a relational classification model according to an embodiment of the present disclosure is provided. The training method can be applied to servers or terminal devices, where the servers may include data processing servers such as cloud servers and distributed servers, and the terminal devices may include smart mobile communication terminals, tablet computers, laptop computers, desktop computers, wearable devices, and the like.

[0086] like Figure 1 As shown, the training method of the relationship classification model includes the following steps S101-S105:

[0087] In step S101, a training sample set is obtained;

[0088] The training sample set includes original text and entity text information, and the entity text information is used to indicate at least one preset entity.

[0089] In step S102, the original text is input into the relational classification model to obtain original prediction score information;

[0090] The original prediction score information includes threshold class information and at least one relationship information, the relationship information is used to indicate the probability that the corresponding entity pair in the original text belongs to the corresponding relationship, and the threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship;

[0091] In step S103, a first probability score of the target entity pair belonging to the first relationship is obtained based on the threshold class information and the first relationship information corresponding to the first relationship, and a second probability score of the target entity pair belonging to the second relationship is obtained based on the threshold class information and the second relationship information corresponding to the second relationship;

[0092] The first relationship belongs to a positive relationship of the target entity pair, the second relationship belongs to a negative relationship of the target entity pair, and the entities in the target entity pair belong to preset entities.

[0093] In step S104, a first predicted loss is obtained according to the first probability score and the second probability score;

[0094] In step S105, a target prediction loss is obtained based on at least the first prediction loss, and a relational classification model is trained based on the target prediction loss.

[0095] In one embodiment of the present disclosure, when the above-mentioned training method is applied to a server, obtaining a training sample set can be understood as reading a pre-stored training sample set, or as receiving a training sample set sent by a terminal device that matches the server through a wired or wireless network. When the above-mentioned training method is applied to a terminal device, obtaining a training sample set can be understood as reading a pre-stored training sample set, or as receiving a training sample set sent by a server that matches the terminal device through a wired or wireless network, or as obtaining an input training sample set by a human-computer interaction device on the terminal device (such as a touch screen, microphone, etc.) or a human-computer interaction device connected to the terminal device (such as a keyboard, mouse, etc.). The present disclosure does not specifically limit the implementation method of obtaining a training sample set.

[0096] In one embodiment of the present disclosure, the original text can be understood as a text including at least one paragraph of text, which may include multiple entities, where the entities can be accumulated into words, such as "Oriental Pearl Radio and Television Tower", "Shanghai", "Beijing", "China", etc.

[0097] In one embodiment of the present disclosure, entity text information may be understood as text including multiple preset entities, or may be understood as a form, a database, etc. storing multiple preset entities.

[0098] In one embodiment of the present disclosure, the relational classification model can be understood as an ATLOP model, or as a neural network (NN) model, a convolutional neural network (CNN) model, or a long short-term memory network (LSTM) model.

[0099] In one embodiment of the present disclosure, the raw prediction score information can be understood as a raw prediction score vector, wherein the raw prediction score vector may include components corresponding to multiple relationships and threshold classes, wherein the components corresponding to the relationships can be understood as relationship information, and the components corresponding to the threshold classes can be understood as threshold class information; or, the raw prediction score information can also be understood as a table or a database, which can store threshold class information and at least one relationship information.

[0100] In one embodiment of the present disclosure, obtaining a first probability score based on threshold-type information and first relationship information can be understood as substituting the threshold-type information and the first relationship information into a calculation based on a pre-acquired first probability score algorithm to obtain the first probability score; it can also be understood as inputting the threshold-type information and the first relationship information into a pre-acquired first probability score model to obtain the first probability score output by the first probability score model.

[0101] Obtaining the second probability score based on the threshold-type information and the second relationship information can be understood as substituting the threshold-type information and the second relationship information into a calculation based on a pre-acquired second probability score algorithm to obtain the second probability score; it can also be understood as inputting the threshold-type information and the second relationship information into a pre-acquired second probability score model to obtain the second probability score output by the second probability score model.

[0102] In one embodiment of the present disclosure, a positive relationship of a target entity pair can be understood as a relationship that has been predefined among the relationships corresponding to the target entity pair. Conversely, a negative relationship of a target entity pair can be understood as a relationship that has not been predefined among the relationships corresponding to the target entity pair.

[0103] In one embodiment of the present disclosure, obtaining the first predicted loss based on the first probability score and the second probability score can be understood as substituting a probability score and a second probability score into a calculation based on a pre-acquired first prediction loss algorithm to obtain the first predicted loss; it can also be understood as inputting the first probability score and the second probability score into a pre-acquired first prediction loss model to obtain the first predicted loss output by the first prediction loss model.

[0104] In one embodiment of the present disclosure, obtaining the target prediction loss at least based on the first prediction loss can be understood as determining the first prediction loss as the target prediction loss, or can be understood as substituting the first prediction loss into the calculation according to the pre-acquired first target prediction loss algorithm to obtain the target prediction loss.

[0105] In the technical solution provided by the present disclosure, a training sample set is obtained, raw text is input into a relation classification model to obtain raw prediction score information, a first probability score of the target entity pair belonging to the first relation is obtained based on threshold class information and first relation information corresponding to the first relation, a second probability score of the target entity pair belonging to the second relation is obtained based on the threshold class information and second relation information corresponding to the second relation, a first prediction loss is obtained based on the first probability score and the second probability score, a target prediction loss is obtained based on at least the first prediction loss, and the relation classification model is trained based on the target prediction loss. In the above solution, when calculating the first probability score of the target entity pair belonging to the first relation in its positive class relations, the first probability score is not calculated based on the probability of the target entity pair belonging to each relation in all positive class relations and the probability of the target entity pair belonging to the threshold class. Instead, the first probability score is calculated based only on the threshold class information and the first relation information corresponding to the first relation. Therefore, even if the probability of the target entity pair belonging to the first relation increases, the probability of the target entity pair belonging to other positive class relations of the target entity pair will not be reduced, nor will a forced ranking be introduced between the probabilities of the target entity pair belonging to different positive class relations, thereby not reducing the distinguishability between the target entity pair belonging to the positive class relation and the target entity pair belonging to the threshold class. Similarly, when calculating the second probability score of the target entity pair belonging to the second relationship in its negative class relationship, the above scheme is not based on the probability of the target entity pair belonging to each relationship in all negative class relationships and the probability of the target entity pair belonging to the threshold class, but only obtains the second probability score based on the threshold class information and the second relationship information corresponding to the second relationship. Therefore, even if the probability of the target entity pair belonging to the second relationship increases, it will not reduce the probability of the target entity pair belonging to other negative class relationships of the target entity pair, nor will it introduce forced sorting between the probabilities of the target entity pair belonging to different negative class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the negative class relationship and the target entity pair belonging to the threshold class. In summary, in the above scheme, when training the relationship classification model, forced sorting between the probabilities of the entity pair belonging to different positive class relationships or forced sorting between the probabilities of the entity pair belonging to different negative class relationships will not be introduced, thereby enhancing the distinguishability of the entity pair belonging to the positive class relationship and the entity pair belonging to the threshold class, as well as the distinguishability of the entity pair belonging to the negative class relationship and the entity pair belonging to the threshold class, thereby ensuring that the trained relationship classification model can accurately identify different relationships and improve the user experience.

[0106] In one implementation of the present disclosure, the method further includes the following steps:

[0107] Calculating a first information entropy between the first relationship information and the threshold class information according to the first probability score, calculating a second information entropy between the second relationship information and the threshold class information according to the second probability score, and obtaining a second prediction loss according to the first information entropy and the second information entropy;

[0108] In step S105, obtaining the target prediction loss based on at least the first prediction loss can be achieved by the following steps:

[0109] A target prediction loss is obtained based on at least the first prediction loss and the second prediction loss.

[0110] In one embodiment of the present disclosure, the first information entropy can be understood as the uncertainty between the probability that the corresponding entity pair indicated by the first target entity belongs to the corresponding relationship and the lower limit of the probability of determining that the entity pair belongs to the relationship. The larger the first information entropy, the higher the uncertainty.

[0111] In one embodiment of the present disclosure, calculating the first information entropy of the first relationship information and the threshold type information according to the first probability score can be understood as substituting the first probability score into the calculation according to the pre-acquired first information entropy algorithm to obtain the first information entropy; it can also be understood as inputting the first probability score into the pre-acquired first information entropy model to obtain the first information entropy output by the first information entropy model.

[0112] In one embodiment of the present disclosure, the second information entropy can be understood as the uncertainty between the probability that the corresponding entity pair indicated by the second target entity belongs to the corresponding relationship and the lower limit of the probability of determining that the entity pair belongs to the relationship. The larger the second information entropy, the higher the uncertainty.

[0113] In one embodiment of the present disclosure, calculating the second information entropy of the second relationship information and the threshold type information according to the second probability score can be understood as substituting the second probability score into the calculation according to the pre-acquired second information entropy algorithm to obtain the second information entropy; it can also be understood as inputting the second probability score into the pre-acquired second information entropy model to obtain the second information entropy output by the second information entropy model.

[0114] In one embodiment of the present disclosure, obtaining the second prediction loss based on the first information entropy and the second information entropy can be understood as substituting the first information entropy and the second information entropy into the calculation according to the pre-acquired second prediction loss algorithm to obtain the second prediction loss; it can also be understood as inputting the first information entropy and the second information entropy into the pre-acquired second prediction loss model to obtain the second prediction loss output by the second prediction loss model.

[0115] In one embodiment of the present disclosure, obtaining the target prediction loss based on at least the first prediction loss and the second prediction loss can be understood as substituting the first prediction loss and the second prediction loss into the calculation according to the pre-acquired second prediction loss algorithm to obtain the target prediction loss; it can also be understood as inputting the first prediction loss and the second prediction loss into the pre-acquired target prediction loss model to obtain the target prediction loss output by the target prediction loss model.

[0116] In the technical solution provided in the present disclosure, by calculating the first information entropy of the first relationship information and the threshold class information according to the first probability score, calculating the second information entropy of the second relationship information and the threshold class information according to the second probability score, and obtaining the second prediction loss according to the first information entropy and the second information entropy, at least obtaining the target prediction loss according to the first prediction loss and the second prediction loss, it can be ensured that when the relationship classification model is trained based on the target prediction loss, the information entropy corresponding to each relationship is encouraged to be as small as possible when compared with the threshold class, thereby ensuring that the difference between the probability corresponding to the relationship and the probability corresponding to the threshold class is as large as possible, so that the trained relationship classification model can optimize the difference between the probability of each relationship and the probability corresponding to the threshold class in a more balanced manner, further ensuring that the trained relationship classification model can accurately identify different relationships.

[0117] In one implementation of the present disclosure, the raw prediction score information includes a raw prediction score vector;

[0118] Obtaining a first probability score of the target entity belonging to the first relationship based on the threshold class information and the first relationship information corresponding to the first relationship can be achieved through the following steps:

[0119] Obtain a first probability score according to the component corresponding to the first relationship in the original prediction score vector, namely the first relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information;

[0120] Specifically, according to Get the first probability score

[0121] Where h,t is the target entity, r1 is the relationship label with the first relationship and r1∈Ρ h,t , f r1 is the component corresponding to the first relationship in the original prediction score vector, i.e., the first relationship information, f η is the component corresponding to the threshold class in the original prediction score vector, i.e., the threshold class information;

[0122] Obtaining a second probability score of the target entity belonging to the second relationship based on the threshold class information and the second relationship information corresponding to the second relationship can be achieved through the following steps:

[0123] The second probability score is obtained according to the component corresponding to the second relationship in the original prediction score vector, namely the second relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information.

[0124] Specifically, according to Get the second probability score

[0125] Where r2 is the relation label of the second relation, and r2∈N h,t , f r2 is the component in the original prediction score vector corresponding to the second relationship;

[0126] Can be based on Get the first prediction loss

[0127] where Q h,t is the relation label set of the positive relation of the target entity pair, N h,t is the set of relation labels for the negative class relations of the target entity pair.

[0128] In the technical solution disclosed herein, the first probability score is obtained by comparing the first relationship with the threshold class pair by pair, and the second probability score is obtained by comparing the second relationship with the threshold class pair by pair. Therefore, the first prediction loss is calculated based on the first probability score corresponding to each relationship in the positive class relationship of the target entity pair and the second probability score corresponding to each relationship in the negative class relationship of the target entity pair. This ensures that when the relationship classification model is trained based on at least the first prediction loss, the learning of the relationship classification model can focus on optimizing the comparison between each relationship class and the threshold class, thereby helping to increase the discrimination between the probability corresponding to the relationship class and the probability corresponding to the threshold class.

[0129] In one implementation of the present disclosure, obtaining the second prediction loss according to the first information entropy and the second information entropy can be achieved through the following steps:

[0130] A second prediction loss is obtained according to the first information entropy, the second information entropy, the first temperature adjustment factor, and the second temperature adjustment factor.

[0131] Specifically, according to Get the first information entropy H h,t (r1);

[0132] Can be based on Get the second information entropy H h,t (r2);

[0133] Can be based on Get the second prediction loss

[0134] Wherein γ1 is the first temperature adjustment factor, and γ2 is the second temperature adjustment factor.

[0135] In one embodiment of the present disclosure, the first temperature adjustment factor and the second temperature adjustment factor may be acquired in advance, or may be input through a corresponding human-computer interaction device.

[0136] In the technical solution disclosed in the present invention, by introducing the first information entropy corresponding to each relationship in the positive class relationship of the target entity pair and the second information entropy corresponding to each relationship in the negative class relationship of the target entity pair when calculating the second prediction loss, it can be ensured that when the relationship classification model is trained based on at least the target prediction loss obtained according to the first prediction loss and the second prediction loss, it can be encouraged that when the probability of each positive class relationship of the target entity pair and the probability of each negative class relationship are compared with the probability corresponding to the threshold class, the smaller the information entropy of the probability distribution, the better, that is, the greater the difference between the above-mentioned compared probabilities, the better, thereby helping the relationship classification model to balance and optimize the difference between the probability corresponding to each entity relationship and the probability corresponding to the threshold class, and further ensure that the trained relationship classification model can accurately identify different relationships.

[0137] In one implementation of the present disclosure, obtaining the target prediction loss based on at least the first prediction loss and the second prediction loss can be achieved through the following steps:

[0138] A third prediction loss is obtained based on the first prediction loss and the second prediction loss, and a target prediction loss is obtained based at least on the third prediction loss.

[0139] Specifically, according to Obtain a third prediction loss L1, and obtain a target prediction loss at least based on the third prediction loss L1, where B is a set of entity pairs in the original text.

[0140] In one embodiment of the present disclosure, obtaining the target prediction loss based on the third prediction loss can be understood as determining the third prediction loss as the target prediction loss, or it can be understood as substituting the third prediction loss into the pre-acquired second target prediction loss algorithm for calculation to obtain the target prediction loss.

[0141] In the technical solution disclosed herein, by calculating the sum of the first prediction loss and the second prediction loss of each entity pair in the original text to obtain the third prediction loss, it can be ensured that when the relationship classification model is trained based on at least the target prediction loss obtained based on the third prediction loss, the learning of the relationship classification model can focus on optimizing the comparison between each relationship class and the threshold class, thereby helping to increase the distinction between the probability corresponding to the relationship class and the probability corresponding to the threshold class. At the same time, it can also enable the relationship classification model to balance the difference between the probability corresponding to each entity relationship and the probability corresponding to the threshold class, thereby ensuring that the trained relationship classification model can accurately identify different relationships.

[0142] In one implementation of the present disclosure, inputting the original text into the relational classification model to obtain the original prediction score information can be achieved through the following steps:

[0143] Input the original text into the relation classification model to obtain the original prediction score information and the final representation vector of each entity pair in the original text;

[0144] The above method may further comprise the following steps:

[0145] Obtaining a fourth prediction loss based on the number of relational entity pairs in the target text that have at least one identical relationship with the target entity pair, the final representation vector of the relational entity pair, the final representation vector of the target entity pair, the final representation vector of the entity pairs in the target text that are different from the target entity pair, and the third temperature adjustment factor;

[0146] Obtaining a fifth prediction loss based on a final representation vector of the target entity pair, a final representation vector of entity pairs in the target text that are different from the target entity pair, and a third temperature adjustment factor;

[0147] Obtaining a sixth predicted loss based on the fourth predicted loss and the fifth predicted loss;

[0148] Obtain the target prediction loss based on at least the third prediction loss, including:

[0149] A seventh predicted loss is obtained based on the third predicted loss and the sixth predicted loss, and a target predicted loss is obtained based at least on the seventh predicted loss.

[0150] Specifically, according to Get the fourth prediction loss

[0151] Among them S h,t is the set of entity pairs in the target text that have at least one common relation with the target entity pair, |S h,t |For S h,t The number of entity pairs in S, p is h,t The entity pair index of the entity pair in , xh,t is the final representation vector of the target entity pair, x p For S h,t The final representation vector of the entity pair in B, t is the third temperature adjustment factor, and d is the entity pair in B that is different from the target entity pair;

[0152] Can be based on Get the fifth prediction loss

[0153] Can be based on Get the sixth prediction loss L2;

[0154] Among them, B′ is the set of all positive samples in B;

[0155] The seventh prediction loss L can be obtained according to L=L1+λL2, and the target prediction loss can be obtained at least according to the seventh prediction loss L, where λ is a weight factor.

[0156] In one implementation of the present disclosure, It is an operator of the indicator function. When the condition in {} is met, the output of the indicator function is 1; when the condition in {} is not met, the output of the indicator function is 0.

[0157] In one embodiment of the present disclosure, the weight factor and the third temperature adjustment factor may be acquired in advance, or may be input through a corresponding human-computer interaction device.

[0158] In the technical solution disclosed in the present invention, considering that in the document-level relationship extraction dataset, the number of entity pairs corresponding to each type of relationship presents a typical long-tail distribution, wherein the number of entity pairs corresponding to the relationships at the tail accounts for a very small proportion of the number of entity pairs, it is difficult to find two entity pairs with the same relationship label at the tail in the same batch of entity pairs. When faced with such a situation, in order to ensure that the trained relationship classification model can accurately identify different relationships at the tail in the long-tail distribution, by introducing the final representation vector of each entity pair, it can be ensured that when the relationship classification model is trained based on at least the target prediction loss obtained according to the seventh prediction loss, the relationship classification model can learn that the representation vectors of entity pairs belonging to the same relationship gradually approach, while the representation vectors of entity pairs belonging to different relationships gradually move away, thereby helping to enhance the ability of the trained relationship classification model to distinguish different entity pairs.

[0159] In one implementation of the present disclosure, the method further includes the following steps:

[0160] Obtaining a third probability score based on a component in the original prediction score vector corresponding to a random negative class relationship and a component in the original prediction score vector corresponding to a threshold class, where the random negative class relationship belongs to a subset randomly selected from the negative class relationships of NA samples, where the NA samples are entity pairs without a predefined relationship;

[0161] Obtaining a third information entropy according to the third probability score, and obtaining an eighth predicted loss according to the third information entropy, the third probability score, and the second temperature adjustment coefficient;

[0162] Obtaining a target prediction loss based on at least the first prediction loss and the second prediction loss includes:

[0163] Obtain a ninth predicted loss based on the eighth predicted loss, the first predicted loss, and the second predicted loss;

[0164] The tenth predicted loss is obtained according to the ninth predicted loss and the sixth predicted loss, and the target predicted loss is obtained according to the tenth predicted loss.

[0165] Specifically, according to Get the third probability score

[0166] Among them, r is the h ' ,t The relationship label, N h ' ,t is a subset randomly selected from the set of relationship labels of negative class relationships of NA samples. NA samples are entity pairs without predefined relationships. r is the component corresponding to r in the original prediction score vector;

[0167] Can be based on Get the third information entropy H h,t (r);

[0168] Can be based on Get the eighth prediction loss L′;

[0169] Among them, B N is the set of all negative samples in B;

[0170] Can be based on Get the ninth prediction loss

[0171] Can be based on Get the tenth prediction loss L NA , and according to the tenth prediction loss L NA Get the target prediction loss.

[0172] In one embodiment of the present disclosure, obtaining the target prediction loss based on the tenth prediction loss can be understood as determining the tenth prediction loss as the target prediction loss, or it can be understood as substituting the tenth prediction loss into the pre-acquired third target prediction loss algorithm for calculation to obtain the target prediction loss.

[0173] In the technical solution disclosed herein, considering that in a document-level relationship extraction dataset, many samples with NA labels, i.e., the relationships of NA samples, are not undefined relationships, but are caused by omissions in the labeling work, such document-level relationship extraction datasets may contain a lot of label noise. If the NA samples are still regarded as correctly labeled samples, it will cause serious misleading to the training of the relationship classification model, so that the trained relationship classification model cannot accurately identify different relationships. In the above solution, by obtaining the third probability score of the entity pair in a subset randomly extracted from the relationship label set of the negative class relationship of the NA sample, and calculating the third information entropy based on the third probability score, calculating the eighth prediction loss based on the third information entropy and the third probability score, calculating the ninth prediction loss based on the eighth prediction loss, the first prediction loss and the second prediction loss, and calculating the tenth prediction loss based on the ninth prediction, it can be ensured that when the relationship classification model is trained based on the target prediction loss obtained according to the tenth prediction loss, some samples in the NA sample will not be incorrectly labeled, and the error penalty is originally a correctly labeled sample, thereby enhancing the resistance of the trained relationship classification model to the noise generated by the incorrect labeling, and helping the trained relationship classification model to accurately identify different relationships.

[0174] Figure 2 A flowchart of a relationship classification method according to an embodiment of the present disclosure is shown. The relationship classification method can be applied to servers or terminal devices, where the servers may include data processing servers such as cloud servers and distributed servers, and the terminal devices may include smart mobile communication terminals, tablet computers, laptop computers, desktop computers, wearable devices, etc.

[0175] like Figure 2 As shown, the training method of the relationship classification model includes the following steps S201-S203:

[0176] In step S201, the text to be processed is obtained;

[0177] In step S202, a relationship classification model is obtained;

[0178] Wherein, the relation classification model is obtained by training according to any of the above relation classification models;

[0179] In step S203, the text to be processed is input into the relation classification model to obtain target prediction score information output by the relation classification model;

[0180] Among them, the target prediction score information includes target threshold class information and at least one target relationship information. The target relationship information is used to indicate the probability that the corresponding entity in the text to be processed belongs to the corresponding relationship, and the target threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship.

[0181] In one embodiment of the present disclosure, when the above-mentioned relationship classification method is applied to a server, obtaining the text to be processed can be understood as reading the text to be processed stored in advance, or can be understood as receiving the text to be processed sent by a terminal device that matches the server through a wired or wireless network. When the above-mentioned relationship classification method is applied to a terminal device, obtaining the text to be processed can be understood as reading the text to be processed stored in advance, or receiving the text to be processed sent by a server that matches the terminal device through a wired or wireless network, or can be understood as the human-computer interaction device on the terminal device (such as a touch screen, microphone, etc.) or the human-computer interaction device connected to the terminal device (such as a keyboard, mouse, etc.) obtaining the input text to be processed. The present disclosure does not specifically limit the implementation method of obtaining the text to be processed.

[0182] In one embodiment of the present disclosure, the text to be processed can be understood as a text including at least one paragraph of text, which may include multiple entities, where the entities can be accumulated into words, such as "Oriental Pearl Radio and Television Tower", "Shanghai", "Beijing", "China", etc.

[0183] In one embodiment of the present disclosure, the relational classification model can be understood as an ATLOP model, or as a neural network (NN) model, a convolutional neural network (CNN) model, or a long short-term memory network (LSTM) model.

[0184] In one embodiment of the present disclosure, the target prediction score information can be understood as a target prediction score vector, wherein the target prediction score vector may include components corresponding to multiple relationships and threshold classes, wherein the components corresponding to the relationships can be understood as target relationship information, and the components corresponding to the threshold classes can be understood as target threshold class information; or, the target prediction score information can also be understood as a form or database, which can store target threshold class information and at least one target relationship information.

[0185] In the technical solution provided by the present disclosure, a relation classification model is obtained by obtaining a text to be processed, and the text to be processed is input into the relation classification model to obtain target prediction score information output by the relation classification model. In the process of training the relation classification model used in the above solution, since when calculating the first probability score of the target entity pair belonging to the first relationship in its positive class relationship, it is not based on the probability of the target entity pair belonging to each relationship in all positive class relationships and the probability of the target entity pair belonging to the threshold class, but only based on the threshold class information and the first relationship information corresponding to the first relationship to obtain the first probability score, even if the probability of the target entity pair belonging to the first relationship increases, that is, the probability of the target entity pair belonging to other positive class relationships of the target entity pair will not be reduced, and no forced sorting between the probabilities of the target entity pair belonging to different positive class relationships will be introduced, thereby not reducing the distinguishability of the target entity pair belonging to the positive class relationship and the target entity pair belonging to the threshold class. Similarly, in the process of training the relation classification model used in the above scheme, when calculating the second probability score of the target entity pair belonging to the second relationship in its negative class relationship, it is not based on the probability of the target entity pair belonging to each relationship in all negative class relationships and the probability of the target entity pair belonging to the threshold class, but only based on the threshold class information and the second relationship information corresponding to the second relationship to obtain the second probability score. Therefore, even if the probability of the target entity pair belonging to the second relationship increases, it will not reduce the probability of the target entity pair belonging to other negative class relationships of the target entity pair, nor will it introduce a forced sorting between the probabilities of the target entity pair belonging to different negative class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the negative class relationship and the target entity pair belonging to the threshold class. To summarize, during the training of the relation classification model used in the above scheme, no forced sorting between the probabilities of entity pairs belonging to different positive class relations or the forced sorting between the probabilities of entity pairs belonging to different negative class relations will be introduced, thereby enhancing the distinguishability of entity pairs belonging to positive class relations and entity pairs belonging to threshold classes, as well as the distinguishability of entity pairs belonging to negative class relations and entity pairs belonging to threshold classes, thereby ensuring that different relations in the text to be processed are accurately identified based on the target prediction score information output by the trained relation classification model, thereby improving the user experience.

[0186] Figure 3 A structural block diagram of a training device for a relational classification model according to an embodiment of the present disclosure is shown, wherein the device can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0187] like Figure 3 As shown, the training device 200 for the relationship classification model includes:

[0188] The sample set acquisition module 201 is configured to acquire a training sample set, where the training sample set includes original text and entity text information, where the entity text information is used to indicate at least one preset entity;

[0189] Prediction score acquisition module 202 is configured to input raw text into the relationship classification model to obtain raw prediction score information, where the raw prediction score information includes threshold class information and at least one relationship information, where the relationship information indicates the probability that a corresponding entity pair in the raw text belongs to a corresponding relationship, and the threshold class information indicates a lower limit of the probability of determining that the entity pair belongs to a relationship;

[0190] The probability score acquisition module 203 is configured to acquire a first probability score that the target entity pair belongs to the first relationship based on the threshold class information and the first relationship information corresponding to the first relationship, and acquire a second probability score that the target entity pair belongs to the second relationship based on the threshold class information and the second relationship information corresponding to the second relationship, wherein the first relationship belongs to a positive class relationship of the target entity pair, the second relationship belongs to a negative class relationship of the target entity pair, and the entities in the target entity pair belong to preset entities;

[0191] The predicted loss acquisition module 204 is configured to acquire a first predicted loss according to the first probability score and the second probability score;

[0192] The model training module 205 is configured to obtain a target prediction loss based on at least the first prediction loss, and train a relationship classification model based on the target prediction loss.

[0193] In the technical solution provided by the present disclosure, a training sample set is obtained, raw text is input into a relation classification model to obtain raw prediction score information, a first probability score of the target entity pair belonging to the first relation is obtained based on threshold class information and first relation information corresponding to the first relation, a second probability score of the target entity pair belonging to the second relation is obtained based on the threshold class information and second relation information corresponding to the second relation, a first prediction loss is obtained based on the first probability score and the second probability score, a target prediction loss is obtained based on at least the first prediction loss, and the relation classification model is trained based on the target prediction loss. In the above solution, when calculating the first probability score of the target entity pair belonging to the first relation in its positive class relations, the first probability score is not calculated based on the probability of the target entity pair belonging to each relation in all positive class relations and the probability of the target entity pair belonging to the threshold class. Instead, the first probability score is calculated based only on the threshold class information and the first relation information corresponding to the first relation. Therefore, even if the probability of the target entity pair belonging to the first relation increases, the probability of the target entity pair belonging to other positive class relations of the target entity pair will not be reduced, nor will a forced ranking be introduced between the probabilities of the target entity pair belonging to different positive class relations, thereby not reducing the distinguishability between the target entity pair belonging to the positive class relation and the target entity pair belonging to the threshold class. Similarly, when calculating the second probability score of the target entity pair belonging to the second relationship in its negative class relationship, the above scheme is not based on the probability of the target entity pair belonging to each relationship in all negative class relationships and the probability of the target entity pair belonging to the threshold class, but only obtains the second probability score based on the threshold class information and the second relationship information corresponding to the second relationship. Therefore, even if the probability of the target entity pair belonging to the second relationship increases, it will not reduce the probability of the target entity pair belonging to other negative class relationships of the target entity pair, nor will it introduce forced sorting between the probabilities of the target entity pair belonging to different negative class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the negative class relationship and the target entity pair belonging to the threshold class. In summary, in the above scheme, when training the relationship classification model, forced sorting between the probabilities of the entity pair belonging to different positive class relationships or forced sorting between the probabilities of the entity pair belonging to different negative class relationships will not be introduced, thereby enhancing the distinguishability of the entity pair belonging to the positive class relationship and the entity pair belonging to the threshold class, as well as the distinguishability of the entity pair belonging to the negative class relationship and the entity pair belonging to the threshold class, thereby ensuring that the trained relationship classification model can accurately identify different relationships and improve the user experience.

[0194] Figure 4 The following is a structural block diagram of a relationship classification device according to an embodiment of the present disclosure, wherein the device can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0195] like Figure 4 As shown, the training device 300 for the relationship classification model includes:

[0196] The to-be-processed text acquisition module 301 is configured to acquire the to-be-processed text.

[0197] The classification model acquisition module 302 is configured to acquire a relational classification model, where the relational classification model is trained according to any of the above-mentioned relational classification model training methods;

[0198] The relationship classification module 303 is configured to input the text to be processed into the relationship classification model to obtain the target prediction score information output by the relationship classification model. The target prediction score information includes target threshold class information and at least one target relationship information. The target relationship information is used to indicate the probability that the corresponding entity in the text to be processed belongs to the corresponding relationship. The target threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship.

[0199] In the technical solution provided by the present disclosure, a relation classification model is obtained by obtaining a text to be processed, and the text to be processed is input into the relation classification model to obtain target prediction score information output by the relation classification model. In the process of training the relation classification model used in the above solution, since when calculating the first probability score of the target entity pair belonging to the first relationship in its positive class relationship, it is not based on the probability of the target entity pair belonging to each relationship in all positive class relationships and the probability of the target entity pair belonging to the threshold class, but only based on the threshold class information and the first relationship information corresponding to the first relationship to obtain the first probability score, even if the probability of the target entity pair belonging to the first relationship increases, that is, the probability of the target entity pair belonging to other positive class relationships of the target entity pair will not be reduced, and no forced sorting between the probabilities of the target entity pair belonging to different positive class relationships will be introduced, thereby not reducing the distinguishability of the target entity pair belonging to the positive class relationship and the target entity pair belonging to the threshold class. Similarly, in the process of training the relation classification model used in the above scheme, when calculating the second probability score of the target entity pair belonging to the second relationship in its negative class relationship, it is not based on the probability of the target entity pair belonging to each relationship in all negative class relationships and the probability of the target entity pair belonging to the threshold class, but only based on the threshold class information and the second relationship information corresponding to the second relationship to obtain the second probability score. Therefore, even if the probability of the target entity pair belonging to the second relationship increases, it will not reduce the probability of the target entity pair belonging to other negative class relationships of the target entity pair, nor will it introduce a forced sorting between the probabilities of the target entity pair belonging to different negative class relationships, thereby not reducing the distinguishability of the target entity pair belonging to the negative class relationship and the target entity pair belonging to the threshold class. To summarize, during the training of the relation classification model used in the above scheme, no forced sorting between the probabilities of entity pairs belonging to different positive class relations or the forced sorting between the probabilities of entity pairs belonging to different negative class relations will be introduced, thereby enhancing the distinguishability of entity pairs belonging to positive class relations and entity pairs belonging to threshold classes, as well as the distinguishability of entity pairs belonging to negative class relations and entity pairs belonging to threshold classes, thereby ensuring that different relations in the text to be processed are accurately identified based on the target prediction score information output by the trained relation classification model, thereby improving the user experience.

[0200] The present disclosure also discloses an electronic device, Figure 5 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0201] like Figure 5 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to an embodiment of the present disclosure.

[0202] In a first aspect, an embodiment of the present disclosure provides a method for training a relationship classification model, the method being applied to a server and comprising:

[0203] Acquire a training sample set, where the training sample set includes original text and entity text information, where the entity text information is used to indicate at least one preset entity;

[0204] Inputting the original text into the relationship classification model to obtain original prediction score information, the original prediction score information including threshold class information and at least one relationship information, the relationship information is used to indicate the probability that the corresponding entity pair in the original text belongs to the corresponding relationship, and the threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship;

[0205] Obtaining a first probability score that the target entity pair belongs to the first relationship based on the threshold class information and first relationship information corresponding to the first relationship, and obtaining a second probability score that the target entity pair belongs to the second relationship based on the threshold class information and second relationship information corresponding to the second relationship, the first relationship belongs to a positive class relationship of the target entity pair, the second relationship belongs to a negative class relationship of the target entity pair, and the entities in the target entity pair belong to preset entities;

[0206] Obtain a first prediction loss according to the first probability score and the second probability score;

[0207] A target prediction loss is obtained based on at least the first prediction loss, and a relation classification model is trained based on the target prediction loss.

[0208] In one implementation of the present disclosure, the method further includes:

[0209] Calculating a first information entropy between the first relationship information and the threshold class information according to the first probability score, calculating a second information entropy between the second relationship information and the threshold class information according to the second probability score, and obtaining a second prediction loss according to the first information entropy and the second information entropy;

[0210] Obtain the target prediction loss based on at least the first prediction loss, including:

[0211] A target prediction loss is obtained based on at least the first prediction loss and the second prediction loss.

[0212] In one implementation of the present disclosure, the raw prediction score information includes a raw prediction score vector;

[0213] Obtaining a first probability score of the target entity pair belonging to the first relationship according to the threshold class information and the first relationship information corresponding to the first relationship includes:

[0214] Obtain a first probability score according to the component corresponding to the first relationship in the original prediction score vector, namely the first relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information;

[0215] Obtaining a second probability score of the target entity belonging to the second relationship according to the threshold class information and the second relationship information corresponding to the second relationship includes:

[0216] The second probability score is obtained according to the component corresponding to the second relationship in the original prediction score vector, namely the second relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information.

[0217] In one implementation of the present disclosure,

[0218] Calculating a second information entropy of the second relationship information and the threshold class information according to the second probability score includes:

[0219] Obtaining a second prediction loss based on the first information entropy, the second information entropy, the first temperature adjustment factor, and the second temperature adjustment factor. In one implementation of the present disclosure, obtaining a target prediction loss based on at least the first prediction loss and the second prediction loss includes:

[0220] A third predicted loss is obtained based on the first predicted loss and the second predicted loss, and the target predicted loss is obtained at least based on the third predicted loss.

[0221] In one implementation of the present disclosure, the original text is input into the relational classification model to obtain original prediction score information, including:

[0222] Input the original text into the relation classification model to obtain the original prediction score information and the final representation vector of each entity pair in the original text;

[0223] The method also includes:

[0224] Obtaining a fourth prediction loss based on the number of relational entity pairs in the target text that have at least one identical relationship with the target entity pair, the final representation vector of the relational entity pair, the final representation vector of the target entity pair, the final representation vector of entity pairs in the target text that are different from the target entity pair, and a third temperature adjustment factor;

[0225] Obtaining a fifth prediction loss according to the final representation vector of the target entity pair, the final representation vector of an entity pair in the target text that is different from the target entity pair, and the third temperature adjustment factor;

[0226] Obtaining a sixth predicted loss based on the fourth predicted loss and the fifth predicted loss;

[0227] The obtaining the target prediction loss at least according to the third prediction loss includes:

[0228] A seventh predicted loss is obtained based on the third predicted loss and the sixth predicted loss, and the target predicted loss is obtained based at least on the seventh predicted loss.

[0229] In one implementation of the present disclosure, the method further includes:

[0230] Obtaining a third probability score based on a component of the original prediction score vector corresponding to a random negative class relationship and a component of the original prediction score vector corresponding to a threshold class, wherein the random negative class relationship belongs to a subset randomly selected from the negative class relationships of NA samples, and the NA samples are entity pairs without a predefined relationship;

[0231] Obtaining a third information entropy according to the third probability score, and obtaining an eighth predicted loss according to the third information entropy, the third probability score, and the second temperature adjustment coefficient;

[0232] The obtaining the target predicted loss at least according to the first predicted loss and the second predicted loss includes:

[0233] Obtain a ninth predicted loss based on the eighth predicted loss, the first predicted loss, and the second predicted loss;

[0234] The tenth predicted loss is obtained based on the ninth predicted loss and the sixth predicted loss, and the target predicted loss is obtained based on the tenth predicted loss.

[0235] In a second aspect, an embodiment of the present disclosure provides a relationship classification method, which is applied to a server and includes:

[0236] Get the text to be processed,

[0237] Acquire a relationship classification model, where the relationship classification model is trained according to the method described in any one of the first aspect or any one of the implementations of the first aspect;

[0238] The text to be processed is input into the relational classification model to obtain target prediction score information output by the relational classification model. The target prediction score information includes target threshold class information and at least one target relationship information. The target relationship information is used to indicate the probability that the corresponding entity in the text to be processed belongs to the corresponding relationship. The target threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship.

[0239] Figure 6 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0240] like Figure 6 As shown, the computer system includes a processing unit, which can execute the various methods in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The processing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0241] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs a communication process via a network such as the Internet. The drive is also connected to the I / O interface as needed. Removable media, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed. Among them, the processing unit can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0242] In particular, according to embodiments of the present disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for executing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication component and / or installed from a removable medium.

[0243] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0244] The units or modules involved in the embodiments described in this disclosure may be implemented by software or programmable hardware. The units or modules described may also be provided in a processor, and the names of these units or modules do not, in certain circumstances, constitute limitations on the units or modules themselves.

[0245] As another aspect, the present disclosure further provides a computer-readable storage medium. This computer-readable storage medium may be included in the electronic device or computer system described in the above embodiments, or may be a standalone computer-readable storage medium not incorporated into the device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the methods described in the present disclosure.

[0246] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. A method for training a relational classification model, comprising: Acquire a training sample set, where the training sample set includes original text and entity text information, where the entity text information is used to indicate at least one preset entity; Inputting the original text into a relationship classification model to obtain original prediction score information, the original prediction score information including threshold class information and at least one relationship information, the relationship information being used to indicate a probability that a corresponding entity pair in the original text belongs to a corresponding relationship, and the threshold class information being used to indicate a lower limit of the probability of determining that the entity pair belongs to a relationship; Obtaining a first probability score that the target entity pair belongs to the first relationship based on the threshold class information and first relationship information corresponding to the first relationship, and obtaining a second probability score that the target entity pair belongs to the second relationship based on the threshold class information and second relationship information corresponding to the second relationship, wherein the first relationship belongs to a positive class relationship of the target entity pair, the second relationship belongs to a negative class relationship of the target entity pair, and the entities in the target entity pair belong to the preset entities; Obtain a first predicted loss according to the first probability score and the second probability score; A target prediction loss is obtained at least according to the first prediction loss, and the relationship classification model is trained according to the target prediction loss.

2. The training method according to claim 1, wherein: The method further comprises: Calculating a first information entropy between the first relationship information and the threshold type information according to the first probability score, calculating a second information entropy between the second relationship information and the threshold type information according to the second probability score, and obtaining a second prediction loss according to the first information entropy and the second information entropy; The obtaining a target prediction loss at least according to the first prediction loss includes: The target prediction loss is obtained based on at least the first prediction loss and the second prediction loss.

3. The training method according to claim 2, wherein: The original prediction score information includes an original prediction score vector; The obtaining, according to the threshold class information and the first relationship information corresponding to the first relationship, a first probability score of the target entity belonging to the first relationship includes: Obtaining the first probability score according to the component corresponding to the first relationship in the original prediction score vector, namely the first relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information; The obtaining, according to the threshold class information and the second relationship information corresponding to the second relationship, a second probability score of the target entity belonging to the second relationship includes: A second probability score is obtained according to the component corresponding to the second relationship in the original prediction score vector, namely the second relationship information, and the component corresponding to the threshold class in the original prediction score vector, namely the threshold class information.

4. The training method according to claim 3, wherein: Obtaining a second prediction loss according to the first information entropy and the second information entropy includes: The second prediction loss is obtained according to the first information entropy, the second information entropy, a first temperature adjustment factor, and a second temperature adjustment factor.

5. The training method according to claim 4, wherein: The obtaining the target predicted loss at least according to the first predicted loss and the second predicted loss includes: A third predicted loss is obtained based on the first predicted loss and the second predicted loss, and the target predicted loss is obtained at least based on the third predicted loss.

6. The training method according to claim 5, wherein: Inputting the original text into the relational classification model to obtain original prediction score information includes: Inputting the original text into the relation classification model to obtain the original prediction score information and the final representation vector of each entity pair in the original text; The method further comprises: Obtaining a fourth prediction loss based on the number of relational entity pairs in the target text that have at least one identical relationship with the target entity pair, the final representation vector of the relational entity pair, the final representation vector of the target entity pair, the final representation vector of entity pairs in the target text that are different from the target entity pair, and a third temperature adjustment factor; Obtaining a fifth prediction loss according to the final representation vector of the target entity pair, the final representation vector of an entity pair in the target text that is different from the target entity pair, and the third temperature adjustment factor; Obtaining a sixth predicted loss based on the fourth predicted loss and the fifth predicted loss; The obtaining the target prediction loss at least according to the third prediction loss includes: A seventh predicted loss is obtained based on the third predicted loss and the sixth predicted loss, and the target predicted loss is obtained based at least on the seventh predicted loss.

7. The training method according to claim 6, wherein: The method further comprises: Obtaining a third probability score based on a component of the original prediction score vector corresponding to a random negative class relationship and a component of the original prediction score vector corresponding to a threshold class, wherein the random negative class relationship belongs to a subset randomly selected from the negative class relationships of NA samples, and the NA samples are entity pairs without a predefined relationship; Obtaining a third information entropy according to the third probability score, and obtaining an eighth predicted loss according to the third information entropy, the third probability score, and the second temperature adjustment coefficient; The obtaining the target predicted loss at least according to the first predicted loss and the second predicted loss includes: Obtain a ninth predicted loss based on the eighth predicted loss, the first predicted loss, and the second predicted loss; The tenth predicted loss is obtained based on the ninth predicted loss and the sixth predicted loss, and the target predicted loss is obtained based on the tenth predicted loss.

8. A relationship classification method, comprising: Get the text to be processed; Obtain a relationship classification model, wherein the relationship classification model is trained according to any one of claims 1-7; The text to be processed is input into a relational classification model to obtain target prediction score information output by the relational classification model, wherein the target prediction score information includes target threshold class information and at least one target relationship information, wherein the target relationship information is used to indicate the probability that the corresponding entity in the text to be processed belongs to the corresponding relationship, and the target threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship.

9. A training device for a relational classification model, comprising: A sample set acquisition module is configured to acquire a training sample set, wherein the training sample set includes original text and entity text information, wherein the entity text information is used to indicate at least one preset entity; a prediction score acquisition module configured to input the original text into a relationship classification model to obtain original prediction score information, the original prediction score information including threshold class information and at least one relationship information, the relationship information being used to indicate a probability that a corresponding entity pair in the original text belongs to a corresponding relationship, and the threshold class information being used to indicate a lower limit of the probability of determining that the entity pair belongs to a relationship; a probability score acquisition module, configured to acquire a first probability score that the target entity pair belongs to the first relationship based on the threshold class information and first relationship information corresponding to the first relationship, and to acquire a second probability score that the target entity pair belongs to the second relationship based on the threshold class information and second relationship information corresponding to the second relationship, wherein the first relationship belongs to a positive class relationship of the target entity pair, the second relationship belongs to a negative class relationship of the target entity pair, and the entities in the target entity pair belong to the preset entities; a predicted loss acquisition module, configured to acquire a first predicted loss according to the first probability score and the second probability score; The model training module is configured to obtain a target prediction loss based on at least the first prediction loss, and train the relationship classification model based on the target prediction loss.

10. A relationship classification device, comprising: A module for obtaining text to be processed, configured to obtain text to be processed; A classification model acquisition module, configured to acquire a relationship classification model, wherein the relationship classification model is trained according to any one of claims 1-7; The relationship classification module is configured to input the text to be processed into the relationship classification model to obtain the target prediction score information output by the relationship classification model, wherein the target prediction score information includes target threshold class information and at least one target relationship information, wherein the target relationship information is used to indicate the probability that the corresponding entity in the text to be processed belongs to the corresponding relationship, and the target threshold class information is used to indicate the lower limit of the probability of determining that the entity pair belongs to the relationship.

11. An electronic device comprising a memory and a processor; wherein: The memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps of any one of claims 1 to 8.

12. A computer-readable storage medium having computer instructions stored thereon, wherein: When the computer instructions are executed by a processor, the method steps according to any one of claims 1 to 8 are implemented.

13. A computer program product comprising a computer program / instructions, wherein: When the computer program / instructions are executed by a processor, the method steps of any one of claims 1 to 8 are implemented.

Images