A sample optimization method and system based on relation extraction model

Abstract

This application proposes a sample optimization method and system based on a relationship extraction model, which belongs to the field of data processing technology. The method includes: obtaining a training sample set, wherein the training sample set includes M samples, each sample is composed of multiple labeled entities and the relationship between entities; constructing a relationship list of known entities; according to the relationship list of known entities, in the training sample set, if the entity and the relationship between the entity do not exist in the relationship list of known entities, then delete the entity and the relationship between the entity in the training sample set to obtain a new training sample set; take out any sample in the new training sample set, transform any sample into a set of positive samples and negative samples, and all samples are subjected to threshold processing to obtain the final input sample. This application reduces the hardware resource consumption of "relationship extraction" and improves computing efficiency without affecting the accuracy of the training results.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and specifically relates to a sample optimization method and system based on a relationship extraction model. Background Art

[0002] Relationship extraction is an important part of building a knowledge graph. Through relationship extraction, the nodes and edges of the knowledge graph can be determined. The nodes represent concepts and entities, and the edges represent attributes and relationships. For example, for a knowledge graph related to disease prediction, by extracting diseases, symptoms, common groups, and the relationships between them, it is possible to predict the possibility of a patient suffering from the disease based on his or her symptoms. Figure 7 The figure shows the entities and relationships of the knowledge graph extracted from a medical statement. The extracted entities are: small cell lung cancer, elderly smokers, cough, chest pain, hemoptysis, dyspnea and weight loss. Small cell lung cancer is the name of a disease, elderly smokers are a high-risk group, and cough, chest pain, hemoptysis, dyspnea and weight loss are the symptoms of this disease. By extracting the relationship between small cell lung cancer, elderly smokers, cough, chest pain, hemoptysis, dyspnea and weight loss, a knowledge graph is constructed. If a patient's symptoms are known, the possibility of the patient suffering from small cell lung cancer can be predicted. It can be seen that relationship extraction plays an important role in the construction and application of knowledge graphs.

[0003] However, the existing technology has the following problems: the hardware resource consumption of relationship extraction is very high. For medium-sized data, when the GPU model is Tesla V100 and the memory is 16G, a relationship extraction usually runs for several hours to more than ten hours, which seriously affects the application of relationship extraction models in the medical field.

[0004] Therefore, how to reduce the hardware resource consumption of relation extraction and improve computing efficiency without affecting the accuracy of relation extraction is a technical problem that needs to be solved urgently. Summary of the invention

[0005] Based on the above technical problems, the present application proposes a sample optimization method and system based on a relationship extraction model, which can significantly reduce the running time of the relationship extraction model without affecting the recognition accuracy.

[0006] In the first aspect, the present application proposes a sample optimization method based on a relation extraction model, comprising the following steps:

[0007] Step S1: Obtain a training sample set, wherein the training sample set includes M samples, each sample consisting of a plurality of labeled entities and relations between entities;

[0008] Step S2: construct a relationship list of known entities, that is, perform permutations and combinations for every two known entities to obtain all known entity pairs, and save all known entity pairs and the relationships between the known entity pairs into the relationship list;

[0009] Step S3: According to the relationship list of known entities, in the training sample set, if the entity and the relationship between the entities do not exist in the relationship list of known entities, then delete the entity and the relationship between the entities in the training sample set to obtain a new training sample set;

[0010] Step S4: Take any sample from the new training sample set and transform it into a set of positive samples and negative samples, that is, perform permutations and combinations for every two labeled entities in any sample. If the negative sample exceeds the set threshold, the labeled entity is prevented from combining with other labeled entities, and the permutations and combinations between the labeled entities and the corresponding relationship types are obtained;

[0011] Step S5: until all samples in the new training sample set are subjected to the threshold processing of step S4, the final input sample is obtained.

[0012] The method comprises the following steps: according to the relationship list of known entities, if the entity and the relationship between the entity do not exist in the relationship list of known entities in the training sample set, then deleting the entity and the relationship between the entity in the training sample set to obtain a new training sample set:

[0013] Step S3.1: traverse M samples in the training sample set;

[0014] Step S3.2: traverse the labeled entities and the relationships between the labeled entities in each sample;

[0015] Step S3.3: Determine whether the marked entity and the relationship between the marked entities exist in the relationship list of known entities;

[0016] Step S3.4: if the marked entity and the relationship between the marked entities exist in the relationship list of known entities, then the marked entity and the relationship between the marked entities are retained;

[0017] Step S3.5: If the marked entity and the relationship between the marked entities do not exist in the relationship list of known entities, the corresponding marks of the marked entity and the relationship between the marked entities are deleted.

[0018] The positive sample is defined as if there is a relationship between the labeled entities in any sample taken, that is, the relationship type is an existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called positive samples; the negative sample is defined as if there is no relationship between the labeled entities in any sample taken, that is, the relationship type is an absence of relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called negative samples.

[0019] The method of taking any sample from the new training sample set and transforming the sample into a set of positive samples and negative samples includes the following steps:

[0020] Step S4.1: In any sample, perform permutations and combinations for every two labeled entity pairs in any sample;

[0021] Step S4.2: when the labeled entity is arranged and combined with the entity, identifying whether there is a relationship between the two in any of the samples;

[0022] Step S4.3: If there is a relationship, the permutation and combination of the marked entities and the corresponding relationship type are saved as positive samples;

[0023] Step S4.4: If there is no relationship, determine whether the number of negative samples in any of the samples exceeds a set threshold;

[0024] Step S4.5: If the threshold is exceeded, the marked entity is prevented from being combined with other marked entities;

[0025] Step S4.6: If the set threshold is not exceeded, the marked entity is allowed to continue to be combined with other marked entities, and the permutations and combinations of the marked entities and the corresponding relationship types are saved as negative samples.

[0026] The sum of the number of the negative samples and the positive samples is: N×(N-1), where N represents the number of labeled entities in any sample taken.

[0027] The final input sample is an input sample of the relation extraction model.

[0028] In the second aspect, the present application proposes a sample optimization system based on a relation extraction model, including: a sample set acquisition module, an entity pair combination module, a new sample set acquisition module, a threshold processing module, and an input sample acquisition module;

[0029] The sample set acquisition module, entity pair combination module, new sample set acquisition module, threshold processing module, and input sample acquisition module are connected in sequence;

[0030] The sample set acquisition module is used to acquire a training sample set, wherein the training sample set includes M samples, each of which is composed of a plurality of labeled entities and relations between entities;

[0031] The entity pair combination module is used to construct a relationship list of known entities, that is, to perform permutations and combinations on every two known entities to obtain all known entity pairs, and to save all known entity pairs and the relationships between the known entity pairs into the relationship list;

[0032] The new sample set acquisition module is used to delete the entity and the relationship between the entity in the training sample set according to the relationship list of the known entities, if the entity and the relationship between the entity do not exist in the relationship list of the known entities, to obtain a new training sample set;

[0033] The threshold processing module is used to take out any sample from the new training sample set and transform any sample into a set of positive samples and negative samples, that is, to perform permutations and combinations for every two labeled entities in any sample. If the negative sample exceeds the set threshold, the labeled entity is prevented from combining with other labeled entities, and the permutations and combinations and corresponding relationship types between the labeled entities are obtained;

[0034] The input sample acquisition module is used to obtain final input samples until all samples in the new training sample set are subjected to threshold processing by the threshold processing module.

[0035] The positive sample in the threshold processing module is defined as if there is a relationship between the labeled entities in any sample taken, that is, the relationship type is existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called positive samples; the negative sample is defined as if there is no relationship between the labeled entities in any sample taken, that is, the relationship type is non-existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called negative samples.

[0036] The new sample set acquisition module includes a traversal unit and a processing unit;

[0037] The traversal unit is connected to the processing unit;

[0038] The traversal unit is used to traverse the M samples in the training sample set, traverse the labeled entities and the relationship between the labeled entities in each sample, and send the labeled entities and the relationship between the labeled entities in each sample to the processing unit;

[0039] The processing unit is used to determine whether the marked entity and the relationship between the marked entities exist in the relationship list of known entities; if the marked entity and the relationship between the marked entities exist in the relationship list of known entities, the marked entity and the relationship between the marked entities are retained; if the marked entity and the relationship between the marked entities do not exist in the relationship list of known entities, the corresponding labels of the marked entity and the relationship between the marked entities are deleted to obtain a new training sample set.

[0040] The threshold processing module includes: an arrangement and combination unit, a recognition unit, a positive sample storage unit, and a negative sample processing unit;

[0041] The permutation and combination unit, the recognition unit, the positive sample storage unit, and the negative sample processing unit are connected in sequence;

[0042] The permutation and combination unit is used to perform permutation and combination on every two labeled entity pairs in any sample;

[0043] The recognition unit is used to recognize whether there is a relationship between the marked entity and the entity in any sample when the marked entity and the entity are arranged and combined;

[0044] The positive sample storage unit is used to determine if there is a relationship, and then save the arrangement and combination of the marked entities and the corresponding relationship type as a positive sample;

[0045] The negative sample processing unit is used to determine whether the number of negative samples in any of the samples exceeds a set threshold if no relationship exists; if it exceeds the set threshold, the marked entity is prevented from being combined with other marked entities; if it does not exceed the set threshold, the marked entity is allowed to continue to be combined with other marked entities, and the permutations and combinations of the marked entities and the corresponding relationship types are saved as negative samples.

[0046] The sum of the number of the negative samples and the positive samples in the threshold processing module is: N×(N-1), where N represents the number of labeled entities in any sample taken.

[0047] The final input sample in the input sample acquisition module is the input sample of the relation extraction model.

[0048] In a third aspect, the present application provides a computer device, the computer device comprising: a processor;

[0049] The processor is used to implement the steps of the sample optimization method based on the relationship extraction model when executing the computer program stored in the memory.

[0050] In a fourth aspect, the present application proposes a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the sample optimization method based on the relational extraction model.

[0051] Beneficial technical effects:

[0052] This application proposes a sample optimization method and system based on a relationship extraction model, which improves the model training efficiency without affecting the accuracy of the training results, greatly shortens the model training time, reduces the hardware resource consumption of "relationship extraction", and improves computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flow chart of a sample optimization method based on a relationship extraction model according to an embodiment of the present application;

[0054] Figure 2 A flow chart of obtaining a new training sample set according to an embodiment of the present application;

[0055] Figure 3 This is a flow chart of threshold processing in an embodiment of the present application;

[0056] Figure 4 A principle block diagram of a sample optimization system based on a relationship extraction model according to an embodiment of the present application;

[0057] Figure 5 This is a block diagram of the internal structure principle of the new sample set acquisition module in an embodiment of the present application;

[0058] Figure 6 This is a block diagram of the internal structure principle of the threshold processing module of an embodiment of the present application;

[0059] Figure 7 A diagram showing examples of entities and relationships. DETAILED DESCRIPTION

[0060] The present application is further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present application.

[0061] The present application proposes a sample optimization method and system based on a relation extraction model, which can significantly reduce the running time of the relation extraction model without affecting the recognition accuracy.

[0062] The task of the relationship extraction model is to determine the type of relationship between two entities. The relationship type includes two types of relationships: relationship and no relationship. If the relationship type is relationship, the two entities constitute a positive sample; if the relationship type is no relationship, the two entities constitute a negative sample. For example, there are currently five entity types, A, B, C, D, and E, and there are only two types of relationships: relationship and no relationship. When constructing the input, the relationship extraction model will arrange and combine all entities, resulting in an imbalance of positive and negative samples due to their relationship types. A single sample is shown in Table 1:

[0063] Table 1: Permutations and combinations of entity pairs A to E

[0064]

[0065]

[0066] It can be seen that the total number of positive and negative samples can be obtained by the formula N×(N-1) (N represents the number of entity types labeled for a single sample), the number of positive samples is 5, and the number of negative samples is 15. The number of negative samples far exceeds the number of positive samples. As the number of entity types increases, the gap in the ratio of positive and negative samples will become larger and larger, increasing the training load of the model.

[0067] From the above problems, we found that if some negative samples can be removed without affecting the training results, the training efficiency of the model can be greatly improved and the hardware resources of the process can be avoided. Therefore, negative sampling optimization is performed in the following ways:

[0068] In the first aspect, the present application proposes a sample optimization method based on a relation extraction model, such as Figure 1 As shown, the following steps are included:

[0069] Step S1: Obtain a training sample set, wherein the training sample set includes M samples, each sample consisting of a plurality of labeled entities and relations between entities;

[0070] Step S2: construct a relationship list of known entities, that is, perform permutations and combinations for every two known entities to obtain all known entity pairs, and save all known entity pairs and the relationships between the known entity pairs into the relationship list;

[0071] Step S3: According to the relationship list of known entities, in the training sample set, if the entity and the relationship between the entities do not exist in the relationship list of known entities, then delete the entity and the relationship between the entities in the training sample set to obtain a new training sample set;

[0072] Step S4: Take any sample from the new training sample set and transform it into a set of positive samples and negative samples, that is, perform permutations and combinations for every two labeled entities in any sample. If the negative sample exceeds the set threshold, the labeled entity is prevented from combining with other labeled entities, and the permutations and combinations between the labeled entities and the corresponding relationship types are obtained;

[0073] Step S5: until all samples in the new training sample set are subjected to the threshold processing of step S4, the final input sample is obtained.

[0074] In step S3, according to the relationship list of known entities, in the training sample set, if the entity and the relationship between the entities do not exist in the relationship list of known entities, then the entity and the relationship between the entities in the training sample set are deleted to obtain a new training sample set, such as Figure 2 As shown, the following steps are included:

[0075] Step S3.1: traverse M samples in the training sample set;

[0076] Step S3.2: traverse the labeled entities and the relationships between the labeled entities in each sample;

[0077] Step S3.3: Determine whether the marked entity and the relationship between the marked entities exist in the relationship list of known entities;

[0078] Step S3.4: if the marked entity and the relationship between the marked entities exist in the relationship list of known entities, then the marked entity and the relationship between the marked entities are retained;

[0079] Step S3.5: If the marked entity and the relationship between the marked entities do not exist in the relationship list of known entities, the corresponding marks of the marked entity and the relationship between the marked entities are deleted.

[0080] The positive sample is defined as if there is a relationship between the labeled entities in any sample taken, that is, the relationship type is an existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called positive samples; the negative sample is defined as if there is no relationship between the labeled entities in any sample taken, that is, the relationship type is an absence of relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called negative samples.

[0081] In step S4, any sample is taken from the new training sample set, and any sample is transformed into a set of positive samples and negative samples, such as Figure 3 As shown, the following steps are included:

[0082] Step S4.1: In any sample, perform permutations and combinations for every two labeled entity pairs in any sample;

[0083] Step S4.2: when the labeled entity is arranged and combined with the entity, identifying whether there is a relationship between the two in any of the samples;

[0084] Step S4.3: If there is a relationship, the permutation and combination of the marked entities and the corresponding relationship type are saved as positive samples;

[0085] Step S4.4: If there is no relationship, determine whether the number of negative samples in any of the samples exceeds a set threshold;

[0086] Step S4.5: If the threshold is exceeded, the marked entity is prevented from being combined with other marked entities;

[0087] Step S4.6: If the set threshold is not exceeded, the marked entity is allowed to continue to be combined with other marked entities, and the permutations and combinations of the marked entities and the corresponding relationship types are saved as negative samples.

[0088] After step S3, the number of remaining negative samples is still much larger than the number of positive samples, so it is processed again through steps S4 and S5. The specific implementation is as follows: set a threshold value. When the relationship type of the marked A entity and other marked entities in a single sample is arranged and combined, if it is a positive sample, it will not be processed. If it is a negative sample, according to the set threshold value, if it exceeds the threshold, A and other entities will no longer be combined, and B entity and other entities will be combined. According to the combination of A, if it exceeds the threshold, C will be carried out, and so on. Finally, the final input sample is obtained, that is, the input sample of the relationship extraction model. Compared with the input sample of the original relationship extraction model, the input sample of the relationship extraction model processed by the above steps S1 to S5 has a much smaller data volume than the input sample data volume of the original relationship extraction model, so that the model training time is improved.

[0089] Experimental results:

[0090] The following are some experimental data of real samples, as shown in Table 2:

[0091] Table 2: Experimental results data

[0092]

[0093] Among them, in Table 2, Negative sampling / 3 represents that there are 3 negative samples, and Negative sampling / ALL represents that the negative samples are all negative samples; It can be clearly seen from Table 2 that when all negative samples are included in the sampling, the training time is the longest (the training time for cancer data is 31276s, and the training time for head disease data is 110054s), and the accuracy is also relatively high (95.94% for cancer data and 84.03% for head disease data). When some of the negative samples are removed by the method of the present application, the training time is greatly reduced (when 3 negative samples are used). This will significantly reduce the hardware resources occupied and have little impact on the accuracy (in the cancer data, when 3 negative samples are used, the accuracy is 95.15%, which is only 0.75% lower than when all negative samples are used). It is even more accurate than when all negative samples are used (in the head disease data, when 3 negative samples are used, the accuracy is 88.35%, which is 4.32% higher than when all negative samples are used).

[0094] In the second aspect, the present application proposes a sample optimization system based on a relation extraction model, such as Figure 4 As shown, it includes: a sample set acquisition module, an entity pair combination module, a new sample set acquisition module, a threshold processing module, and an input sample acquisition module;

[0095] The sample set acquisition module, entity pair combination module, new sample set acquisition module, threshold processing module, and input sample acquisition module are connected in sequence;

[0096] The sample set acquisition module is used to acquire a training sample set, wherein the training sample set includes M samples, each of which is composed of a plurality of labeled entities and relations between entities;

[0097] The entity pair combination module is used to construct a relationship list of known entities, that is, to perform permutations and combinations on every two known entities to obtain all known entity pairs, and to save all known entity pairs and the relationships between the known entity pairs into the relationship list;

[0098] The new sample set acquisition module is used to delete the entity and the relationship between the entity in the training sample set according to the relationship list of the known entities, if the entity and the relationship between the entity do not exist in the relationship list of the known entities, to obtain a new training sample set;

[0099] The threshold processing module is used to take out any sample from the new training sample set and transform any sample into a set of positive samples and negative samples, that is, to perform permutations and combinations for every two labeled entities in any sample. If the negative sample exceeds the set threshold, the labeled entity is prevented from combining with other labeled entities, and the permutations and combinations and corresponding relationship types between the labeled entities are obtained;

[0100] The input sample acquisition module is used to obtain final input samples until all samples in the new training sample set are subjected to threshold processing by the threshold processing module.

[0101] The positive sample in the threshold processing module is defined as if there is a relationship between the labeled entities in any sample taken, that is, the relationship type is existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called positive samples; the negative sample is defined as if there is no relationship between the labeled entities in any sample taken, that is, the relationship type is non-existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called negative samples.

[0102] The new sample set acquisition module is as follows: Figure 5 As shown, it includes a traversal unit and a processing unit;

[0103] The traversal unit is connected to the processing unit;

[0104] The traversal unit is used to traverse the M samples in the training sample set, traverse the labeled entities and the relationship between the labeled entities in each sample, and send the labeled entities and the relationship between the labeled entities in each sample to the processing unit;

[0105] The processing unit is used to determine whether the marked entity and the relationship between the marked entities exist in the relationship list of known entities; if the marked entity and the relationship between the marked entities exist in the relationship list of known entities, the marked entity and the relationship between the marked entities are retained; if the marked entity and the relationship between the marked entities do not exist in the relationship list of known entities, the corresponding labels of the marked entity and the relationship between the marked entities are deleted to obtain a new training sample set.

[0106] The threshold processing module, such as Figure 6 As shown, it includes: an arrangement and combination unit, a recognition unit, a positive sample storage unit, and a negative sample processing unit;

[0107] The permutation and combination unit, the recognition unit, the positive sample storage unit, and the negative sample processing unit are connected in sequence;

[0108] The permutation and combination unit is used to perform permutation and combination on every two labeled entity pairs in any sample;

[0109] The recognition unit is used to recognize whether there is a relationship between the marked entity and the entity in any sample when the marked entity and the entity are arranged and combined;

[0110] The positive sample storage unit is used to determine if there is a relationship, and then save the arrangement and combination of the marked entities and the corresponding relationship type as a positive sample;

[0111] The negative sample processing unit is used to determine whether the number of negative samples in any of the samples exceeds a set threshold if no relationship exists; if it exceeds the set threshold, the marked entity is prevented from being combined with other marked entities; if it does not exceed the set threshold, the marked entity is allowed to continue to be combined with other marked entities, and the permutations and combinations of the marked entities and the corresponding relationship types are saved as negative samples.

[0112] The sum of the number of the negative samples and the positive samples in the threshold processing module is: N×(N-1), where N represents the number of labeled entities in any sample taken.

[0113] The final input sample in the input sample acquisition module is the input sample of the relation extraction model.

[0114] In a third aspect, the present application provides a computer device, the computer device comprising: a processor;

[0115] The processor is used to implement the steps of the sample optimization method based on the relationship extraction model when executing the computer program stored in the memory.

[0116] In a fourth aspect, the present application proposes a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the sample optimization method based on the relational extraction model.

[0117] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A sample optimization method based on a relation extraction model, characterized in that: The steps include: Step S1: Obtain a training sample set, wherein the training sample set includes M samples, each sample consisting of a plurality of labeled entities and relations between entities; Step S2: construct a relationship list of known entities, that is, perform permutations and combinations for every two known entities to obtain all known entity pairs, and save all known entity pairs and the relationships between the known entity pairs into the relationship list; Step S3: According to the relationship list of known entities, in the training sample set, if the entity and the relationship between the entities do not exist in the relationship list of known entities, then delete the entity and the relationship between the entities in the training sample set to obtain a new training sample set; Step S4: Take any sample from the new training sample set and transform it into a set of positive samples and negative samples, that is, perform permutations and combinations for every two labeled entities in any sample. If the negative sample exceeds the set threshold, the labeled entity is prevented from combining with other labeled entities, and the permutations and combinations between the labeled entities and the corresponding relationship types are obtained; The method of taking any sample from the new training sample set and transforming the sample into a set of positive samples and negative samples includes the following steps: Step S4.1: In any sample, perform permutations and combinations for every two labeled entity pairs in any sample; Step S4.2: when the labeled entity is arranged and combined with the entity, identifying whether there is a relationship between the two in any of the samples; Step S4.3: If there is a relationship, the permutation and combination of the marked entities and the corresponding relationship type are saved as positive samples; Step S4.4: If there is no relationship, determine whether the number of negative samples in any of the samples exceeds a set threshold; Step S4.5: If the threshold is exceeded, the marked entity is prevented from being combined with other marked entities; Step S4.6: If the threshold is not exceeded, the marked entity is allowed to continue to be combined with other marked entities, and the permutations and combinations of the marked entities and the corresponding relationship types are saved as negative samples; Step S5: until all samples in the new training sample set are subjected to the threshold processing of step S4, the final input sample is obtained.

2. The sample optimization method based on the relationship extraction model according to claim 1, characterized in that: The method comprises the following steps: according to the relationship list of known entities, if the entity and the relationship between the entity do not exist in the relationship list of known entities in the training sample set, then deleting the entity and the relationship between the entity in the training sample set to obtain a new training sample set: Step S3.1: traverse M samples in the training sample set; Step S3.2: traverse the labeled entities and the relationships between the labeled entities in each sample; Step S3.3: Determine whether the marked entity and the relationship between the marked entities exist in the relationship list of known entities; Step S3.4: if the marked entity and the relationship between the marked entities exist in the relationship list of known entities, then the marked entity and the relationship between the marked entities are retained; Step S3.5: If the marked entity and the relationship between the marked entities do not exist in the relationship list of known entities, the corresponding marks of the marked entity and the relationship between the marked entities are deleted.

3. The sample optimization method based on the relationship extraction model according to claim 1, characterized in that: The positive sample is defined as if there is a relationship between the labeled entities in any sample taken, that is, the relationship type is an existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called positive samples; the negative sample is defined as if there is no relationship between the labeled entities in any sample taken, that is, the relationship type is an absence of relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called negative samples.

4. The sample optimization method based on the relationship extraction model according to claim 3, characterized in that: The sum of the number of negative samples and positive samples is: ,in, N Indicates the number of labeled entities in any sample taken.

5. The sample optimization method based on the relationship extraction model according to claim 1, characterized in that: The final input sample is an input sample of the relation extraction model.

6. A sample optimization system based on a relation extraction model, characterized in that: include: Sample set acquisition module, entity pair combination module, new sample set acquisition module, threshold processing module, input sample acquisition module; The sample set acquisition module, entity pair combination module, new sample set acquisition module, threshold processing module, and input sample acquisition module are connected in sequence; The sample set acquisition module is used to acquire a training sample set, wherein the training sample set includes M samples, each of which is composed of a plurality of labeled entities and relations between entities; The entity pair combination module is used to construct a relationship list of known entities, that is, to perform permutations and combinations on every two known entities to obtain all known entity pairs, and to save all known entity pairs and the relationships between the known entity pairs into the relationship list; The new sample set acquisition module is used to delete the entity and the relationship between the entity in the training sample set according to the relationship list of the known entities, if the entity and the relationship between the entity do not exist in the relationship list of the known entities, to obtain a new training sample set; The threshold processing module is used to take out any sample from the new training sample set and transform any sample into a set of positive samples and negative samples, that is, to perform permutations and combinations for every two labeled entities in any sample. If the negative sample exceeds the set threshold, the labeled entity is prevented from combining with other labeled entities, and the permutations and combinations and corresponding relationship types between the labeled entities are obtained; The threshold processing module includes: an arrangement and combination unit, a recognition unit, a positive sample storage unit, and a negative sample processing unit; The permutation and combination unit, the recognition unit, the positive sample storage unit, and the negative sample processing unit are connected in sequence; The permutation and combination unit is used to perform permutation and combination on every two labeled entity pairs in any sample; The recognition unit is used to recognize whether there is a relationship between the marked entity and the entity in any sample when the marked entity and the entity are arranged and combined; The positive sample storage unit is used to determine if there is a relationship, and then save the permutation and combination of the marked entities and the corresponding relationship type as a positive sample; The negative sample processing unit is used to determine whether the number of negative samples in any sample exceeds a set threshold if there is no relationship; If the threshold is exceeded, the marked entity is prevented from being combined with other marked entities; if the threshold is not exceeded, the marked entity is allowed to continue to be combined with other marked entities, and the permutations and combinations of the marked entities and the corresponding relationship types are saved as negative samples; The input sample acquisition module is used to obtain final input samples until all samples in the new training sample set are subjected to threshold processing by the threshold processing module.

7. The sample optimization system based on the relation extraction model according to claim 6, characterized in that: The positive sample in the threshold processing module is defined as if there is a relationship between the labeled entities in any sample taken, that is, the relationship type is existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called positive samples; the negative sample is defined as if there is no relationship between the labeled entities in any sample taken, that is, the relationship type is non-existence relationship, then the permutation and combination of the labeled entity pairs and the corresponding relationship type are called negative samples.

8. The sample optimization system based on the relation extraction model according to claim 6, characterized in that: The new sample set acquisition module includes a traversal unit and a processing unit; The traversal unit is connected to the processing unit; The traversal unit is used to traverse the M samples in the training sample set, traverse the labeled entities and the relationship between the labeled entities in each sample, and send the labeled entities and the relationship between the labeled entities in each sample to the processing unit; The processing unit is used to determine whether the marked entity and the relationship between the marked entities exist in the relationship list of known entities; If the marked entity and the relationship between the marked entities exist in the relationship list of known entities, the marked entity and the relationship between the marked entities are retained; If the marked entity and the relationship between the marked entities do not exist in the relationship list of known entities, the corresponding labels of the marked entity and the relationship between the marked entities are deleted to obtain a new training sample set.

9. The sample optimization system based on the relation extraction model according to claim 6, characterized in that: The sum of the number of negative samples and positive samples in the threshold processing module is: ,in, N Indicates the number of labeled entities in any sample taken.

10. The sample optimization system based on the relation extraction model according to claim 6, characterized in that: The final input sample in the input sample acquisition module is the input sample of the relation extraction model.

11. A computer device, characterized in that: The computer device comprises: a processor; The processor is used to implement the steps of the method according to any one of claims 1 to 5 when executing the computer program stored in the memory.

12. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images