A denoising method, device and equipment for medical remote supervision relation extraction

Abstract

The present application relates to the technical field of medical data processing, in particular to a medical remote supervision relation extraction denoising method, device and equipment, the medical remote supervision relation extraction denoising method comprising: obtaining a medical training set; according to the medical training set, the initial training of relation classifier is carried out, and the prediction probability of each relation type is obtained; according to the prediction probability, the uncertainty of each relation type is determined; according to the uncertainty and prediction probability of each relation type, the sample is re-labeled; according to the re-labeled sample, the medical training set is reconstructed; the reconstructed medical training set is input into the relation classifier, and new prediction probability is output; the above method is repeated to iteratively train the relation classifier, and a relation extraction model is generated; the medical sentence is input into the trained relation extraction model, and the prediction probability is output. The present application improves the accuracy of the relation extraction model in predicting medical data by reconstructing the medical training set in a denoising manner.

Description

Technical Field

[0001] The present invention relates to the technical field of medical data processing, and in particular to a denoising method, device and equipment for medical remote supervision relationship extraction. Background Art

[0002] With the development of information technology, the scale of medical data is growing and the sources are becoming increasingly complex. Knowledge graphs, as a structured way to represent and store knowledge, can help people better organize and understand medical knowledge. However, building a large-scale, high-quality knowledge graph requires significant manpower and time. It requires mining relational facts from a large amount of medical corpus to enrich the knowledge graph's connections and alleviate its sparsity to further support downstream medical knowledge reasoning tasks.

[0003] Distant Supervision (DS) is a method for automatically obtaining labeled training data. Its principle is to align the original entity annotation text to the existing knowledge base to quickly obtain large-scale relationship annotation samples, and use the obtained training data with relationship annotations to train the model, and finally obtain a model that can be used for relationship extraction. However, since the relationship facts of medical data are usually complex and diverse, there may be multiple relationships between different types of entities, resulting in a large difference between the relationship annotations of the training data obtained through remote supervision and the actual relationships. Therefore, the model trained with this training data has weak predictive ability in processing medical data, and the results of the model output are prone to errors. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the accuracy of the prediction probability of medical data relationship extraction.

[0005] To solve the above problems, the present invention provides a denoising method, apparatus and device for medical remote supervision relationship extraction.

[0006] In a first aspect, the present invention provides a denoising method for medical remote supervision relationship extraction, comprising:

[0007] S1, obtaining a medical training set, wherein the medical training set includes a plurality of samples, and assigning labels to the samples according to relationship types of the samples;

[0008] S2, performing at least one round of initial training on the relationship classifier based on the medical training set to obtain a predicted probability of each sample corresponding to all the relationship types;

[0009] S3, determining the uncertainty of each of the relationship types according to the predicted probability of each of the samples corresponding to all of the relationship types;

[0010] S4, screening the samples in the medical training set according to the uncertainty of each of the relationship types and the predicted probability of each sample corresponding to all of the relationship types, and reassigning the labels to the screened samples;

[0011] S5, reconstructing the medical training set according to the samples with reallocated labels to generate a denoised medical training set;

[0012] S6, inputting the denoised medical training set into the relationship classifier, and outputting the new predicted probability of each sample corresponding to all the relationship types;

[0013] S7, repeatedly executing steps S3 to S6 to perform iterative training on the relationship classifier for a preset number of times to generate a relationship extraction model;

[0014] S8, inputting the medical sentence into the trained relationship extraction model, and outputting the predicted probability of the medical sentence corresponding to each relationship type.

[0015] Optionally, determining the uncertainty of each of the relationship types according to the predicted probability of each of the samples corresponding to all of the relationship types includes:

[0016] Determine the entropy value of the sample according to the predicted probability of the sample corresponding to all the relationship types and the predicted entropy value formula;

[0017] Determine the expected entropy of the relationship type according to the entropy values ​​of all the samples of the same relationship type and an expected entropy formula;

[0018] The expected entropy is determined as the uncertainty of the relationship type.

[0019] Optionally, the predicted entropy value formula is:

[0020]

[0021] The expected entropy formula is:

[0022]

[0023] Among them, e i is the entropy value of the i-th sample in the medical training set, C is the number of relationship types in the medical training set, is the predicted probability of the jth relationship type of the i-th sample, E(e j ) is the expected entropy of the jth relationship type, N j is the number of samples of the jth relationship type, S is the medical training set, Si is the i-th sample in the medical training set.

[0024] Optionally, screening the samples in the medical training set according to the uncertainty of each relationship type and the predicted probability of each sample corresponding to all the relationship types, and reallocating the labels to the screened samples, includes:

[0025] Determining a label filtering threshold for the relationship type according to the uncertainty of the relationship type, the maximum predicted probability of all the samples corresponding to the relationship type, and a label filtering threshold formula;

[0026] When the predicted probability of the relationship type corresponding to the sample in the relationship type is less than the label filtering threshold of the relationship type, determining that the sample is the sample with a reallocated label;

[0027] The labels of the samples with reallocated labels are reallocated.

[0028] Optionally, reallocating the labels of the samples with reallocated labels includes:

[0029] Determine a label reallocation threshold for the relationship type according to the uncertainty of the relationship type, the maximum predicted probability of all the samples corresponding to the relationship type, and a label filtering threshold formula;

[0030] When the maximum predicted probability of the samples to be reassigned labels is greater than the label reassignment threshold of the relationship type corresponding to the maximum predicted probability, reassigning the labels of the samples according to the relationship type;

[0031] When the maximum predicted probability of the sample to be reassigned labels is less than or equal to the label reassignment threshold of the relationship type corresponding to the maximum predicted probability, the original label of the sample is retained.

[0032] Optionally, the tag filtering threshold formula is:

[0033]

[0034] The label reallocation threshold is:

[0035]

[0036] in, is the label filtering threshold of the j-th relationship type, is the label reallocation threshold, U (j) is the uncertainty of the j-th relationship type, THF is the global filtering threshold hyperparameter, TH R is the global label redistribution threshold hyperparameter, α and β are hyperparameters, and σ is the activation function.

[0037] Optionally, it also includes:

[0038] When the number of iterative training rounds is greater than a first preset threshold, the iterative training is stopped, and / or

[0039] When the number of samples that need to reallocate the labels is less than a second preset threshold, stopping the iterative training, and / or

[0040] When the index of the validation set is less than the preset index, the iterative training is stopped.

[0041] Optionally, it also includes:

[0042] Determining the relationship type corresponding to the maximum predicted probability of the output medical sentence as the relationship type of the medical sentence;

[0043] The medical sentences are assigned labels according to the relationship types.

[0044] In a second aspect, the present invention provides a denoising device for medical remote supervision relationship extraction, comprising:

[0045] an acquisition module, configured to acquire a medical training set, wherein the medical training set includes a plurality of samples, and assign labels to the samples according to relationship types of the samples;

[0046] a training module configured to perform at least one round of initial training on the relationship classifier based on the medical training set to obtain a predicted probability that each sample corresponds to all the relationship types; input the denoised medical training set into the relationship classifier to output a new predicted probability that each sample corresponds to all the relationship types; and repeatedly perform steps S3 to S6 to perform a preset number of iterative training on the relationship classifier to generate a relationship extraction model;

[0047] A processing module, configured to determine the uncertainty of each of the relationship types according to the predicted probability of each sample corresponding to all the relationship types;

[0048] a screening module, configured to screen the samples in the medical training set according to the uncertainty of each of the relationship types and the predicted probability of each sample corresponding to all of the relationship types, and reallocate the labels to the screened samples;

[0049] a reconstruction module, configured to reconstruct the medical training set according to the samples with reallocated labels to generate a denoised medical training set;

[0050] The relationship extraction module is used to input the medical sentence into the trained relationship extraction model and output the predicted probability of the medical sentence corresponding to each relationship type.

[0051] In a third aspect, the present invention provides an electronic device comprising a memory and a processor;

[0052] The memory is used to store computer programs;

[0053] The processor is configured to implement any one of the methods described in the first aspect when executing the computer program.

[0054] The beneficial effects of the denoising method, device and equipment for medical remote supervision relationship extraction of the present invention are as follows: performing at least one round of initial training on the relationship classifier through the medical training set, so that it has a certain discrimination ability for each relationship type, improving the accuracy of the output prediction probability, determining the uncertainty of each probability according to the prediction probability of all categories in the data set corresponding to each sample, measuring the discrimination ability of the relationship classifier for each relationship type by uncertainty, processing the sample data in the relationship type according to the uncertainty, further improving the discrimination ability of the trained relationship classifier for the relationship type, screening the samples in the medical training set according to the uncertainty of each relationship type and the prediction probability of each sample corresponding to all relationship types output by the relationship classifier, and classifying the relationship classifier according to the uncertainty. The classifier screens the relationship types with weak discrimination ability, and then determines the screened samples according to the predicted probabilities of each relationship type corresponding to the samples, and reallocates labels to the screened samples, and screens the noise samples, and changes the relationship types of the samples by reallocating labels to achieve the effect of noise reduction. This can improve the relationship classifier's ability to discriminate relationship types with weak discrimination ability, reconstruct the medical training set according to the samples with reallocated labels, and input the relationship classifier again. The relationship classifier that has undergone initial training is iteratively trained through the above method, and the relationship classifier's ability to discriminate relationship types with weak discrimination ability is continuously improved through iterative training, thereby improving the accuracy of the predicted probability output by the relationship extraction model obtained after iterative training, and ultimately achieving accurate relationship extraction of medical data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of a flow chart of a denoising method for medical remote supervision relationship extraction according to an embodiment of the present invention;

[0056] Figure 2 Schematic diagram of the structure of a relation classifier according to an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of label filtering and reallocation according to an embodiment of the present invention;

[0058] Figure 4 This is a schematic structural diagram of a noise reduction device for medical remote supervision relationship extraction according to an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram showing how the evaluation index changes with the number of iterations according to an embodiment of the present invention;

[0060] Figure 6 This is a schematic diagram showing how the number of label reallocations changes with the number of iterations according to an embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of a first relationship category threshold change according to an embodiment of the present invention;

[0062] Figure 8 This is a schematic diagram of a change in the second relationship category threshold value according to an embodiment of the present invention;

[0063] Figure 9 This is a schematic diagram of a change in the third relationship category threshold value according to an embodiment of the present invention;

[0064] Figure 10 This is a schematic diagram of a change in the fourth relationship category threshold value according to an embodiment of the present invention;

[0065] Figure 11 PR curve schematic diagram of model 1 according to an embodiment of the present invention;

[0066] Figure 12 PR curve schematic diagram of Model 2 of an embodiment of the present invention;

[0067] Figure 13 PR curve schematic diagram of Model 3 of an embodiment of the present invention;

[0068] Figure 14 Schematic diagram of the PR curve of model 4 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0070] It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present invention described herein can be practiced in an order other than that illustrated or described herein.

[0071] Since its proposal, remote supervision methods have been widely recognized by the academic community. The main research point of remote supervision-based relationship extraction research methods is how to improve the accuracy of the relationship extraction model through denoising. In the field of remote supervision relationship extraction, the multi-instance learning denoising method is mainly used, which divides samples of the same entity pairs into packages and classifies them based on package-level samples. Package-level relationship extraction models all classify predefined relationship types based on the representation of the package, but this will result in the sentences in the package having no explicit labels, thus limiting the application of sentence-level downstream tasks. Therefore, some studies have begun to focus on denoising algorithms based on sentence-level samples, but there are few extraction methods suitable for fine-grained medical relationship types.

[0072] A remote supervised relationship extraction method based on multiple instance learning (MIL). Multiple instance learning is a derivative method of supervised learning algorithms, which regards a bag consisting of multiple examples as a whole to learn a classification model. In the training set, each bag consists of multiple unlabeled examples. The goal is to learn a classifier from these examples, and the task is to predict the label of an unseen bag. Multiple instance learning is a weakly supervised learning method between unsupervised learning and supervised learning, which is achieved by relaxing the supervised learning constraints. If the bag contains positive samples, the label of the bag is positive, and the bag is labeled as negative only when all the examples in the bag are negative samples. Taking binary classification as an example, let bag B = {x1, x2, ..., x n}, indicating that the package contains n examples, example x i Corresponding label y i , only if there exists y i = +1, the label of packet B is positive.

[0073] Multi-instance learning (MIL) has a wide range of applications, including text classification and image classification. In remotely supervised relationship extraction tasks, MIL is an effective denoising method, as noise may exist in the set of relationship samples automatically annotated by the knowledge base. Using noisy data to train a model can lead to problems such as reduced accuracy and poor generalization of the relationship extraction model. Therefore, existing research typically uses MIL for denoising. Specifically, MIL groups relationship samples of the same entity pair into a package. The MIL-based relationship extraction model learns package-level features based on the features of each relationship sample in the package. By learning package-level features, the relationship extraction model can perform relationship classification.

[0074] Table 1 is an example of relationship sample subcontracting:

[0075] Table 1

[0076]

[0077] The MIL-based relation extraction model learns package-level features based on the features of each relation sample in the package, thereby performing relation classification.

[0078] The distantly supervised denoising framework (SENT) is based on a sentence-level denoising framework for distantly supervised relation extraction. This method incorporates the assumption that "in the probability distribution predicted by the model, noise data is mostly distributed in the low-confidence region, while positive data tends to be distributed in the medium-to-high-confidence region." The core components of the model are label filtering and reassignment algorithms. Specifically, when the probability of a sample's predicted category is less than a filtering threshold, the sample's label is filtered out and assigned a label that meets the reassignment threshold. However, due to the varying degrees of convergence for samples from different categories during training, setting a uniform threshold results in poor training results. Therefore, SENT implements dynamic category threshold filtering, using the reassigned samples as new training samples. This framework iterates repeatedly until the model converges. Based on its assumptions, the SENT framework can be understood as first forcing the relation classification model to learn an approximate data distribution, while label filtering can be viewed as identifying outliers in the approximate distribution. Ultimately, the real data distribution is continuously approximated by reassigning sample labels.

[0079] The SENT denoising framework performs denoising by setting dynamic filtering thresholds and label redistribution thresholds for each category. It uses the model's predicted maximum probability for each category to calculate the dynamic filtering threshold, and uses fixed hyperparameters to set the global label redistribution threshold. However, using only the maximum probability of each category to calculate the filtering threshold is too coarse-grained, making it difficult to quantitatively determine the model's degree of convergence for samples from each category. Furthermore, using hyperparameters to set the global redistribution threshold may cause the model to tend to redistribute labels to categories with a larger number of samples, thereby biasing the model.

[0080] like Figure 1 As shown, an embodiment of the present invention provides a denoising method for medical remote supervision relationship extraction, comprising:

[0081] Step S1: Acquire a medical training set, wherein the medical training set includes a plurality of samples, and assign labels to the samples according to their relationship types.

[0082] Specifically, the relationship types of samples in the medical training set are statistically classified, and labels of corresponding relationship types are assigned to the samples according to the relationship types of the samples.

[0083] For example, a medical training set with a large number of samples is selected, which contains 11 relationship types in total, and each relationship type includes multiple samples.

[0084] Table 2 shows the sample size statistics for each relationship type:

[0085] Table 2

[0086] Relationship Type quantity Relationship Type quantity Symptoms\clinical manifestations\diseases 14475 Disease\Occurrence site\Human body 1459 Disease\Available Tests\Tests 5430 Specific populations\high-risk populations\diseases 881 Treatment\Treatable\Diseases 3375 Drugs\Other Names\Drugs 320 Disease\Alias\Disease 3103 Check_Alias_Check 346 Disease\with complications\disease 2548 Disease\Clinic Department\Department 47 Disease\upper and lower levels\disease 2326

[0087] Assign labels to samples according to the statistical relationship types of the samples to generate a medical training set.

[0088] Step S2: performing at least one round of initial training on the relationship classifier based on the medical training set to obtain the predicted probability of each sample corresponding to all the relationship types.

[0089] Specifically, the relationship classifier is initially trained for multiple rounds using a medical training set, and after its ability to discriminate samples of each category is evaluated using validation indicators, the predicted probability of the sample corresponding to all relationship types is output.

[0090] For example, Figure 2 As shown in the figure, the text "resection surgery site is blood vessel" is input into the relational classifier, where the head entity is "resection surgery", the tail entity is "blood vessel", and the entity relationship is "operation / act on / human body". Let each character of the sentence be represented by t = [t1, t2, ..., t N ], the head entity position is [s beg ,s end ], the tail entity position is [o beg ,o end ], the sentence length is N. First, each character in the sentence is input into the embedding layer, and the vector representation of each character is output e=[e1,e,...,e N ], where e i ∈R d , d is the character embedding vector dimension, and R is the vector space.

[0091] Then, the embedding vector is input into the sequence encoding layer. To more effectively capture the bidirectional contextual semantic features, the encoding layer uses a bidirectional long short-term memory network. The output vectors of the forward LSTM and reverse LSTM hidden layers are concatenated as the context representation of each character. The character concatenation formula is:

[0092]

[0093] Among them, h i is the i-th splicing character, h∈R 2×l , l is the hidden layer dimension of LSTM, and R is the vector space.

[0094] Relation extraction can be viewed as a text classification task. However, unlike traditional text classification tasks, which focus more on the overall context of a sentence and classify based on contextual semantic features, relation extraction focuses more on the dependency between the head and tail entities in a sentence. Therefore, we perform an average pooling operation on the hidden layer outputs of the head and tail entity positions and then concatenate them to obtain a text relation representation vector. The formula for the text relation representation vector is:

[0095]

[0096] Among them, h * is the text relationship representation vector, h * ∈R, is the head entity position relation vector, is the tail entity position relationship vector, and R is the vector space.

[0097] Finally, the text relationship representation vector is input into the fully connected layer, and after softmax activation, the probability calculation formula for each relationship type is output as o=Wh * +b, where W∈R C×4l , b∈R C , R is the vector space, l is the hidden layer dimension of LSTM, C is the number of relationship types, h * Represents a vector for textual relations.

[0098] Step S3: determining the uncertainty of each of the relationship types according to the predicted probability of each of the samples corresponding to all of the relationship types.

[0099] Specifically, the uncertainty of each relationship type is calculated based on the predicted probability of each relationship type corresponding to the sample. The uncertainty can be used to judge the discrimination ability of the relationship classifier for each relationship type. The uncertainty can be calculated through expected entropy, low confidence and Gini coefficient.

[0100] Step S4: screening the samples in the medical training set according to the uncertainty of each of the relationship types and the predicted probability of each sample corresponding to all of the relationship types, and reallocating the labels to the screened samples.

[0101] Specifically, the samples in the medical training set are screened according to the uncertainty of each relationship type and the predicted probability of each sample corresponding to all relationship types output by the relationship classifier. The relationship types with weak discrimination ability of the relationship classifier are screened through uncertainty, and then the screened samples are determined according to the predicted probability of each relationship type corresponding to the samples, and the labels of the screened samples are reassigned.

[0102] Step S5: reconstruct the medical training set according to the samples with reallocated labels to generate a denoised medical training set.

[0103] Specifically, the medical dataset is reconstructed using samples with reallocated labels and samples that have not been screened out, and a new medical training set is generated. The new medical training set that has undergone denoising is used for the next round of model training.

[0104] Step S6: input the denoised medical training set into the relationship classifier, and output the new predicted probability of each sample corresponding to all the relationship types.

[0105] Step S7, repeatedly executing steps S3 to S6 to perform iterative training on the relationship classifier for a preset number of times to generate a relationship extraction model.

[0106] Specifically, the new medical numerical training set that has undergone denoising is input into the relational classifier again, and the output result is re-executed with the above-mentioned process of reconstructing the medical training set by redistributing labels to generate a new medical training set that has been denoised and reconstructed again. In this way, the relational classifier is iteratively trained for a predetermined number of times. After completing the preset number of iterative training, a trained relation extraction model is generated.

[0107] Furthermore, since the relation extraction task contains multiple predefined relation types, it is defined as a multi-classification task. The training process uses the cross entropy loss function. For the input s and label y * =[y1,y2,...,y C ],y i ∈{0,1}, where y * is a one-hot vector.

[0108] The loss function is:

[0109]

[0110] Among them, L(θ, y * ) is the difference between the output prediction probability distribution and the prediction probability distribution before inputting the relation classifier, P k is the predicted probability of the kth relationship type, y k is the label of the kth relationship type, and C is the number of relationship types.

[0111] Step S8: input the medical sentence into the trained relationship extraction model, and output the predicted probability of the medical sentence corresponding to each relationship type.

[0112] In this embodiment, the relationship classifier is initially trained for at least one round using a medical training set, so that it has a certain ability to discriminate each relationship type, thereby improving the accuracy of the output prediction probability. The uncertainty of each probability is determined based on the prediction probability of all categories in the data set corresponding to each sample. The ability of the relationship classifier to discriminate each relationship type is measured by the uncertainty. The sample data in the relationship type is processed based on the uncertainty to further improve the ability of the trained relationship classifier to discriminate the relationship type. The samples in the medical training set are screened based on the uncertainty of each relationship type and the prediction probability of each sample corresponding to all relationship types output by the relationship classifier. The relationship types with weak discrimination ability of the relationship classifier are further classified based on the uncertainty. The process of screening is carried out, and then the screened samples are determined according to the predicted probabilities of each relationship type corresponding to the samples, and the screened samples are reassigned labels, and the noise samples are screened, and the relationship types of the samples are changed by reassigning labels to achieve the effect of noise reduction. The ability of the relationship classifier to distinguish relationship types with weak discrimination ability can be improved, and the medical training set is reconstructed according to the samples with reallocated labels, and input into the relationship classifier again. The relationship classifier that has undergone initial training is iteratively trained through the above method, and the ability of the relationship classifier to distinguish relationship types with weak discrimination ability is continuously improved through iterative training, thereby improving the accuracy of the predicted probability output by the relationship extraction model obtained after iterative training, and finally achieving accurate relationship extraction of medical data.

[0113] Optionally, determining the uncertainty of each of the relationship types according to the predicted probability of each of the samples corresponding to all of the relationship types includes:

[0114] Determine the entropy value of the sample according to the predicted probability of the sample corresponding to all the relationship types and the predicted entropy value formula;

[0115] Determine the expected entropy of the relationship type according to the entropy values ​​of all the samples of the same relationship type and an expected entropy formula;

[0116] The expected entropy is determined as the uncertainty of the relationship type.

[0117] Specifically, the uncertainty of each category is calculated based on the predicted probability of each relationship type of the sample output by the relationship classifier, and the calculation method adopts the expected entropy of each relationship type sample.

[0118] Furthermore, the entropy value of each sample is calculated by the predicted entropy value formula, and then the expected entropy of the entropy value of each category sample is calculated according to the entropy value and the expected entropy formula, wherein the number of samples of each relationship type is counted by the indicator function.

[0119] In this optional embodiment, since entropy is a measure of the state of matter, it can be used to measure the uncertainty of the system. The larger the entropy value, the more unstable the system. By calculating the expected entropy of each relationship type, the discriminant ability of the relationship classifier for samples of each relationship type can be measured. The larger the expected entropy, the higher the uncertainty of the relationship type, which means that the classifier has a weaker discriminant ability for samples of this category. Conversely, the relationship classifier has a stronger discriminant ability for samples of this relationship type and the predicted probability is more confident.

[0120] Optionally, the predicted entropy value formula is:

[0121]

[0122] The expected entropy formula is:

[0123]

[0124] Among them, e i is the entropy value of the i-th sample in the medical training set, C is the number of relationship types in the medical training set, is the predicted probability of the jth relationship type of the i-th sample, E(e j ) is the expected entropy of the jth relationship type, N j is the number of samples of the jth relationship type, S is the medical training set, S i is the i-th sample in the medical training set.

[0125] Optionally, screening the samples in the medical training set according to the uncertainty of each relationship type and the predicted probability of each sample corresponding to all the relationship types, and reallocating the labels to the screened samples, includes:

[0126] Determining a label filtering threshold for the relationship type according to the uncertainty of the relationship type, the maximum predicted probability of all the samples corresponding to the relationship type, and a label filtering threshold formula;

[0127] When the predicted probability of the relationship type corresponding to the sample in the relationship type is less than the label filtering threshold of the relationship type, determining that the sample is the sample with a reallocated label;

[0128] The labels of the samples with reallocated labels are reallocated.

[0129] Optionally, reallocating the labels of the samples with reallocated labels includes:

[0130] Determine a label reallocation threshold for the relationship type according to the uncertainty of the relationship type, the maximum predicted probability of all the samples corresponding to the relationship type, and a label filtering threshold formula;

[0131] When the maximum predicted probability of the samples to be reassigned labels is greater than the label reassignment threshold of the relationship type corresponding to the maximum predicted probability, reassigning the labels of the samples according to the relationship type;

[0132] When the maximum predicted probability of the sample to be reassigned labels is less than or equal to the label reassignment threshold of the relationship type corresponding to the maximum predicted probability, the original label of the sample is retained.

[0133] For example, Figure 3 As shown in the figure, the medical training set contains three relationship types, each with three samples. These samples are fed into the current relationship classifier for prediction, resulting in multiple probability distributions. The global label filtering threshold hyperparameter is 0.2, the global label reassignment threshold hyperparameter is 0.8, the α hyperparameter is 0.05, and the β hyperparameter is 1.5. These hyperparameters affect uncertainty and need to be adjusted based on actual training results. Calculation and comparison of expected entropy and dual thresholds reveal that relationship type 0 has a maximum predicted probability of 0.8, an expected entropy of 0.68, a label filtering threshold of 0.15, and a label reassignment threshold of 0.76. For relationship type 1, the maximum predicted probability is 0.6, the expected entropy is 0.82, the label filtering threshold is 0.11, and the label reassignment threshold is 0.59. For relationship type 2, the maximum predicted probability is 0.7, the expected entropy is 0.77, the label filtering threshold is 0.13, and the label reassignment threshold is 0.69. In the medical training set processed by the relationship classifier, the predicted probability of relationship type 1 corresponding to the third sample of relationship type 1 is 0.05, which is less than the label filtering threshold of 0.11 for relationship type 1 and meets the conditions for reallocation of labels. The maximum value of the predicted probability in the third sample is 0.8, which is greater than the label reallocation threshold of relationship type 0. Therefore, category 0 is the label of this sample in the next round of training set, and the data reconstruction of the current round is finally completed.

[0134] Furthermore, if the maximum value of the predicted probability in the third sample is less than or equal to the label reallocation threshold of relation type 0, the third sample retains its original label unchanged.

[0135] Furthermore, the label redistribution process formula is:

[0136]

[0137] in, is the reallocated label for the n+1th round, is the predicted probability of the kth relationship type of the i-th sample, Threshold for label reassignment.

[0138] In this optional embodiment, the label filtering threshold and the label reallocation threshold are used to screen samples of the relationship type whose discrimination ability of the relationship classifier determined by uncertainty is weak, so as to realize noise reduction processing of the noise samples in the medical training set, so that the relationship classifier trained by the reconstructed medical training set gradually improves the discrimination ability of the relationship type samples, and the prediction becomes gradually more confident, then the uncertainty of the relationship type is reduced, the label filtering threshold is increased, and the label reallocation threshold is reduced, and finally the constraints on the label filtering of the relationship type samples and the reallocation of other category samples to this category are gradually reduced.

[0139] Optionally, the tag filtering threshold formula is:

[0140]

[0141] The label reallocation threshold is:

[0142]

[0143] in, is the label filtering threshold of the j-th relationship type, is the label reallocation threshold, U (j) is the uncertainty of the j-th relationship type, TH F is the global filtering threshold hyperparameter, TH R is the global label redistribution threshold hyperparameter, α and β are hyperparameters, and σ is the activation function.

[0144] Optionally, it also includes:

[0145] When the number of iterative training rounds is greater than a first preset threshold, the iterative training is stopped, and / or

[0146] When the number of samples that need to reallocate the labels is less than a second preset threshold, stopping the iterative training, and / or

[0147] When the index of the validation set is less than the preset index, the iterative training is stopped.

[0148] Specifically, to prevent overfitting, an early stopping mechanism is set up for model training. For example, if the number of iterations exceeds 8 and / or the number of samples with reallocated labels does not exceed 50, the iterative training is stopped. At the same time, the changes in the indicators of the validation set are detected. If the indicators of the validation set do not improve, the iterative training is also stopped.

[0149] In this optional embodiment, by setting up an early stopping mechanism, iterative training can be stopped when the data cannot be further denoised by reconstructing the medical training set during the iterative training process, or when the iterative training does not significantly improve the prediction probability of the model, thereby effectively preventing overfitting, avoiding unnecessary repeated training, saving resources, and improving model training efficiency.

[0150] Optionally, it also includes:

[0151] Determining the relationship type corresponding to the maximum predicted probability of the output medical sentence as the relationship type of the medical sentence;

[0152] Assigning labels to the medical sentences according to the relationship type

[0153] Specifically, the relationship type corresponding to the maximum predicted probability output by the relationship extraction model is the relationship type with the highest accuracy between the medical sentence and the actual relationship type, and the medical sentence is assigned a label based on the relationship type, thereby realizing the mining of medical sentence relationship facts and constructing a medical knowledge graph.

[0154] like Figure 4 As shown, another embodiment of the present invention provides a denoising device for medical remote supervision relationship extraction, comprising:

[0155] an acquisition module, configured to acquire a medical training set, wherein the medical training set includes a plurality of samples, and assign labels to the samples according to relationship types of the samples;

[0156] a training module configured to perform at least one round of initial training on the relationship classifier based on the medical training set to obtain a predicted probability that each sample corresponds to all the relationship types; input the denoised medical training set into the relationship classifier to output a new predicted probability that each sample corresponds to all the relationship types; and repeatedly perform steps S3 to S6 to perform a preset number of iterative training on the relationship classifier to generate a relationship extraction model;

[0157] A processing module, configured to determine the uncertainty of each of the relationship types according to the predicted probability of each sample corresponding to all the relationship types;

[0158] a screening module, configured to screen the samples in the medical training set according to the uncertainty of each of the relationship types and the predicted probability of each sample corresponding to all of the relationship types, and reallocate the labels to the screened samples;

[0159] a reconstruction module, configured to reconstruct the medical training set according to the samples with reallocated labels to generate a denoised medical training set;

[0160] The relationship extraction module is used to input the medical sentence into the trained relationship extraction model and output the predicted probability of the medical sentence corresponding to each relationship type.

[0161] A denoising device for medical remote supervision relationship extraction in an embodiment of the present invention has similar technical effects to the denoising method for medical remote supervision relationship extraction described above, and will not be described in detail here.

[0162] Another embodiment of the present invention provides an electronic device comprising: a memory and a processor; the memory is used to store a computer program; the processor is used to implement the above-mentioned denoising method for medical remote supervision relationship extraction when executing the computer program.

[0163] An electronic device in an embodiment of the present invention has a technical effect similar to that of the above-mentioned noise reduction method for extracting medical remote supervision relationships, and will not be described in detail here.

[0164] Furthermore, to construct data for medical remote supervision relationship extraction, it is necessary to align medical entity text data with an existing medical knowledge base. The first 26 chapters of Diagnostics were selected as the medical text data. Compared to data sources such as web crawling, using professional books ensures the accuracy of knowledge. The exposition and text formatting of professional books are relatively uniform, which facilitates the model's learning of text features. The Wi Lab's medical knowledge graph, the Network Intelligent Laboratory Medical Knowledge Graph (WiMedKG), was also used as the medical knowledge base. This contains 29 entity types and 118 relationship types.

[0165] Table 3 shows the entity and quantity statistics in the medical knowledge graph:

[0166] Table 3

[0167] entity quantity entity quantity Clinical findings 110 Check results 516 human body 3592 treat 1205 enterprise 5415 substance 2520 Mode of transmission 230 Specific populations 5606 Health risk factors 448 disease 68999 Dosage form 19 Disease classification 48 Medical insurance area 2 symptom 34955 Medical insurance classification 4 sociology 968 Inspection status 3 Department 135 Prescription status 7 drug 206711 External events 4786 drug 18687 microorganism 18 Drug prescriptions 7 operate 1748 Storage conditions 3320 Effect 252 Anesthesia method 6 examine 6791

[0168] We reverse-annotate entities in medical text using the longest string matching method. If a corresponding medical fact exists in WiMedKG, the relationship is annotated; otherwise, it is annotated as no relationship (NA). Next, we align the entity text data to WiMedKG using distant supervision, limiting the maximum textual distance between entity pairs to 80 characters. We also package the existing sentence-level relationship annotation data.

[0169] Table 4 shows the sample data statistics at the sentence level:

[0170] Table 4

[0171] Relationship Type quantity Relationship Type quantity No relationship 948761 Disease\subClassOf\Disease 3895 Disease\Available Tests\Tests 53529 Operation\Object\Human Body 3792 Symptoms\clinical manifestations\diseases 41345 Examination\Clinic Department\Department 3400 Disease\Occurrence site\Human body 26190 Disease\Cause\Symptoms 3094 Disease\Alias\Disease 25820 Drugs\Ingredients\Drugs 2313 Disease\Clinic Department\Department 25467 Check\upper and lower positions\check 1969 Treatment\Treatable\Diseases 23463 Drugs\upper and lower levels\drugs 1679 Check\Affected Object\Human Body 22502 Operation\Operation Object\Human Body 1605 Symptoms\is the basis for diagnosis\disease 21388 Action\Alias\Action 1288 Symptoms\Department of treatment\Department 18215 Drugs\Other Names\Drugs 1267 Disease\with complications\disease 17511 Symptoms\accompanied by\symptoms 766 Symptoms\Check available\Check 12900 Operation\Clinic Department\Department 761 Specific populations\high-risk groups\diseases 9267 Symptoms\Occurrence site\Human body 693 Drugs\Treatable\Diseases 7618 Operation\Contraindications\Specific groups 566 Drugs\Contraindicated groups\Specific groups 5087 Drugs\Effective\Effective 527 Check_Alias_Check 4684 Disease\English name\Disease 247 Operation\Treatable\Disease 4458

[0172] Table 5 shows the sample data statistics at the package level:

[0173] Table 5

[0174]

[0175]

[0176] Because some of the designed experiments used multi-instance learning methods, the existing sentence-level relationship annotation data was subpackaged. Statistics show that the number of sentence-level samples in the constructed relationship extraction data is 1,540,173, covering a total of 33 relationship types. Among them, samples of the no-relationship (NA) category account for 3 / 4, and the ratio of positive and negative samples is 1:3, showing a clear long-tail distribution. The large proportion of no-relationship samples poses great difficulties for model denoising. After subpackaged the sentence-level data, 884,554 package-level samples were obtained, with an average of 3.4 sentences per non-NA package. A comparison shows that the proportion of samples of each relationship type before and after subpackage is similar, demonstrating a high degree of consistency in label distribution before and after subpackaged.

[0177] For data reconstruction, we used sentence-level data from the Diagnostics journal, using remotely supervised sentence-level data. We implemented the ULFR framework (ULFR) based on category uncertainty using a natural language deep learning framework (Pytorch). Hyperparameters were set as follows: the maximum text length was 120 characters; texts exceeding this length were truncated, and shorter texts were padded with "[PAD]" characters. The sentences and their head and tail entity positions were input into a relational classifier with an embedding layer dimension of 300 and a sequence encoding hidden layer dimension of 256. The dataset contained 33 relation types, resulting in a final output layer dimension of 33. The core relational parameters of the framework were set as follows: the classifier was initially trained for two epochs, with a global filtering threshold of 0.2, a filtering weight α of 0.05, and a global redistribution threshold of 0.8. The redistribution weight β was set to 1.5, the model training learning rate was 0.001, the dropout probability was 0.5, the training batch size was 128, and the training epochs were 64. Furthermore, to prevent overfitting, an early stopping mechanism was implemented in the model training.

[0178] like Figure 5As shown in the figure, the micro-averaged F1 (MicroF1) values ​​of non-NA samples in the test set change with each training round during the framework iteration. Although the distant supervision data contains noise, according to the assumption, the noise accounts for a small proportion of the total data. Therefore, the test set evaluation metrics are only used to observe the model's discriminative ability and convergence level and do not represent the true performance of the test set. As can be seen, after two rounds of initial training, the MicroF1 of positive examples in the test set is close to 0.85, indicating that the model has achieved preliminary discriminative ability for these samples. Among them, the least confidence (LC) is another uncertainty calculation method. Compared with the threshold calculation method and LC uncertainty threshold calculation method used in the sentence-level denoising framework (SENT) based on negative example training, the model based on the entropy threshold calculation method has smoother training and more stable convergence. In subsequent training, the entropy threshold calculation method continued to show superior performance, and the final model achieved a MicroF1 of positive examples on the test set close to 0.89.

[0179] like Figure 6 The number of label reallocation samples shown changes with training rounds. The number of label reallocation samples for all three threshold calculation methods decreases with each training round, indicating that the data distribution gradually stabilizes with the continuous iteration of the denoising framework. In contrast to the fluctuations seen in some training phases of SENT and LC, the number of label reallocation samples for the entropy threshold calculation method decreases, resulting in a more stable data distribution.

[0180] like Figure 7 、 Figure 8 、 Figure 9 and Figure 10 As shown in the figure, the thresholds of the filtering threshold (Filtering Threshold) and the relabeling threshold (Relabeling Threshold) for four randomly sampled relationship types change with the number of iterations under the ULFR framework. As the number of iterations increases, the label filtering threshold shows an overall upward trend, and the label relabeling threshold shows a downward trend. However, the threshold changes of some categories may fluctuate during the training process. By analyzing the threshold calculation method of the ULFR framework, this may be related to the amount of noise in each category of the original data. The lower the expected entropy, the more stable the threshold change. If there is more noise in this category in the original data, the data distribution of this category will change more significantly, and the threshold change will be more unstable.

[0181] The present invention adopts the CMEIE supervised relation extraction dataset and uses this data to test the performance of the relation extraction model in the denoising framework.

[0182] Table 6 is a list of test result data:

[0183] Table 6

[0184] frame Acc(%) MicroF(%) MacroF(%) NoDenoising 27.777 42.564 24.224 SENT 28.323 43.363 25.985 ULFR(LC) 34.453 49.978 29.700 ULFR(Ent) 35.226 50.840 29.079

[0185] Among them, "NoDenoising" means that no denoising algorithm is added, and sentence-level remote supervision data is directly used for model training. "SENT" is the remote supervision denoising framework mentioned above. "LC" and "Ent" respectively represent the uncertainty calculation methods of the label filtering and reallocation framework (ULFR) based on category uncertainty. The former calculates the minimum confidence of the sample, and the latter calculates the entropy.

[0186] The relation extraction model trained directly without any denoising algorithm exhibited the worst test set evaluation metrics, indicating a significant impact of noise on model learning and demonstrating the necessity of denoising remotely supervised data. Compared to the other two denoising methods, ULFR(Ent) achieved the best performance in both accuracy and micro-average F-value, only slightly lower than ULFR(LC) in macro-average F1, demonstrating the effectiveness of the ULFR denoising framework. Furthermore, compared to the SENT framework, the ULFR framework exhibits superior denoising capabilities, resulting in a trained relation extraction model with greater robustness and generalization capabilities. Furthermore, considering factors such as the training process and changes in data distribution, entropy calculation is a more effective method for measuring category uncertainty.

[0187] Overall, the CMEIE dataset has lower evaluation metrics than the distant supervision test set, especially the macro-average F1 index, which is less than 0.3. This may be due to the large differences in text representation and data distribution caused by different data sources.

[0188] The above reconstructed data is divided into packages, and multiple relation extraction models based on multi-instance learning are applied to verify that the performance of the relation extraction model trained on the denoised and reconstructed package-level data is stronger than that trained on the original package-level data, that is, ablation is performed only at the training data level to verify the performance of the relation extraction model. The following four relation extraction models based on multi-instance learning are selected, among which Model 1 is the PCNN+ONE model based on the segmented pooling convolutional neural network (PCNN), which selects the sentence with the highest confidence from the package to participate in learning; Model 2 is PCNN+ATT, which is also based on PCNN and uses the attention mechanism to aggregate sentence representations as package representations; Model 3 is SEG, which is based on the PCNN structure and adds entity attention and gating mechanisms to enhance representation capabilities; Model 4 is CIL, which introduces contrastive learning to make the representations of the same triplet examples as similar as possible in the semantic space, such as Figure 11 As shown in the figure, the PR curve comparison results of the PCNN+ONE model on the test set are as follows: Figure 12 As shown, the PR curve comparison results of the PCNN+ATT model on the test set are shown, where the area under the PR curve, Figure 13As shown in the figure, the PR curve comparison results of the SEG model on the test set are as follows: Figure 14 The following figure shows the PR curve comparison results of the CIL model on the test set.

[0189] AUC is the area under the PR curve; P@1000: the precision of the top 1000 most confident samples; P@2000: the precision of the top 2000 most confident samples; P@3000: the precision of the top 3000 most confident samples; P@M: the mean of P@1000, P@2000, and P@3000.

[0190] Table 7 shows the specific values ​​of AUC and P@N indicators:

[0191] Table 7

[0192] method AUC P@1000 P@2000 P@3000 P@M PCNN+ONE 0.523 0.871 0.871 0.657 0.770 SENT-PCNN+ONE 0.580 0.994 0.839 0.961 0.824 ULFR-PCNN+ONE 0.601 0.941 0.844 0.736 0.840 PCNN+ATT 0.662 0.904 0.895 0.884 0.894 SENT-PCNN+ATT 0.668 0.926 0.913 0.906 0.915 ULFR-PCNN+ATT 0.732 0.952 0.937 0.926 0.938 SEG 0.811 0.984 0.974 0.967 0.975 SENT-SEG 0.827 0.977 0.971 0.965 0.971 ULFR-SEG 0.835 0.981 0.974 0.967 0.974 CIL 0.807 0.938 0.935 0.919 0.930 SENT-CIL 0.820 0.922 0.912 0.905 0.913 ULFR-CIL 0.836 0.951 0.951 0.919 0.940

[0193] Comprehensive chart analysis shows that the model can achieve a precision of nearly 0.9 or higher for high-confidence prediction samples, and also achieves excellent results in the Area Under the Curve (AUC) metric on the test set. In most cases, models trained on denoised and reconstructed data outperform relation extraction models trained on the original data on the same test set, with improvements of approximately 2% to 8% in AUC. This is because the denoising process brings the samples in the training set closer to the true distribution, thereby improving the generalization ability of the relation extraction model and demonstrating the effectiveness of the denoising framework. Experiments with two more basic models, PCNN+ONE and PCNN+ATT, show significant performance improvements by denoising and reconstructing the training data. In contrast, in experiments with the SEG and CIL models, the performance improvements on the test set for models trained on denoised data are less pronounced. This is because multi-instance learning can be used for sample denoising, and models based on this method inherently have a certain degree of noise resistance. Furthermore, the complex structures of the SEG and CIL models enable them to achieve excellent results on the test set even when the training data is not denoised. Therefore, for the SEG and CIL models, improving training data quality only leads to limited improvements in model performance. Furthermore, the ULFR framework exhibits stronger denoising capabilities compared to the SENT framework. This is because the ULFR framework introduces a more fine-grained uncertainty dual-threshold calculation method and verifies the effectiveness of its denoising approach using both labeled data testing and remotely supervised data evaluation.

[0194] Furthermore, the ULFR framework was compared on examples before and after denoising of the Diagnostics remote supervision data, and several samples were randomly selected from them.

[0195] Table 8 shows a random sampling example:

[0196] Table 8

[0197]

[0198] In the first example, there is no relationship fact between "dietary conditioning" and "hyperlipoproteinemia" in WiMedKG, so it is labeled as no relationship or empty relationship after remote supervision. However, the entity pair is expressed as a "treatable" relationship in the sentence, and the ULFR framework correctly labels it, proving that the framework can pay attention to true negative examples (True Negative). In the second example, the entity pair has a relationship fact in the knowledge base, but in the specific sentence there is only a parallel relationship, and no "clinical manifestation" relationship. The framework ultimately assigns it as no relationship, which also proves that the framework can pay attention to false positives (False Positive). The following two use case analyses are examples of label reallocation between non-NA relationships, where the head and tail entity types are the same, further demonstrating the denoising ability of the ULFR framework on fine-grained many-to-many medical relationship categories.

[0199] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM). In this application, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present invention. In addition, the functional units in the various embodiments of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or software functional units.

[0200] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A denoising method for medical remote supervision relationship extraction, characterized in that: include: S1, obtaining a medical training set, wherein the medical training set includes a plurality of samples, and assigning labels to the samples according to relationship types of the samples; S2, performing at least one round of initial training on the relationship classifier based on the medical training set to obtain a predicted probability of each sample corresponding to all the relationship types; S3, determining the uncertainty of each of the relationship types based on the predicted probability of each of the samples corresponding to all of the relationship types; including: determining the entropy value of the sample based on the predicted probability of the sample corresponding to all of the relationship types and a predicted entropy value formula; determining the expected entropy of the relationship type based on the entropy values ​​of all of the samples of the same relationship type and an expected entropy formula; and determining the expected entropy as the uncertainty of the relationship type; S4, screening the samples in the medical training set according to the uncertainty of each relationship type and the predicted probability of each sample corresponding to all the relationship types, and reallocating the labels to the screened samples; including: determining the label filtering threshold of the relationship type according to the uncertainty of the relationship type, the maximum predicted probability of all the samples corresponding to the relationship type, and the label filtering threshold formula; when the predicted probability of the relationship type corresponding to the sample in the relationship type is less than the label filtering threshold of the relationship type, determining the sample as the sample with the reallocated label; and reallocating the label of the sample with the reallocated label; S5, reconstructing the medical training set according to the samples with reallocated labels to generate a denoised medical training set; S6, inputting the denoised medical training set into the relationship classifier, and outputting the new predicted probability of each sample corresponding to all the relationship types; S7, repeatedly executing steps S3 to S6 to perform iterative training on the relationship classifier for a preset number of times to generate a relationship extraction model; S8, inputting the medical sentence into the trained relationship extraction model, and outputting the predicted probability of the medical sentence corresponding to each relationship type.

2. The denoising method for medical remote supervision relationship extraction according to claim 1, characterized in that: The predicted entropy value formula is: ； The expected entropy formula is: ； in, is the entropy value of the i-th sample in the medical training set, C is the number of relationship types in the medical training set, is the predicted probability of the jth relationship type of the i-th sample, is the expected entropy of the j-th relation type, is the number of samples of the jth relationship type, S is the medical training set, is the i-th sample in the medical training set.

3. The denoising method for medical remote supervision relationship extraction according to claim 1, characterized in that: The reallocating the labels of the samples with reallocated labels includes: Determine a label reallocation threshold for the relationship type according to the uncertainty of the relationship type, the maximum predicted probability of all the samples corresponding to the relationship type, and a label filtering threshold formula; When the maximum predicted probability of the samples to be reassigned labels is greater than the label reassignment threshold of the relationship type corresponding to the maximum predicted probability, reassigning the labels of the samples according to the relationship type; When the maximum predicted probability of the sample to be reassigned labels is less than or equal to the label reassignment threshold of the relationship type corresponding to the maximum predicted probability, the original label of the sample is retained.

4. The denoising method for medical remote supervision relationship extraction according to claim 1, characterized in that: The label filtering threshold formula is: ； The label reallocation threshold is: ； in, is the label filtering threshold of the j-th relationship type, Reassign a threshold value for the label, is the uncertainty of the j-th relationship type, is the global filtering threshold hyperparameter, is the global label redistribution threshold hyperparameter, α and β are hyperparameters, and σ is the activation function.

5. The denoising method for medical remote supervision relationship extraction according to claim 1, characterized in that: Also includes: When the number of iterative training rounds is greater than a first preset threshold, the iterative training is stopped, and / or When the number of samples that need to reallocate the labels is less than a second preset threshold, stopping the iterative training, and / or When the index of the validation set is less than the preset index, the iterative training is stopped.

6. The denoising method for medical remote supervision relationship extraction according to claim 1, characterized in that: Also includes: Determining the relationship type corresponding to the maximum predicted probability of the output medical sentence as the relationship type of the medical sentence; The medical sentences are assigned labels according to the relationship types.

7. A noise reduction device for medical remote supervision relationship extraction, characterized in that: include: an acquisition module, configured to acquire a medical training set, wherein the medical training set includes a plurality of samples, and assign labels to the samples according to relationship types of the samples; A training module is used to perform at least one round of initial training on the relationship classifier according to the medical training set to obtain the predicted probability of each sample corresponding to all the relationship types; determine the uncertainty of each relationship type according to the predicted probability of each sample corresponding to all the relationship types; including: determining the entropy value of the sample according to the predicted probability of the sample corresponding to all the relationship types and the predicted entropy value formula; determining the expected entropy of the relationship type according to the entropy values ​​of all the samples of the same relationship type and the expected entropy formula; determining the expected entropy as the uncertainty of the relationship type; screening the samples in the medical training set according to the uncertainty of each relationship type and the predicted probability of each sample corresponding to all the relationship types, and reallocating the screened samples to the label; including: determining the label filtering threshold of the relationship type according to the uncertainty of the relationship type, the maximum predicted probability of all the samples corresponding to the relationship type and the label filtering threshold formula; when the predicted probability of the relationship type corresponding to the sample in the relationship type is less than the label filtering threshold of the relationship type, determining that the sample is the sample of the reallocated label; reallocating the label of the sample of the reallocated label; reconstructing the medical training set according to the sample of the reallocated label to generate a denoised medical training set; inputting the denoised medical training set into the relationship classifier, and outputting the new predicted probability of each sample corresponding to all the relationship types; performing a preset number of iterative training on the relationship classifier through the above steps to generate a relationship extraction model; The relationship extraction module is used to input the medical sentence into the trained relationship extraction model and output the predicted probability of the medical sentence corresponding to each relationship type.

8. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the denoising method for medical remote supervision relationship extraction according to any one of claims 1 to 6 when executing the computer program.

Images