Network model training method, sample relationship class acquisition method, and electronic device

Abstract

The application discloses a network model training method, a sample relation category acquisition method and electronic equipment. The method comprises the following steps: obtaining a supervision data set, wherein the supervision data set comprises a plurality of training samples; performing clustering processing on the supervision data set according to sample relation categories of the plurality of training samples to obtain a plurality of clusters; and performing iterative training on an initial network model by using the plurality of clusters to obtain a target network model, wherein the initial network model and the target network model are used for extracting supervision relations. The application solves the technical problem that the relation extraction network model in the prior art has low learning ability for complex samples and inhibits the learning of diverse samples.

Description

Technical Field

[0001] This invention relates to the field of network model training, and more specifically, to a network model training method, a method for obtaining sample relationship categories, and an electronic device. Background Technology

[0002] Relation extraction is of great significance in related technologies. The massive amount of data and literature in this field contains a wealth of knowledge. If this knowledge can be structured, it will not only facilitate computer storage, but also provide convenience for many downstream applications, such as retrieval and question answering.

[0003] Previous research in the field of supervised relation extraction has largely focused on how to minimize the interference of noisy data during the learning process of the model on the supervised dataset. Whether it is through multi-instance learning to select the sample with the highest posterior probability in a bag to represent the entire dataset and use it as the feature of the entire bag, or through attention mechanisms to weight all samples in the dataset, the model always suppresses the influence of noisy data by learning from a few representative samples in a dataset.

[0004] However, relation extraction relies on large-scale labeled data. Although supervision can automatically generate labeled entity relation extraction training data using existing knowledge bases, the generated data has many problems, including the in-dataset sample noise problem that has been widely discussed in previous studies, as well as the diversity suppression problem proposed for the first time in this application.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This invention provides a network model training method, a sample relationship category acquisition method, and an electronic device to at least address the technical problem in the prior art where relationship extraction network models have low learning ability for complex samples and suppress the learning of diverse samples.

[0007] According to one aspect of the present invention, a network model training method is provided, comprising: acquiring a supervised dataset, wherein the supervised dataset includes: multiple training samples; performing clustering processing on the supervised dataset according to the sample relationship categories of the multiple training samples to obtain multiple clusters; and iteratively training an initial network model using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0008] According to another aspect of the present invention, a method for obtaining sample relationship categories is also provided, comprising: obtaining a sample to be predicted; constructing a sample feature vector using the sample to be predicted; inputting the sample feature vector into a target network model and outputting the sample relationship category corresponding to the sample to be predicted, wherein the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset comprising: multiple samples to be trained, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple samples to be trained to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, and the initial network model and the target network model being used to extract supervised relationships.

[0009] According to another aspect of the present invention, a method for obtaining sample relationship categories is also provided, comprising: receiving a currently input sample to be predicted; sending the sample to be predicted to a server; receiving a sample relationship category corresponding to the sample to be predicted from the server, wherein the sample relationship category is obtained by the server using a target network model, the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset comprising: multiple training samples, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model being used to extract supervised relationships; and displaying the sample relationship category locally on a client.

[0010] According to another aspect of the present invention, a method for obtaining sample relationship categories is also provided, comprising: receiving a sample to be predicted from a client; obtaining the sample relationship category corresponding to the sample to be predicted using a target network model, wherein the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset comprising: multiple training samples, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model being used to extract supervised relationships; returning the sample relationship category to the client and displaying the sample relationship category locally on the client.

[0011] According to another aspect of the present invention, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, the device where the non-volatile storage medium is located executes any of the above-described network model training methods and any of the above-described sample relationship category acquisition methods.

[0012] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions to perform the following processing steps: acquiring a supervised dataset, wherein the supervised dataset comprises: a plurality of training samples; clustering the supervised dataset according to the sample relationship categories of the plurality of training samples to obtain a plurality of clusters; iteratively training an initial network model using the plurality of clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0013] In this embodiment of the invention, a supervised dataset is obtained, which includes multiple training samples; the supervised dataset is clustered according to the sample relationship categories of the multiple training samples to obtain multiple clusters; the initial network model is iteratively trained using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0014] It is noteworthy that this application proposes a new method for constructing a dataset. During the training process, the construction of the supervised dataset is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples of each sample's relation category. The supervised dataset is then clustered to obtain multiple clusters. These multiple clusters are then used to iteratively train the initial network model to obtain better performance and the target network model.

[0015] Therefore, the embodiments of this application achieve the goal of improving the learning ability of the network model for complex samples, thereby realizing the technical effect of improving the performance of extracting supervised relations, and thus solving the technical problem that the existing relation extraction network model has a low learning ability for complex samples and inhibits the learning of diverse samples. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a network model training method is shown.

[0018] Figure 2 This is a flowchart of a network model training method according to an embodiment of the present invention;

[0019] Figure 3This is a flowchart of a method for obtaining sample relationship categories according to an embodiment of the present invention;

[0020] Figure 4 This is a flowchart of another method for obtaining sample relationship categories according to an embodiment of the present invention;

[0021] Figure 5 This is a flowchart of another method for obtaining sample relationship categories according to an embodiment of the present invention;

[0022] Figure 6 This is a schematic diagram of the structure of a network model training device according to an embodiment of this application;

[0023] Figure 7 This is a schematic diagram of a sample relationship category acquisition device according to an embodiment of this application;

[0024] Figure 8 This is a schematic diagram of another sample relationship category acquisition device according to an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of another sample relationship category acquisition device according to an embodiment of this application;

[0026] Figure 10 This is a structural block diagram of another computer terminal according to an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0030] Relation extraction refers to the extraction of structured triplet information from large-scale unstructured or semi-structured natural language text.

[0031] Multiple-instance learning (MIB) is a method that evolved from supervised learning. It defines a "package" as a collection of multiple instances and has wide applications. Instead of receiving a single set of individually labeled instances, learners receive a set of labeled packages, each containing multiple instances.

[0032] Attention mechanism: refers to the ability of an attention mechanism to enable a neural network to focus on a subset of its inputs (or features), that is, to select specific inputs.

[0033] Example 1

[0034] According to an embodiment of the present invention, a network model training method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a network model training method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the network model training method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned network model training method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory located relative to the processor 102, and these memories can be connected to the computer terminal 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0040] Under the aforementioned operating environment, this application provides the following: Figure 2 The network model training method shown. Figure 2 This is a flowchart of a network model training method according to an embodiment of the present invention, such as... Figure 2 As shown, the above network model training methods include:

[0041] Step S202: Obtain a supervised dataset, wherein the supervised dataset includes: multiple training samples;

[0042] Step S204: Cluster the supervised dataset according to the sample relationship categories of the multiple training samples to obtain multiple clusters;

[0043] Step S206: Iteratively train the initial network model using the aforementioned multiple clusters to obtain the target network model, wherein the initial network model and the target network model are used to extract supervision relationships.

[0044] In this embodiment of the invention, a supervised dataset is obtained, which includes multiple training samples; the supervised dataset is clustered according to the sample relationship categories of the multiple training samples to obtain multiple clusters; the initial network model is iteratively trained using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0045] It is noteworthy that this application proposes a new method for constructing a dataset. During the training process, the construction of the supervised dataset is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples of each sample's relation category. The supervised dataset is then clustered to obtain multiple clusters. These multiple clusters are then used to iteratively train the initial network model to obtain better performance and the target network model.

[0046] Therefore, the embodiments of this application achieve the goal of improving the learning ability of the network model for complex samples, thereby realizing the technical effect of improving the performance of extracting supervised relations, and thus solving the technical problem that the existing relation extraction network model has a low learning ability for complex samples and inhibits the learning of diverse samples.

[0047] It should be noted that the above network model training method can essentially be understood as a method for extracting sentence-level supervision relations based on clustering enhancement, and it can be applied, but is not limited to, application scenarios that extract supervision relations based on clustering enhancement.

[0048] For example, it can be applied to medical fields that require the construction of knowledge graphs using information extraction technology, including family doctor knowledge Q&A, breast cancer knowledge graphs, and orthopedic knowledge Q&A.

[0049] As an optional embodiment, this application embodiment can use information extraction technology to construct a general knowledge graph in the medical field, encompassing various types of knowledge in the medical field, including diseases, drugs, and medical service processes, and build an intelligent dialogue system on top of it to provide various Q&A services for primary care physicians and patients.

[0050] This application's embodiments enable network models to learn samples with complex representations while suppressing noisy data. This application proposes a new dataset construction method: clustering augmentation datasets. Unlike previous studies that treated instances containing identical entity pairs as a single dataset (where the number of samples in all datasets is fixed during training), the construction of clustering augmentation datasets is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples from each category together. Each cluster formed by the clustering algorithm on this set is considered a dataset. Simultaneously, deep neural networks or other feature extractors can update their model parameters based on this dataset. These two modules are interdependent and mutually reinforcing, achieving better performance through iterative training.

[0051] In one optional embodiment, obtaining the above-mentioned supervised dataset includes:

[0052] Step S302: Obtain unstructured text;

[0053] Step S304: Align the above unstructured text with the associated knowledge base to obtain the above supervised dataset.

[0054] Optionally, in this embodiment of the application, a large amount of the aforementioned unstructured text can be obtained from the Internet, and the obtained unstructured text can be aligned with the associated knowledge base to generate a supervised dataset.

[0055] In one optional embodiment, obtaining the aforementioned unstructured text includes:

[0056] Step S402: Determine the list of root web pages to be crawled;

[0057] Step S404: Use a web crawler and a depth-first search algorithm to obtain the corresponding original text from each webpage in the root webpage list above;

[0058] Step S406: Segment the original text to obtain the unstructured text.

[0059] Optionally, in this embodiment of the application, a list of root web pages to be crawled is collected, and a crawler tool is used to crawl each web page in the root web page list using a depth-first search algorithm; then, the original text of the obtained web pages is extracted, and the original text is processed into sentences to obtain unstructured text.

[0060] In an optional embodiment, the unstructured text described above is aligned with the associated knowledge base described above to obtain the supervised dataset, which includes:

[0061] Step S502: Determine whether the sentences in the above unstructured text contain at least one entity in the triples of the above-mentioned associated knowledge base.

[0062] Step S504: When the above sentence contains two entities in the above triple, the above sentence is labeled as a positive sample of the above sample relation category; when the above sentence contains one entity in the above triple and the above sentence also contains another entity of the same category as the other entity in the above triple, the above sentence is labeled as a negative sample of the above sample relation category.

[0063] Step S506: The labeled positive samples and labeled negative samples are determined as the above-mentioned supervised dataset.

[0064] In this embodiment of the application, by determining whether each sentence of the unstructured text contains at least one entity in the triples of the aforementioned associated knowledge base, if the sentence contains two entities of a triple, it is labeled as a positive sample of the relation category; if it contains one entity of a triple and contains another entity of the same category as the other entity in the triple, it is labeled as a negative sample of the relation category; and the labeled positive samples and labeled negative samples are determined as the aforementioned supervised dataset.

[0065] In one optional embodiment, the supervised dataset is clustered according to the sample relationship categories of the plurality of training samples to obtain the plurality of clusters, including:

[0066] Step S602: Use density clustering algorithm to cluster the training samples in the above supervised dataset that belong to the same sample relationship category to obtain the above multiple clusters.

[0067] Optionally, in this embodiment of the application, the above-mentioned supervised dataset is denoted as B = {B} 1 B 2 B R}, where the sample relation category is r, For the i-th dataset By merging all supervised datasets belonging to the same sample relation category r, we obtain In BT r The density-based clustering algorithm (DBSCAN) is used to cluster the training samples in the supervised dataset that belong to the same sample relationship category, resulting in C. r Clusters.

[0068] In one optional embodiment, the initial network model is iteratively trained using the aforementioned multiple clusters to obtain the target network model, which includes:

[0069] Step S702: Obtain the first impact factor and the second impact factor;

[0070] Step S704: Calculate the credibility score using the first parameter, the second parameter, the third parameter, the first influence factor, and the second influence factor.

[0071] Step S706: Obtain the supervision signal of the confidence score based on the posterior probability corresponding to each cluster;

[0072] Step S708: Set the confidence score and the supervision signal as input parameters of the loss function of the confidence posterior constraint, and output the first calculation result;

[0073] Step S710: The initial network model is iteratively trained using the first calculation result to obtain the target network model.

[0074] Optionally, the first impact factor is used to quantify the impact of each cluster size, and the second impact factor is used to assess the probability that each cluster contains samples that correctly describe the corresponding sample relationship category.

[0075] In this embodiment of the application, based on the obtained first influence factor u i Second Influence Factor v i The confidence score R can be calculated using the following formula, along with the first parameter θ, the second parameter λ, and the third parameter β. i :

[0076] R i =θ+λ*u i +β*v i ;

[0077] After calculating the aforementioned credibility score based on the first and second impact factors, the likelihood that the clustered dataset contains the correct representation of this type of relationship can be measured from two aspects.

[0078] Optionally, in this embodiment of the application, based on the posterior probability p(r) corresponding to each cluster... i |B i Obtain the supervision signal for the aforementioned confidence score, where the posterior probability p(r) is... i |B i The posterior probability is used to indicate the confidence level of the cluster. The higher the posterior probability, the more confident the network model is in judging the category of the clustered dataset, and the higher the confidence level is.

[0079] Optionally, embodiments of this application utilize logp(r) i |b i As a monitoring signal for the aforementioned confidence score, the first parameter θ, the second parameter λ, and the third parameter β in the confidence score are updated by minimizing the mean squared error MSE as shown in the following formula.

[0080]

[0081] In this embodiment, the confidence score and the supervision signal are set as input parameters of the loss function of the confidence posterior constraint, and a first calculation result is output. The initial network model is iteratively trained using the first calculation result to obtain the target network model.

[0082] In this embodiment, the loss function RBL of the credibility posterior constraint can be calculated according to the following formula. At the same time, the parameters of the selective attention relationship extraction model can also be updated using the gradient method.

[0083]

[0084] In one optional embodiment, obtaining the aforementioned first influence factor includes:

[0085] Step S802: Obtain the first influence factor based on the number of training samples contained in each of the multiple clusters.

[0086] Optionally, in this embodiment of the application, the influence of clustering data size is quantified by defining a clustering dataset size influence factor (CBSIF) u, resulting in the first influence factor u shown below. i :

[0087] u i =tanh(log(1+size(i)));

[0088] Where size(i) represents the clustering dataset B i The number of training samples included.

[0089] In one optional embodiment, obtaining the second influence factor includes:

[0090] Step S902: Obtain the second influencing factor based on the total number of labeled samples corresponding to each sample relationship category, the feature representation of each labeled sample, and the vector representation of the cluster corresponding to each sample relationship category.

[0091] Optionally, in this embodiment, given a very small number of seed human-annotated samples, it is assumed that the more similar the features of the clustering dataset are to the features of these seed human-annotated samples, the greater the likelihood that the clustering dataset contains samples that correctly describe that class of relationships. To reflect the above assumption, this embodiment defines an external annotation data information factor, namely, a second influence factor v. i :

[0092]

[0093] in, It is a relation type r i The total number of corresponding labeled samples, I j b is the feature representation of the j-th sample. i Represents dataset B i The representation vector.

[0094] In an optional embodiment, the above method further includes:

[0095] Step S1002: Construct a sample feature vector based on each of the multiple training samples mentioned above.

[0096] In an optional embodiment, constructing the sample feature vector based on each of the plurality of training samples includes:

[0097] Step S1102: Obtain multiple words contained in the sentence of each training sample, and convert the multiple words into corresponding word vectors;

[0098] Step S1104: Entity merging is performed on the above multiple words, and a first distance value between each word in the above sentence and the first target entity in the above sentence and a second distance value between each word and the second target entity in the above sentence are calculated.

[0099] Step S1106: Obtain a first position information vector from the first random initialization search matrix based on the first distance value and obtain a second position information vector from the second random initialization search matrix based on the second distance value;

[0100] Step S1108: Concatenate the above-mentioned word vectors, the above-mentioned first position information vectors, and the above-mentioned second position information vectors to obtain the word feature vectors corresponding to each word;

[0101] Step S1110: The above sample feature vector is obtained by concatenating the word feature vectors corresponding to each word in the above sentence.

[0102] In this embodiment of the application, the sentence S = {v1, ..., v1} of each training sample is obtained. m The text describes a sequence of words within a given set, using a skip-gram model to train word vectors for each word. i It also merges entities containing multiple words and calculates the distance between each word in the sentence and the two target entities in the sentence.

[0103] In this embodiment, a first distance value d1 is used to represent the distance between each word in the sentence and the first target entity in the sentence, and a second distance value d2 is used to represent the second distance between each word and the second target entity in the sentence. Each word searches for its corresponding first position information vector p in the first random initialization search matrix PF1 and the second random initialization search matrix PF2 according to the first distance value d1 and the second distance value d2, respectively. d1 Second position information vector p d2 .

[0104] For the i-th word in a sentence, in this embodiment, the word feature vector is the concatenation of the word vector and two position vectors (i.e., the first position information vector and the second position information vector). Specifically, the word feature vector can be represented as: w i =concat(v i p d1 p d2 ).

[0105] By extracting word feature vectors from each word in the sentence according to the above formula, this embodiment of the application can concatenate all word feature vectors to obtain the feature matrix C of the sentence. If the length of the sentence is less than m, this embodiment of the application can use zero vectors to pad it, resulting in the feature matrix C = [w1, w2, ..., wm]. m ] T .

[0106] In an optional embodiment, the above method further includes:

[0107] Step S1202: Pre-train the initial network model based on the supervised dataset.

[0108] In an optional embodiment, pre-training the initial network model based on the supervised dataset includes:

[0109] Step S1302: Convert each of the multiple training samples into a corresponding real-valued vector.

[0110] Step S1304: Obtain the attention weight of each training sample based on the multiple real-valued vectors contained in the cluster corresponding to each sample relation category, wherein the attention weight is used to determine the quality of each training sample.

[0111] Step S1306: Calculate the vector representation of the corresponding cluster using the above-mentioned multiple real-valued vectors and the above-mentioned attention weights;

[0112] Step S1308: Set the above vector representation as the input parameter of the loss function and output the second calculation result;

[0113] Step S1310: Pre-train the initial network model based on the second calculation result.

[0114] In the embodiments of this application, sentence-level feature extraction models based on deep neural networks, such as Bi-GRU and PCNN, can be used to transform each training sample into a real-valued vector. For a cluster dataset corresponding to a sample relation category r A computationally selective attention mechanism is used to determine the attention weights for each training sample.

[0115] Then, using the aforementioned multiple real-valued vectors and attention weights, the vector representation α of the corresponding cluster is calculated. j :

[0116]

[0117] Among them, 3) calculate the representation b of the dataset. k It can be based on the features of each sample in the dataset. The attention weights are defined as follows:

[0118]

[0119] The above vector representation is set as the input parameter of the loss function, and the second calculation result is output. Based on the second calculation result, the initial network model is pre-trained to obtain the relation extraction model Loss based on the selection of attention weights.

[0120]

[0121] In this embodiment, a feature vector is constructed for each training sample; a deep learning relation extraction model based on selective attention mechanism is pre-trained on the supervised dataset; training samples of the same relation category in the dataset are merged, and several clusters are obtained on the samples of each relation category using a density clustering algorithm; the clusters are used as the basic training units of the dataset (data packet) level relation extraction model to train the deep learning relation extraction model; new feature vectors of the samples are obtained using the updated deep neural network, and the process jumps back to merge training samples of the same relation category in the dataset, performs iterative training to obtain the target network model, and uses the target network model to classify the unlabeled sentences into relations to obtain new relation pairs.

[0122] Through the embodiments of this application, both multi-instance learning and attention mechanisms are double-edged swords in supervised relation extraction. While they can suppress noisy samples within a packet, they also, to some extent, inhibit the model's learning of a few samples with relatively complex expressions, leading to a decrease in the model's recall rate for some complex samples during prediction. This invention effectively solves the diversity suppression problem, improves the model's ability to learn complex samples, thereby enhancing sentence-level relation extraction performance. Furthermore, by constructing two types of influencing factors to constrain the posterior probability of clustering packets and defining a constrained clustering packet loss function, it integrates manually labeled information while utilizing supervised information.

[0123] This application embodiment also provides a method such as Figure 3 The method for obtaining sample relationship categories is shown. Figure 3 This is a flowchart of a method for obtaining sample relationship categories according to an embodiment of the present invention, such as... Figure 3 As shown, the methods for obtaining the above sample relationship categories include:

[0124] Step S1402: Obtain the sample to be predicted;

[0125] Step S1404: Construct a sample feature vector using the above-mentioned sample to be predicted;

[0126] Step S1406: Input the above sample feature vector into the target network model and output the sample relationship category corresponding to the sample to be predicted. The target network model is obtained by iteratively training the initial network model using a supervised dataset. The supervised dataset includes: multiple training samples. The supervised dataset is used to perform clustering processing according to the sample relationship category of the multiple training samples to obtain multiple clusters. The multiple clusters are used to iteratively train the initial network model to obtain the target network model. The initial network model and the target network model are used to extract supervised relationships.

[0127] In this embodiment of the invention, a sample to be predicted is obtained; a sample feature vector is constructed using the sample to be predicted; the sample feature vector is input into a target network model, and the sample relationship category corresponding to the sample to be predicted is output. The target network model is obtained by iteratively training an initial network model using a supervised dataset. The supervised dataset includes multiple training samples. The supervised dataset is used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters. The multiple clusters are used to iteratively train the initial network model to obtain the target network model. The initial network model and the target network model are used to extract supervised relationships.

[0128] It is noteworthy that the server in this application proposes a new method for constructing a dataset during the training of the target network model. The construction of the supervised dataset is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples of each sample's relation category. The supervised dataset is then clustered to obtain multiple clusters. The initial network model is then iteratively trained using these multiple clusters to achieve better performance and obtain the target network model.

[0129] Therefore, the embodiments of this application achieve the goal of improving the learning ability of the network model for complex samples, thereby realizing the technical effect of improving the performance of extracting supervised relations, and thus solving the technical problem that the existing relation extraction network model has a low learning ability for complex samples and inhibits the learning of diverse samples.

[0130] It should be noted that the above method for obtaining sample relationship categories can essentially be understood as a method for extracting sentence-level supervised relationships based on clustering enhancement. It can be applied, but is not limited to, application scenarios where supervised relationships are extracted based on clustering enhancement.

[0131] For example, it can be applied to medical fields that require the construction of knowledge graphs using information extraction technology, including family doctor knowledge Q&A, breast cancer knowledge graphs, and orthopedic knowledge Q&A.

[0132] As an optional embodiment, this application embodiment can use information extraction technology to construct a general knowledge graph in the medical field, encompassing various types of knowledge in the medical field, including diseases, drugs, and medical service processes, and build an intelligent dialogue system on top of it to provide various Q&A services for primary care physicians and patients.

[0133] This application's embodiments enable network models to learn samples with complex representations while suppressing noisy data. This application proposes a new dataset construction method: clustering augmentation datasets. Unlike previous studies that treated instances containing identical entity pairs as a single dataset (where the number of samples in all datasets is fixed during training), the construction of clustering augmentation datasets is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples from each category together. Each cluster formed by the clustering algorithm on this set is considered a dataset. Simultaneously, deep neural networks or other feature extractors can update their model parameters based on this dataset. These two modules are interdependent and mutually reinforcing, achieving better performance through iterative training.

[0134] This application also provides another method, such as... Figure 4 The method for obtaining sample relationship categories is shown. Figure 4This is a flowchart of another method for obtaining sample relationship categories according to an embodiment of the present invention, such as... Figure 4 As shown, the methods for obtaining the above sample relationship categories include:

[0135] Step S1502: Receive the currently input sample to be predicted;

[0136] Step S1504: Send the above-mentioned sample to be predicted to the server;

[0137] Step S1506: Receive the sample relationship category corresponding to the sample to be predicted from the server. The sample relationship category is obtained by the server using a target network model. The target network model is obtained by iteratively training an initial network model using a supervised dataset. The supervised dataset includes multiple training samples. The supervised dataset is used to cluster the multiple training samples according to their sample relationship categories to obtain multiple clusters. The multiple clusters are used to iteratively train the initial network model to obtain the target network model. The initial network model and the target network model are used to extract supervised relationships.

[0138] Step S1508: Display the above sample relationship categories locally on the client.

[0139] In this embodiment of the invention, a client receives a currently input sample to be predicted; the sample to be predicted is sent to a server; the server receives the sample relationship category corresponding to the sample to be predicted, wherein the sample relationship category is obtained by the server using a target network model, and the target network model is obtained by iteratively training an initial network model using a supervised dataset. The supervised dataset includes: multiple training samples, which are used to cluster according to the sample relationship categories of the multiple training samples to obtain multiple clusters. The multiple clusters are used to iteratively train the initial network model to obtain the target network model. The initial network model and the target network model are used to extract supervised relationships; and the sample relationship category is displayed locally on the client.

[0140] It is noteworthy that the server in this application proposes a new method for constructing a dataset during the training of the target network model. The construction of the supervised dataset is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples of each sample's relation category. The supervised dataset is then clustered to obtain multiple clusters. The initial network model is then iteratively trained using these multiple clusters to achieve better performance and obtain the target network model.

[0141] Therefore, the embodiments of this application achieve the goal of improving the learning ability of the network model for complex samples, thereby realizing the technical effect of improving the performance of extracting supervised relations, and thus solving the technical problem that the existing relation extraction network model has a low learning ability for complex samples and inhibits the learning of diverse samples.

[0142] It should be noted that the above method for obtaining sample relationship categories can essentially be understood as a method for extracting sentence-level supervised relationships based on clustering enhancement. It can be applied, but is not limited to, application scenarios where supervised relationships are extracted based on clustering enhancement.

[0143] For example, it can be applied to medical fields that require the construction of knowledge graphs using information extraction technology, including family doctor knowledge Q&A, breast cancer knowledge graphs, and orthopedic knowledge Q&A.

[0144] As an optional embodiment, this application embodiment can use information extraction technology to construct a general knowledge graph in the medical field, encompassing various types of knowledge in the medical field, including diseases, drugs, and medical service processes, and build an intelligent dialogue system on top of it to provide various Q&A services for primary care physicians and patients.

[0145] This application's embodiments enable network models to learn samples with complex representations while suppressing noisy data. This application proposes a new dataset construction method: clustering augmentation datasets. Unlike previous studies that treated instances containing identical entity pairs as a single dataset (where the number of samples in all datasets is fixed during training), the construction of clustering augmentation datasets is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples from each category together. Each cluster formed by the clustering algorithm on this set is considered a dataset. Simultaneously, deep neural networks or other feature extractors can update their model parameters based on this dataset. These two modules are interdependent and mutually reinforcing, achieving better performance through iterative training.

[0146] This application also provides another method, such as... Figure 5 The method for obtaining sample relationship categories is shown. Figure 5 This is a flowchart of another method for obtaining sample relationship categories according to an embodiment of the present invention, such as... Figure 5 As shown, the methods for obtaining the above sample relationship categories include:

[0147] Step S1602: Receive the sample to be predicted from the client;

[0148] Step S1604: Obtain the sample relationship category corresponding to the sample to be predicted using the target network model. The target network model is obtained by iteratively training the initial network model using a supervised dataset. The supervised dataset includes multiple training samples. The supervised dataset is used to perform clustering processing according to the sample relationship category of the multiple training samples to obtain multiple clusters. The multiple clusters are used to iteratively train the initial network model to obtain the target network model. The initial network model and the target network model are used to extract supervised relationships.

[0149] Step S1606: Return the above sample relationship categories to the client and display the above sample relationship categories locally on the client.

[0150] In this embodiment of the invention, the server receives a sample to be predicted from the client; it uses a target network model to obtain the sample relationship category corresponding to the sample to be predicted, wherein the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset including: multiple training samples, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model being used to extract supervised relationships; the server returns the sample relationship category to the client and displays the sample relationship category locally on the client.

[0151] It is noteworthy that the server in this application proposes a new method for constructing a dataset during the training of the target network model. The construction of the supervised dataset is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples of each sample's relation category. The supervised dataset is then clustered to obtain multiple clusters. The initial network model is then iteratively trained using these multiple clusters to achieve better performance and obtain the target network model.

[0152] Therefore, the embodiments of this application achieve the goal of improving the learning ability of the network model for complex samples, thereby realizing the technical effect of improving the performance of extracting supervised relations, and thus solving the technical problem that the existing relation extraction network model has a low learning ability for complex samples and inhibits the learning of diverse samples.

[0153] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0155] Example 2

[0156] According to an embodiment of this application, an apparatus embodiment for implementing the above-described network model training method is also provided. Figure 6 This is a schematic diagram of the structure of a network model training device according to an embodiment of this application, as shown below. Figure 6 As shown, the device includes: an acquisition module 600, a clustering processing module 602, and a training module 604, wherein:

[0157] The acquisition module 600 acquires a supervised dataset, which includes multiple training samples; the clustering processing module 602 clusters the supervised dataset according to the sample relationship categories of the multiple training samples to obtain multiple clusters; the training module 604 iteratively trains the initial network model using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervision relationships.

[0158] It should be noted that the acquisition module 600, clustering processing module 602, and training module 604 mentioned above correspond to steps S202 to S206 in Embodiment 1. The three modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the device, can run on the computer terminal 10 provided in Embodiment 1.

[0159] According to an embodiment of this application, another apparatus embodiment for implementing the above-described sample relationship category acquisition method is also provided. Figure 7 This is a schematic diagram of a sample relationship category acquisition device according to an embodiment of this application, such as... Figure 7 As shown, the device includes: a first acquisition unit 700, a construction unit 702, and an input unit 704, wherein:

[0160] The first acquisition unit 700 acquires the sample to be predicted; the construction unit 702 constructs a sample feature vector using the sample to be predicted; the input unit 704 inputs the sample feature vector into the target network model and outputs the sample relationship category corresponding to the sample to be predicted. The target network model is obtained by iteratively training an initial network model using a supervised dataset. The supervised dataset includes multiple training samples. The supervised dataset is used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters. The multiple clusters are used to iteratively train the initial network model to obtain the target network model. The initial network model and the target network model are used to extract supervised relationships.

[0161] It should be noted that the first acquisition unit 700, construction unit 702, and input unit 704 mentioned above correspond to steps S1402 to S1406 in Embodiment 1. The three units and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0162] According to an embodiment of this application, another apparatus embodiment for implementing the above-described sample relationship category acquisition method is also provided. Figure 8 This is a schematic diagram of another sample relationship category acquisition device according to an embodiment of this application, such as... Figure 8 As shown, the device includes: a first receiving unit 800, a transmitting unit 802, a second receiving unit 804, and a display unit 806, wherein:

[0163] The first receiving unit 800 is used to receive the currently input sample to be predicted; the sending unit 802 is used to send the sample to be predicted to the server; the second receiving unit 804 is used to receive the sample relationship category corresponding to the sample to be predicted from the server, wherein the sample relationship category is obtained by the server using a target network model, the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset includes: multiple training samples, the supervised dataset is used to perform clustering processing according to the sample relationship category of the multiple training samples to obtain multiple clusters, the multiple clusters are used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model are used to extract supervised relationships; the display unit 806 is used to display the sample relationship category locally on the client.

[0164] It should be noted that the first receiving unit 800, the sending unit 802, the second receiving unit 804, and the display unit 806 mentioned above correspond to steps S1502 to S1508 in Embodiment 1. The four units and the corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0165] According to an embodiment of this application, another apparatus embodiment for implementing the above-described sample relationship category acquisition method is also provided. Figure 9 This is a schematic diagram of another sample relationship category acquisition device according to an embodiment of this application, such as... Figure 9 As shown, the device includes: a third receiving unit 900, a second acquiring unit 902, and a returning unit 904, wherein:

[0166] The third receiving unit 900 is used to receive the sample to be predicted from the client; the second obtaining unit 902 is used to obtain the sample relationship category corresponding to the sample to be predicted using the target network model, wherein the target network model is obtained by iteratively training the initial network model using a supervised dataset, the supervised dataset includes: multiple training samples, the supervised dataset is used to perform clustering processing according to the sample relationship category of the multiple training samples to obtain multiple clusters, the multiple clusters are used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model are used to extract supervised relations; the returning unit 904 is used to return the sample relationship category to the client and display the sample relationship category locally on the client.

[0167] It should be noted that the third receiving unit 900, the second acquiring unit 902, and the returning unit 904 mentioned above correspond to steps S1602 to S1606 in Embodiment 1. The three units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0168] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Method Embodiment 1, and will not be repeated here.

[0169] Example 3

[0170] According to an embodiment of this application, an embodiment of an electronic device is also provided. This electronic device can be any computing device in a group of computing devices. The electronic device includes: a processor and a memory, wherein:

[0171] A processor; and a memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: acquiring a supervised dataset, wherein the supervised dataset includes: multiple training samples; clustering the supervised dataset according to the sample relationship categories of the multiple training samples to obtain multiple clusters; iteratively training an initial network model using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0172] In this embodiment of the invention, a supervised dataset is obtained, which includes multiple training samples; the supervised dataset is clustered according to the sample relationship categories of the multiple training samples to obtain multiple clusters; the initial network model is iteratively trained using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0173] It is noteworthy that this application proposes a new method for constructing a dataset. During the training process, the construction of the supervised dataset is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples of each sample's relation category. The supervised dataset is then clustered to obtain multiple clusters. These multiple clusters are then used to iteratively train the initial network model to obtain better performance and the target network model.

[0174] Therefore, the embodiments of this application achieve the goal of improving the learning ability of the network model for complex samples, thereby realizing the technical effect of improving the performance of extracting supervised relations, and thus solving the technical problem that the existing relation extraction network model has a low learning ability for complex samples and inhibits the learning of diverse samples.

[0175] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0176] Example 4

[0177] According to an embodiment of this application, an embodiment of a computer terminal is also provided. This computer terminal can be any one of a group of computer terminal devices. Optionally, in this embodiment, the aforementioned computer terminal can also be replaced with a mobile terminal or other terminal device.

[0178] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0179] In this embodiment, the computer terminal can execute the program code of the following steps in the video generation method of the application: obtaining a supervised dataset, wherein the supervised dataset includes: multiple training samples; clustering the supervised dataset according to the sample relationship categories of the multiple training samples to obtain multiple clusters; iteratively training an initial network model using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0180] Optionally, Figure 10 This is a structural block diagram of another computer terminal according to an embodiment of this application, such as... Figure 10 As shown, the computer terminal may include: one or more (only one is shown in the figure) processors 1002, memory 1004, and peripheral interfaces 1006.

[0181] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the network model training method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned network model training method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory disposed relative to the processor, which can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0182] The processor can invoke information and applications stored in the memory through the transmission device to perform the following steps: acquiring a supervised dataset, wherein the supervised dataset includes: multiple training samples; clustering the supervised dataset according to the sample relationship categories of the multiple training samples to obtain multiple clusters; iteratively training an initial network model using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0183] Optionally, the processor may also execute program code that performs the following steps: obtain unstructured text; align the unstructured text with the associated knowledge base to obtain the supervised dataset.

[0184] Optionally, the processor may also execute program code that performs the following steps: determining a list of root web pages to be crawled; using a crawler and a depth-first search algorithm to obtain the corresponding raw text from each web page in the root web page list; and performing sentence segmentation on the raw text to obtain the unstructured text.

[0185] Optionally, the processor may also execute program code that performs the following steps: determining whether the sentence of the unstructured text contains at least one entity within the triple of the associated knowledge base; when the sentence contains two entities within the triple, labeling the sentence as a positive sample of the sample relation category; when the sentence contains one entity from the triple and also contains another entity of the same category as the other entity in the triple, labeling the sentence as a negative sample of the sample relation category; and determining the labeled positive and negative samples as the supervised dataset.

[0186] Optionally, the processor may also execute program code that performs the following steps: using a density clustering algorithm to cluster the training samples in the supervised dataset that belong to the same sample relationship category, to obtain the multiple clusters mentioned above.

[0187] Optionally, the processor may also execute program code with the following steps: obtaining a first influence factor and a second influence factor, wherein the first influence factor is used to quantify the influence of each cluster size, and the second influence factor is used to assess the probability that each cluster contains samples that correctly describe the corresponding sample relationship category; calculating a confidence score using a first parameter, a second parameter, a third parameter, the first influence factor, and the second influence factor; obtaining a supervision signal for the confidence score based on the posterior probability corresponding to each cluster; setting the confidence score and the supervision signal as input parameters of the loss function of the confidence posterior constraint, and outputting a first calculation result; and iteratively training the initial network model using the first calculation result to obtain the target network model.

[0188] Optionally, the processor may also execute program code that performs the following steps: obtaining the first influence factor based on the number of training samples contained in each of the plurality of clusters.

[0189] Optionally, the processor may also execute program code that performs the following steps: obtaining the second influence factor based on the total number of labeled samples corresponding to each sample relationship category, the feature representation of each labeled sample, and the vector representation of the cluster corresponding to each sample relationship category.

[0190] Optionally, the processor may also execute program code that performs the following steps: constructing a sample feature vector based on each of the plurality of training samples.

[0191] Optionally, the processor may also execute program code for the following steps: obtaining multiple words contained in the sentence of each training sample and converting the multiple words into corresponding word vectors; merging the multiple words into entities and calculating a first distance value between each word in the sentence and a first target entity in the sentence, and a second distance value between each word and a second target entity in the sentence; obtaining a first position information vector from a first random initialization search matrix based on the first distance value and obtaining a second position information vector from a second random initialization search matrix based on the second distance value; concatenating the word vectors, the first position information vector, and the second position information vector to obtain a word feature vector corresponding to each word; and concatenating the word feature vectors corresponding to each word in the sentence to obtain the sample feature vector.

[0192] Optionally, the processor may also execute program code that performs the following steps: pre-training the initial network model based on the supervised dataset.

[0193] Optionally, the processor may also execute program code that performs the following steps: converting each of the plurality of training samples into a corresponding real-valued vector; obtaining the attention weight of each training sample based on the plurality of real-valued vectors contained in the cluster corresponding to each sample relation category, wherein the attention weight is used to determine the quality of each training sample; calculating the vector representation of the corresponding cluster using the plurality of real-valued vectors and the attention weight; setting the vector representation as the input parameter of the loss function and outputting the second calculation result; and pre-training the initial network model based on the second calculation result.

[0194] The processor can invoke information and application programs stored in the memory through the transmission device to perform the following steps: acquiring a sample to be predicted; constructing a sample feature vector using the sample to be predicted; inputting the sample feature vector into the target network model and outputting the sample relationship category corresponding to the sample to be predicted, wherein the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset including: multiple training samples, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, and the initial network model and the target network model being used to extract supervised relationships.

[0195] The processor can invoke information and applications stored in the memory via a transmission device to perform the following steps: receiving the currently input sample to be predicted; sending the sample to be predicted to the server; receiving the sample relationship category corresponding to the sample to be predicted from the server, wherein the sample relationship category is obtained by the server using a target network model, the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset includes: multiple training samples, the supervised dataset is used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters are used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model are used to extract supervised relationships; and displaying the sample relationship category locally on the client.

[0196] The processor can invoke information and applications stored in the memory via a transmission device to perform the following steps: receiving a sample to be predicted from a client; obtaining the sample relationship category corresponding to the sample to be predicted using a target network model, wherein the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset including: multiple training samples, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model being used to extract supervised relationships; returning the sample relationship category to the client and displaying the sample relationship category locally on the client.

[0197] This invention provides a scheme for training a network model. A supervised dataset is obtained, comprising multiple training samples. The supervised dataset is then clustered according to the sample relationship categories of the multiple training samples to obtain multiple clusters. An initial network model is iteratively trained using these multiple clusters to obtain a target network model. The initial network model and the target network model are used to extract supervised relationships.

[0198] It is noteworthy that this application proposes a new method for constructing a dataset. During the training process, the construction of the supervised dataset is dynamic. Deep neural networks or other feature extractors extract features from each sample and then mix all samples of each sample's relation category. The supervised dataset is then clustered to obtain multiple clusters. These multiple clusters are then used to iteratively train the initial network model to obtain better performance and the target network model.

[0199] Therefore, the embodiments of this application achieve the goal of improving the learning ability of the network model for complex samples, thereby realizing the technical effect of improving the performance of extracting supervised relations, and thus solving the technical problem that the existing relation extraction network model has a low learning ability for complex samples and inhibits the learning of diverse samples.

[0200] Those skilled in the art will understand that Figure 10 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 10 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include components that are more advanced than those described above. Figure 10 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 10 The different configurations shown.

[0201] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable non-volatile storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0202] Example 5

[0203] According to an embodiment of this application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the aforementioned non-volatile storage medium can be used to store the program code executed by the network model training method and the sample relationship category acquisition method provided in Embodiment 1 above.

[0204] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0205] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining a supervised dataset, wherein the supervised dataset includes: multiple training samples; clustering the supervised dataset according to the sample relationship categories of the multiple training samples to obtain multiple clusters; iteratively training an initial network model using the multiple clusters to obtain a target network model, wherein the initial network model and the target network model are used to extract supervised relationships.

[0206] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining unstructured text; aligning the unstructured text with the associated knowledge base to obtain the supervised dataset.

[0207] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining a list of root web pages to be crawled; using a crawler tool and a depth-first search algorithm to obtain the corresponding original text from each web page in the root web page list; and performing sentence segmentation on the original text to obtain the unstructured text.

[0208] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining whether the sentence of the unstructured text contains at least one entity within the triple of the associated knowledge base; when the sentence contains two entities within the triple, labeling the sentence as a positive sample of the sample relation category; when the sentence contains one entity from the triple and also contains another entity of the same category as the other entity in the triple, labeling the sentence as a negative sample of the sample relation category; and determining the labeled positive and negative samples as the supervised dataset.

[0209] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: using a density clustering algorithm to cluster the training samples in the above-mentioned supervised dataset that belong to the same sample relationship category, to obtain the above-mentioned multiple clusters.

[0210] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining a first influence factor and a second influence factor, wherein the first influence factor is used to quantify the influence of each cluster size, and the second influence factor is used to assess the probability that each cluster contains samples that correctly describe the corresponding sample relationship category; calculating a confidence score using a first parameter, a second parameter, a third parameter, the first influence factor, and the second influence factor; obtaining a supervision signal for the confidence score based on the posterior probability corresponding to each cluster; setting the confidence score and the supervision signal as input parameters of a loss function for confidence posterior constraints, and outputting a first calculation result; and iteratively training the initial network model using the first calculation result to obtain the target network model.

[0211] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining the first influence factor based on the number of training samples contained in each of the plurality of clusters.

[0212] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining the second influence factor based on the total number of labeled samples corresponding to each sample relationship category, the feature representation of each labeled sample, and the vector representation of the cluster corresponding to each sample relationship category.

[0213] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: constructing a sample feature vector based on each of the plurality of training samples.

[0214] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining multiple words contained in the sentence of each training sample and converting the multiple words into corresponding word vectors; merging the multiple words into entities and calculating a first distance value between each word in the sentence and a first target entity in the sentence, and a second distance value between each word and a second target entity in the sentence; obtaining a first position information vector from a first random initialization search matrix based on the first distance value and obtaining a second position information vector from a second random initialization search matrix based on the second distance value; concatenating the word vectors, the first position information vector, and the second position information vector to obtain a word feature vector corresponding to each word; and concatenating the word feature vectors corresponding to each word in the sentence to obtain the sample feature vector.

[0215] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: pre-training the initial network model based on the supervised dataset.

[0216] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: converting each of the plurality of training samples into a corresponding real-valued vector; obtaining the attention weight of each training sample based on the plurality of real-valued vectors contained in the cluster corresponding to each sample relation category, wherein the attention weight is used to determine the quality of each training sample; calculating the vector representation of the corresponding cluster using the plurality of real-valued vectors and the attention weight; setting the vector representation as the input parameter of the loss function and outputting a second calculation result; and pre-training the initial network model based on the second calculation result.

[0217] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining a sample to be predicted; constructing a sample feature vector using the sample to be predicted; inputting the sample feature vector into a target network model and outputting the sample relationship category corresponding to the sample to be predicted, wherein the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset including: multiple samples to be trained, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple samples to be trained to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, and the initial network model and the target network model being used to extract supervised relationships.

[0218] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: receiving the currently input sample to be predicted; sending the sample to be predicted to the server; receiving the sample relationship category corresponding to the sample to be predicted from the server, wherein the sample relationship category is obtained by the server using a target network model, the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset includes: multiple training samples, the supervised dataset is used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters are used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model are used to extract supervised relationships; and displaying the sample relationship category locally on the client.

[0219] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: receiving a sample to be predicted from a client; obtaining the sample relationship category corresponding to the sample to be predicted using a target network model, wherein the target network model is obtained by iteratively training an initial network model using a supervised dataset, the supervised dataset including: multiple training samples, the supervised dataset being used to perform clustering processing according to the sample relationship categories of the multiple training samples to obtain multiple clusters, the multiple clusters being used to iteratively train the initial network model to obtain the target network model, the initial network model and the target network model being used to extract supervised relationships; returning the sample relationship category to the client and displaying the sample relationship category locally on the client.

[0220] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0221] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0222] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0223] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0224] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0225] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned non-volatile storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0226] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A network model training method, characterized in that, include: Retrieve unstructured text; Determine whether the sentence in the unstructured text contains at least one entity within a triple of the associated knowledge base; when the sentence contains two entities within the triple, label the sentence as a positive sample of the sample relation category; when the sentence contains one entity from the triple and also contains another entity of the same category as the other entity in the triple, label the sentence as a negative sample of the sample relation category. The labeled positive samples and labeled negative samples are defined as the supervised dataset, wherein the supervised dataset includes: multiple training samples; The initial network model is pre-trained based on the supervised dataset and the loss function, wherein the loss function is determined based on the real-valued vector corresponding to each training sample, the vector representation of the cluster corresponding to each sample relation category, and each sample relation category; The supervised dataset is clustered according to the sample relationship categories of the multiple training samples to obtain multiple clusters; The pre-trained initial network model is iteratively trained using the multiple clusters to obtain the target network model, wherein the pre-trained initial network model and the target network model are used to extract supervision relationships.

2. The network model training method according to claim 1, characterized in that, Obtaining the unstructured text includes: Determine the list of root web pages to be crawled; The original text corresponding to each webpage in the root webpage list is obtained using web crawling tools and a depth-first search algorithm; The original text is segmented into sentences to obtain the unstructured text.

3. The network model training method according to claim 1, characterized in that, Based on the sample relationship categories of the multiple training samples, the supervised dataset is clustered to obtain the multiple clusters, including: Density clustering algorithm is used to cluster the training samples in the supervised dataset that belong to the same sample relationship category to obtain the multiple clusters.

4. The network model training method according to claim 1, characterized in that, The target network model is obtained by iteratively training the pre-trained initial network model using the multiple clusters, including: Obtain a first impact factor and a second impact factor, wherein the first impact factor is used to quantify the impact of each cluster size, and the second impact factor is used to assess the probability that each cluster contains samples that correctly describe the corresponding sample relationship category; The credibility score is calculated using the first parameter, the second parameter, the third parameter, the first impact factor, and the second impact factor. The supervision signal for the confidence score is obtained based on the posterior probability corresponding to each cluster; Set the confidence score and the supervision signal as input parameters of the loss function of the confidence posterior constraint, and output the first calculation result; The target network model is obtained by iteratively training the pre-trained initial network model using the first calculation result.

5. The network model training method according to claim 4, characterized in that, Obtaining the first impact factor includes: The first influence factor is obtained based on the number of training samples contained in each of the plurality of clusters.

6. The network model training method according to claim 4, characterized in that, Obtaining the second impact factor includes: The second influence factor is obtained based on the total number of labeled samples corresponding to each sample relationship category, the feature representation of each labeled sample, and the vector representation of the cluster corresponding to each sample relationship category.

7. The network model training method according to claim 1, characterized in that, The method further includes: Construct a sample feature vector based on each of the plurality of training samples.

8. The network model training method according to claim 7, characterized in that, Constructing the sample feature vector based on each of the plurality of training samples includes: Obtain multiple words contained in the sentence of each training sample, and convert the multiple words into corresponding word vectors; Entity merging is performed on the multiple words, and a first distance value between each word in the sentence and a first target entity in the sentence, and a second distance value between each word and a second target entity in the sentence are calculated; Based on the first distance value, a first position information vector is obtained from a first random initialization search matrix, and based on the second distance value, a second position information vector is obtained from a second random initialization search matrix; The word feature vector corresponding to each word is obtained by concatenating the word vector, the first position information vector, and the second position information vector. The sample feature vector is obtained by concatenating the word feature vectors corresponding to each word in the sentence.

9. The network model training method according to claim 7, characterized in that, The method further includes: The initial network model is pre-trained based on the supervised dataset.

10. The network model training method according to claim 9, characterized in that, Pre-training the initial network model based on the supervised dataset includes: Each of the plurality of training samples is transformed into a corresponding real-valued vector; The attention weights of each training sample are obtained based on the multiple real-valued vectors contained in the cluster corresponding to each sample relation category, wherein the attention weights are used to determine the quality of each training sample. The vector representation of the corresponding cluster is calculated using the multiple real-valued vectors and the attention weights; The vector representation is set as the input parameter of the loss function, and the second calculation result is output. The initial network model is pre-trained based on the second calculation result.

11. A method for obtaining sample relationship categories, characterized in that, include: Obtain the sample to be predicted; Construct a sample feature vector using the sample to be predicted; The sample feature vector is input into the target network model, and the sample relation category corresponding to the sample to be predicted is output. The target network model is obtained by iteratively training the initial network model using a supervised dataset. The supervised dataset is determined based on labeled positive and negative samples. When a sentence contains two entities in a triple, the sentence is labeled as a positive sample of the sample relation category. When the sentence contains one entity in the triple and also contains another entity of the same category as the other entity in the triple, the sentence is labeled as a negative sample of the sample relation category. The supervised dataset includes multiple training samples. The initial network model is pre-trained based on the supervised dataset and a loss function. The loss function is determined based on the real-valued vector corresponding to each training sample, the vector representation of the cluster corresponding to each sample relation category, and each sample relation category. The supervised dataset is used to perform clustering processing according to the sample relation categories of the multiple training samples to obtain multiple clusters. The multiple clusters are used to iteratively train the pre-trained initial network model to obtain the target network model. The pre-trained initial network model and the target network model are used to extract supervised relations.

12. A method for obtaining sample relationship categories, characterized in that, include: Receive the currently input sample to be predicted; Send the sample to be predicted to the server; The system receives the sample relation category corresponding to the sample to be predicted from the server. The sample relation category is obtained by the server using a target network model, which is obtained by iteratively training an initial network model using a supervised dataset. The supervised dataset is determined based on labeled positive and negative samples. When a sentence contains two entities within a triplet, the sentence is labeled as a positive sample of the sample relation category. When a sentence contains one entity from the triplet and also contains another entity of the same category as the other entity in the triplet, the sentence is labeled as a negative sample of the sample relation category. The supervised dataset includes multiple training samples, and the initial network model is pre-trained based on the supervised dataset and a loss function. The loss function is determined based on the real-valued vector corresponding to each training sample, the vector representation of the cluster corresponding to each sample relation category, and each sample relation category. The supervised dataset is used to perform clustering processing according to the sample relation categories of the multiple training samples to obtain multiple clusters. The multiple clusters are used to iteratively train the pre-trained initial network model to obtain the target network model. The pre-trained initial network model and the target network model are used to extract supervised relations. The sample relationship category is displayed locally on the client side.

13. A method for obtaining sample relationship categories, characterized in that, include: Receive the sample to be predicted from the client; The target network model is used to obtain the sample relation category corresponding to the sample to be predicted. The target network model is obtained by iteratively training an initial network model using a supervised dataset. The supervised dataset is determined based on labeled positive and negative samples. When a sentence contains two entities within a triplet, the sentence is labeled as a positive sample of the sample relation category. When a sentence contains one entity from the triplet and also contains another entity of the same category as the other entity in the triplet, the sentence is labeled as a negative sample of the sample relation category. The supervised dataset includes multiple training samples. The initial network model is pre-trained based on the supervised dataset and a loss function. The loss function is determined based on the real-valued vector corresponding to each training sample, the vector representation of the cluster corresponding to each sample relation category, and each sample relation category. The supervised dataset is used to cluster the multiple training samples according to their sample relation categories to obtain multiple clusters. These multiple clusters are used to iteratively train the pre-trained initial network model to obtain the target network model. The pre-trained initial network model and the target network model are used to extract supervised relations. Return the sample relationship category to the client and display the sample relationship category locally on the client.

14. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, the device where the non-volatile storage medium is located is controlled to perform the network model training method according to any one of claims 1 to 10, and the sample relationship category acquisition method according to any one of claims 11 to 13.

15. An electronic device, characterized in that, include: processor; as well as A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Retrieve unstructured text; Determine whether the sentence in the unstructured text contains at least one entity within a triple of the associated knowledge base; when the sentence contains two entities within the triple, label the sentence as a positive sample of the sample relation category; when the sentence contains one entity from the triple and also contains another entity of the same category as the other entity in the triple, label the sentence as a negative sample of the sample relation category. The labeled positive samples and labeled negative samples are defined as the supervised dataset, wherein the supervised dataset includes: multiple training samples; The initial network model is pre-trained based on the supervised dataset and the loss function, wherein the loss function is determined based on the real-valued vector corresponding to each training sample, the vector representation of the cluster corresponding to each sample relation category, and each sample relation category; The supervised dataset is clustered according to the sample relationship categories of the multiple training samples to obtain multiple clusters; The pre-trained initial network model is iteratively trained using the multiple clusters to obtain the target network model, wherein the pre-trained initial network model and the target network model are used to extract supervision relationships.

Images