Data processing method and device, equipment and storage medium
By encoding and clustering structured data, the accuracy of data representation and clustering results are improved by using pre-trained language models. This solves the problem of incomplete querying in structured data and enables the acquisition of comprehensive and accurate target data from standardized datasets.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIBABA (CHINA) CO LTD
- Filing Date
- 2022-05-16
- Publication Date
- 2026-07-14
AI Technical Summary
It is difficult to find comprehensive and accurate target data in structured data.
A pre-trained language model is used to encode text information containing structured data, obtaining context representation vectors for multiple components in the structured data in the text information. Then, clustering is performed based on the context representation vectors of components of the same type, and the first data set is adjusted to obtain a standardized second data set.
This improves the accuracy of the context representation vectors of the constituent elements, enhances the accuracy of the clustering results, and enables the querying of comprehensive and accurate target data from standardized datasets.
Smart Images
Figure CN115145952B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of information technology, and in particular to a data processing method, apparatus, device and storage medium. Background Technology
[0002] With the continuous development of technology, data querying, such as retrieving target data from structured data, has become an increasingly common technology.
[0003] However, the inventors of this application have discovered that in some cases, it is difficult to retrieve comprehensive and accurate target data from structured data. Summary of the Invention
[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a data processing method, apparatus, device, and storage medium to retrieve comprehensive and accurate target data from a standardized second dataset.
[0005] In a first aspect, embodiments of this disclosure provide a data processing method, including:
[0006] A pre-trained language model is used to encode text information containing structured data to obtain context representation vectors corresponding to multiple components of the structured data in the text information.
[0007] Based on the context representation vectors corresponding to multiple components of the same type in the multiple structured data included in the first data set, clustering is performed on the multiple components of the same type to obtain clustering results;
[0008] The first data set is adjusted based on the clustering results to obtain the second data set.
[0009] In a second aspect, embodiments of this disclosure provide a data processing apparatus, comprising:
[0010] The encoding module is used to encode text information containing structured data using a pre-trained language model to obtain context representation vectors corresponding to multiple components in the structured data in the text information.
[0011] The clustering processing module is used to perform clustering processing on the multiple constituent elements of the same type in the multiple structured data included in the first data set according to the context representation vectors corresponding to the multiple constituent elements of the same type, and to obtain the clustering result;
[0012] An adjustment module is used to adjust the first data set according to the clustering results to obtain a second data set.
[0013] Thirdly, embodiments of this disclosure provide an electronic device, including:
[0014] Memory;
[0015] Processor; and
[0016] Computer programs;
[0017] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.
[0018] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described in the first aspect.
[0019] The data processing method, apparatus, device, and storage medium provided in this disclosure encode text information containing structured data using a pre-trained language model, obtaining context representation vectors for each component element in the structured data within the text information. Because the pre-trained language model is pre-trained on a large amount of data, it can determine the meaning of each component element with finer granularity, avoiding ambiguity and improving the accuracy of the context representation vectors for each component element. Furthermore, by clustering multiple components of the same type based on their highly accurate context representation vectors, the accuracy of the clustering results can be improved. Based on these clustering results, a more accurate standardization process can be performed on the first data set to obtain a standardized second data set. Thus, comprehensive and accurate target data can be queried from this standardized second data set. Attached Figure Description
[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0022] Figure 1 A flowchart of a data processing method provided in this embodiment of the disclosure;
[0023] Figure 2 A schematic diagram illustrating an application scenario provided by an embodiment of this disclosure;
[0024] Figure 3A flowchart of a data processing method provided in another embodiment of this disclosure;
[0025] Figure 4 A flowchart of a data processing method provided in another embodiment of this disclosure;
[0026] Figure 5 A schematic diagram illustrating the comparison results of the degree of standardization provided for another embodiment of this disclosure;
[0027] Figure 6 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this disclosure;
[0028] Figure 7 A schematic diagram of the structure of an electronic device embodiment provided in this disclosure. Detailed Implementation
[0029] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0030] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0031] Currently, data querying, such as retrieving target data from structured data, has become an increasingly common technique. However, in some cases, it is difficult to retrieve comprehensive and accurate target data from structured data. To address this problem, this disclosure provides a data processing method, which will be described below with reference to specific embodiments.
[0032] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of this disclosure. The method can be executed by a data processing device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a server or terminal, where the terminal specifically includes a mobile phone, computer, or tablet computer. Specifically, the method can be applied to applications such as... Figure 2In the application scenario shown, there are a terminal 21 and a server 22. The server 22 may have a pre-built open knowledge graph. The server 22 can use the data processing method described in this embodiment to perform canonicalization on the open knowledge graph, thereby obtaining a standardized knowledge graph. Further, based on the query request sent by the terminal 21, the server 22 queries the standardized knowledge graph to obtain target data and feeds the target data back to the terminal 21. Specifically, the process of the terminal 21 sending a query request to the server 22 and the server 22 feeding back the target data to the terminal 21 can be a smart question-and-answer process. Alternatively, the terminal 21 may have a pre-built open knowledge graph. The terminal 21 can use the data processing method described in this embodiment to perform canonicalization on the open knowledge graph, thereby obtaining a standardized knowledge graph. Further, the terminal 21 can query the standardized knowledge graph to obtain target data based on the user's query request and feed the target data back to the user. The following uses... Figure 2 Taking server 22 as an example to illustrate this data processing method, we will introduce it. Figure 1 As shown, the specific steps of this method are as follows:
[0033] S101. A pre-trained language model is used to encode text information containing structured data to obtain context representation vectors corresponding to multiple components in the structured data in the text information.
[0034] In this embodiment, the Open Knowledge Graph (OpenKG) includes multiple structured data sets, each containing multiple constituent elements. For example, the Open Information Extraction (OpenIE) tool is used to mine triples from unstructured text. Specifically, these triples can be subject-verb-object triples, which are then used to construct the Open Knowledge Graph. In other words, each structured data set in the Open Knowledge Graph can be a triple consisting of a subject, a verb, and an object; that is, the triple includes a phrase in the form of a subject, a phrase in the form of a verb, and a phrase in the form of an object. For example, the unstructured text is "Xiaoming and Xiaohong are classmates, both from Province A." The OpenIE tool can mine three triples from this unstructured text: (Xiaoming, classmate, Xiaohong), (Xiaoming, from, Province A), and (Xiaohong, from, Province A). These triples can then form the Open Knowledge Graph.
[0035] In this embodiment, server 22 may deploy a pre-trained language model, which may specifically be a deep bidirectional language representation model based on Tranformer (Bidirectional Encoder Representation from Transformers, BERT), Roberta, SpanBert, or a continuous pre-training framework for language understanding (ERNIE2.0), etc. Tranformer is a network structure based on self-attention. RoBERTa is a robust improvement to BERT.
[0036] Specifically, server 22 can use the pre-trained language model to encode text information containing structured data, thereby obtaining context representation vectors corresponding to multiple components in the structured data in the text information.
[0037] In one possible implementation, the text information is unstructured text; before encoding the text information containing structured data using a pre-trained language model, the method further includes: obtaining the structured data from the unstructured text.
[0038] For example, structured data is a triple (Xiaoming, classmate, Xiaohong), and the text information containing this structured data can be the unstructured text used when extracting the triple, such as "Xiaoming and Xiaohong are classmates, both from Province A". In some other embodiments, the unstructured text can also be denoted as the original sentence providing the triple. Further, the server 22 can use the pre-trained language model to encode the original sentence, thereby obtaining the contextual word embedding corresponding to each component of the triple in the original sentence. In some embodiments, the contextual word embedding can also be denoted as a hidden representation. Here, "Xiaoming", "classmate", and "Xiaohong" are the components of the triple, that is, the triple includes 3 components. For example, the contextual word embedding corresponding to "Xiaoming" is the representation vector of "Xiaoming" obtained based on the context of "Xiaoming" in the original sentence. That is, when the same phrase, word, or phrase is in different sentences, the contextual word embedding of the phrase, word, or phrase may be different. For example, the word "apple" may refer to a fruit or a company name. If the context of "apple" includes other fruits such as "banana" or "pear," the context representation vector of "apple" is the same as the representation vector of "apple" as a fruit. If the context of "apple" includes device names such as "phone" or "tablet," the context representation vector of "apple" is the same as the representation vector of "apple" as a company name. The representation vector of "apple" as a fruit and the representation vector of "apple" as a company name are different, thus effectively avoiding ambiguity caused by the same word.
[0039] It is understandable that the triplet can be obtained from the original sentence before the encoding is performed. The method for obtaining the triplet is as described above and will not be repeated here.
[0040] In another possible implementation, the text information is composed of multiple constituent elements from the structured data.
[0041] For example, in some other embodiments, the text information input to the pre-trained language model is text information composed of three components from a triple (Xiaoming, classmate, Xiaohong). Furthermore, the pre-trained language model can encode this text information to output the context representation vectors corresponding to each of the three components within the text information.
[0042] In another possible implementation, the text information is obtained by concatenating multiple components of the structured data and adding preset characters at the concatenation positions.
[0043] For example, in some other embodiments, the text information input to the pre-trained language model is obtained by concatenating the three components of a triple (Xiaoming, classmate, Xiaohong) and adding a preset character, such as [sep], at the concatenation position. Furthermore, the pre-trained language model can encode this text information to output the context representation vectors corresponding to the three components in the text information.
[0044] In other embodiments, each component element of the triple can be input into a pre-trained language model to obtain a representation vector for that component element.
[0045] S102. Based on the context representation vectors corresponding to multiple components of the same type in the multiple structured data included in the first data set, clustering is performed on the multiple components of the same type to obtain the clustering result.
[0046] Optionally, the multiple components of the structured data include a subject, a predicate, and an object; the subject is a phenotype corresponding to an entity, or the subject is an entity; the object is a phenotype corresponding to an entity, or the object is an entity; the predicate is the relationship between the subject and the object. This relationship can be a relationship between entities, i.e., an entity-relation.
[0047] For example, the open knowledge graph described above can be denoted as a first data set. The first data set can include multiple triples. Assuming the first data set includes 300 triples, since each triple includes three constituent elements—subject, predicate, and object—the first data set includes 300 subjects, 300 predicates, and 300 objects. The 300 subjects can be multiple constituent elements of the same type, the 300 predicates can be multiple constituent elements of the same type, and the 300 objects can be multiple constituent elements of the same type. Alternatively, in other embodiments, if the 300 subjects and the 300 objects are noun phrases, then they can be multiple constituent elements of the same type.
[0048] Specifically, server 22 can cluster multiple constituent elements of the same type based on their respective context representation vectors to obtain clustering results. For example, server 22 can cluster the 300 subjects based on their respective context representation vectors. Specifically, server 22 can use Hierarchical Agglomerative Clustering (HAC) to cluster multiple constituent elements of the same type. HAC is an unsupervised hierarchical clustering algorithm, i.e., a bottom-up hierarchical clustering method.
[0049] For example, if the 300 subjects, or some of the subjects, are phenotypes corresponding to entities, then when clustering these 300 subjects, it is possible to specifically identify at least two phenotypes corresponding to the same entity among these 300 subjects. For instance, Xiaoming is student number 03 in the class, and Xiaoming's common name is Zhang XX. If "student number 03" is considered an entity, i.e., a unique individual, then "student number 03" can have phenotypes "Xiaoming" and "Zhang XX". Therefore, when the 300 subjects include both "Xiaoming" and "Zhang XX", it can be determined that "Xiaoming" and "Zhang XX" correspond to the same entity. This achieves entity-level standardization of subjects.
[0050] For example, if the 300 subjects, or some of the subjects, are entities, then when clustering these 300 subjects, the commonalities between at least two subjects can be determined. For instance, Class 2 of Grade 3 includes 56 students, whose student IDs are numbered 01 to 56. Therefore, when the 300 subjects include "Student 03" and "Student 02," it can be determined that the commonality between "Student 03" and "Student 02" is that they are students of Class 2 of Grade 3. This achieves ontology-level standardization of subjects. In this embodiment, ontology refers to the category relationship between entities and concepts. For example, the entity is "Student 02," and the concept is "student of Class 2 of Grade 3," or the entity is "lawyer," and the concept is "profession." That is, there is an ontology between entities, and this ontology can be the commonalities between two entities.
[0051] It is understood that in some embodiments, the object may also be the phenotype corresponding to the entity, or the object may also be an entity. In this case, the process of clustering the 300 objects can be similar to the process of clustering the 300 subjects, and the specific process will not be elaborated here. That is to say, the objects can also be normalized at the entity level and / or at the ontology level. In addition, it is understood that when the 300 subjects and the 300 objects are all noun phrases, the 300 subjects and the 300 objects can also be clustered as multiple components of the same type. Alternatively, the phenotypes corresponding to the entities in the 300 subjects and the phenotypes corresponding to the entities in the 300 objects can be clustered as multiple components of the same type, and the entities in the 300 subjects and the entities in the 300 objects can be clustered as multiple components of the same type. That is to say, this embodiment does not specifically limit the type division referred to by the same type.
[0052] Additionally, in some embodiments, the predicate can be a relationship between a subject and an object. For example, (Xiaoming, gnaw, apple) is a triple, and (Xiaohong, swallow, dumpling) is another triple, where "gnaw" is the relationship between "Xiaoming" and "apple," and "swallow" is the relationship between "Xiaohong" and "dumpling." When clustering the 300 predicates as described above, it is possible to determine at least two identical relationships among these 300 predicates. For example, "gnaw" and "swallow" can be normalized to "eat," therefore, "gnaw" and "swallow" are identical relationships. The process of normalizing "gnaw" and "swallow" to "eat" achieves relational level normalization of the predicates.
[0053] S103. Adjust the first data set according to the clustering results to obtain the second data set.
[0054] Based on the clustering process described above, the resulting clustering can include at least one of the following: at least two phenotypes of the same entity, commonalities between at least two entities, and at least two identical relationships. Assuming the first data set includes four triples: (Xiaoming, classmate, Xiaohong), (Zhang XX, nibble, apple), (Student 03, Chinese exam, 90 points), and (Student 02, swallow, dumpling), then based on this clustering result, the first data set can be adjusted so that "Xiaoming" and "Zhang XX" point to the same entity "Student 03," and the commonality between "Student 03" and "Student 02" is added to the first data set: they are students in Class 2, Grade 3. Additionally, "nibble" and "swallow" in the first data set can be standardized to "eat." It is understood that the adjustment of the first data set is not limited to this; this is merely illustrative and not specifically limited. In this embodiment, the adjusted first data set is denoted as the second data set, which can be the standardized knowledge graph described above. Server 22 can provide more comprehensive and accurate data query services based on the standardized knowledge graph. For example, when querying "Xiaoming", since the standardized knowledge graph shows that "Xiaoming" and "Zhang XX" point to the same entity "Student 03", and "Student 03" and "Student 02" share the commonality of being students in Class 2, Grade 3, and "gnaw" and "swallow" are standardized to "eat", the query result could be "Xiaoming and Xiaohong are students in Class 2, Grade 3, Xiaoming eats an apple, and Xiaoming scored 90 points on his Chinese exam". If the first dataset is not standardized, when querying "Xiaoming", since the first dataset does not know that "Xiaoming" and "Zhang XX" point to the same entity "Student 03", the query result obtained from the first dataset would only be "Xiaoming and Xiaohong are classmates". Therefore, querying data based on the standardized knowledge graph can provide more comprehensive and accurate query results. Furthermore, if "Xiaoming" and "Zhang XX" share the same character "Xiaoming", then when querying "Xiaoming" in the first data set, two triples can be found in that first data set: (Xiaoming, classmate, Xiaohong) and (Zhang XX, gnaw, apple). In this case, the query result obtained from the first data set is "Xiaoming and Xiaohong are classmates, Zhang XX gnaws on an apple". "Xiaoming" and "Zhang XX" are actually redundant content. It can be seen that when querying data based on this standardized knowledge graph, redundant content in the query results can be avoided, and the query results after removing redundant content are clearer and more accurate.
[0055] This embodiment employs a pre-trained language model to encode text information containing structured data, obtaining context representation vectors for each component element within the structured data. Because the pre-trained language model is trained on a large dataset, it can determine the meaning of each component element with finer granularity, avoiding ambiguity and improving the accuracy of the context representation vectors for each component element. Furthermore, by clustering multiple components of the same type based on their highly accurate context representation vectors, the accuracy of the clustering results can be improved. Based on these clustering results, the first dataset can be standardized more accurately to obtain a standardized second dataset. Thus, comprehensive and accurate target data can be retrieved from this standardized second dataset.
[0056] Figure 3 This is a flowchart of a data processing method provided in another embodiment of this disclosure. It is understood that the process of encoding text information containing structured data using a pre-trained language model can be either a usage phase or an inference phase of the pre-trained language model. Before the usage or inference phase, the pre-trained language model can be pre-trained, and this pre-training process can be performed by… Figure 4 The pre-training module shown is used to perform this process, which can be as described in S301-S303 below. For example... Figure 3 As shown, the specific steps of this method are as follows:
[0057] S301. Delete any component element in the structured data from the text information to obtain the remaining text information.
[0058] like Figure 4 The pre-training module shown can also be referred to as a triple pre-training module. For example... Figure 4 As shown, text information is involved in both the encoding and pre-training processes. However, the text information involved in the encoding and pre-training processes can be the same or different. In other words, the text information used by the pre-trained language model in the inference and pre-training stages can be the same or different.
[0059] For example, the text information involved in the pre-training process is the original sentence as described above, which includes a subject, predicate, and object. Further, the subject, predicate, or object in the original sentence is randomly deleted or masked; that is, one of the subject, predicate, or object is deleted or masked with a probability of 1 / 3. Suppose the subject is deleted from the original sentence, resulting in the remaining text information.
[0060] S302. Input the remaining text information into the pre-trained language model, which is used to predict any of the deleted components based on the remaining text information.
[0061] For example, pre-trained language models can include Figure 4 The pre-training module shown further incorporates the remaining text information as described above, allowing it to predict the deleted subject based on this information. Alternatively, the pre-training module can predict the deleted subject based on the remaining predicates and objects.
[0062] S303. Train the pre-trained language model based on any deleted component predicted by the pre-trained language model and any actually deleted component.
[0063] Understandably, since the subject predicted by the pre-training module may not be the actual deleted subject, it can use both the predicted and deleted subjects to train the pre-trained language model, thus updating its parameters. Further, by selecting other textual information and using a similar approach to iteratively update the parameters of the pre-trained language model multiple times, a process similar to link prediction in knowledge graph representation learning.
[0064] Understandably, when the parameters of the pre-trained language model gradually converge, or when the number of iterations reaches a preset number, pre-training of the pre-trained language model is stopped, i.e., pre-training is complete. After pre-training is complete, the pre-trained language model can be used to encode text information containing structured data.
[0065] S304. A pre-trained language model is used to encode text information containing structured data to obtain context representation vectors corresponding to multiple components in the structured data in the text information.
[0066] Optionally, a pre-trained language model is used to encode text information containing structured data to obtain context representation vectors corresponding to multiple components of the structured data in the text information. This includes: encoding text information containing structured data using a pre-trained language model to obtain multiple representation vectors corresponding to any component of the structured data in the text information; and determining the context representation vector corresponding to any component in the text information based on the multiple representation vectors corresponding to any component in the text information.
[0067] In some cases, the subject, predicate, or object in the original sentence may be quite long. Taking the subject as an example, suppose the subject of the original sentence is "student number 03". Because "student number 03" is long, when the pre-trained language model encodes the original sentence, it may output the context representation vector of "03" and the context representation vector of "student". The context representation vectors of "03" and "student" can be denoted as multiple representation vectors corresponding to "student number 03". Furthermore, the context representation vector of "student number 03" can be calculated based on the context representation vectors of "03" and "student". This calculation method includes span representation methods such as mean pooling, average pooling, and diffsum.
[0068] S305. Based on the context representation vectors corresponding to multiple components of the same type in the multiple structured data included in the first data set, clustering is performed on the multiple components of the same type to obtain clustering results.
[0069] Optionally, clustering is performed on the multiple constituent elements of the same type, including at least one of the following: if the multiple constituent elements of the same type are multiple phenotypes corresponding to at least one entity, then at least two phenotypes corresponding to the same entity are determined; if the multiple constituent elements of the same type are multiple entities, then commonalities between at least two entities are determined; if the multiple constituent elements of the same type are multiple relations, then at least two identical relations are determined. This relation can be a relationship between entities, i.e., an entity-relation.
[0070] For example, the first data set includes 300 subject-verb-object triples, therefore, the first data set includes 300 subjects, 300 verbs, and 300 objects.
[0071] If the 300 subjects, or some of the subjects, are phenotypes corresponding to entities, then clustering these 300 subjects can specifically identify at least two phenotypes corresponding to the same entity among them. For example, Xiaoming is student number 03 in the class, and Xiaoming's common name is Zhang XX. If "student number 03" is considered an entity, i.e., a unique individual, then "student number 03" can have phenotypes "Xiaoming" and "Zhang XX". Therefore, when the 300 subjects include both "Xiaoming" and "Zhang XX", it can be determined that "Xiaoming" and "Zhang XX" correspond to the same entity. This achieves entity-level standardization of the subjects.
[0072] For example, if the 300 subjects, or some of the subjects, are entities, then when clustering these 300 subjects, the commonalities between at least two subjects can be determined. For instance, Class 2 of Grade 3 includes 56 students, whose student IDs are numbered 01 to 56. Therefore, when the 300 subjects include "Student 03" and "Student 02," it can be determined that the commonality between "Student 03" and "Student 02" is that they are students of Class 2 of Grade 3. This achieves ontology-level standardization of subjects. In this embodiment, ontology refers to the category relationship between entities and concepts. For example, the entity is "Student 02," and the concept is "student of Class 2 of Grade 3," or the entity is "lawyer," and the concept is "profession." That is, there is an ontology between entities, and this ontology can be the commonalities between two entities.
[0073] It is understandable that in some embodiments, the object may also be the phenotype corresponding to the entity, or the object may also be an entity. In this case, the process of clustering the 300 objects can be similar to the process of clustering the 300 subjects, and the specific process will not be elaborated here. That is to say, the objects can also be normalized at the entity level and / or at the ontology level.
[0074] Additionally, in some embodiments, the predicate can be a relationship between a subject and an object. For example, (Xiaoming, gnaw, apple) is a triple, and (Xiaohong, swallow, dumpling) is another triple, where "gnaw" is the relationship between "Xiaoming" and "apple," and "swallow" is the relationship between "Xiaohong" and "dumpling." When clustering the 300 predicates as described above, it is possible to determine at least two identical relationships among these 300 predicates. For example, "gnaw" and "swallow" can be normalized to "eat," therefore, "gnaw" and "swallow" are identical relationships. The process of normalizing "gnaw" and "swallow" to "eat" achieves relational level normalization of the predicates.
[0075] Optionally, the clustering results include at least one of the following: at least two phenotypes of the same entity, commonalities between at least two entities, and at least two identical relationships.
[0076] For example, based on the clustering process described above, the resulting clustering can include at least one of the following: at least two phenotypes of the same entity, commonalities between at least two entities, or at least two identical relations. Specifically, at least two phenotypes of the same entity can be used as entity-level standardization results, commonalities between at least two entities can be used as ontology-level standardization results, and at least two identical relations can be used as relation-level standardization results. In other words, the clustering process mainly involves entity-level and ontology-level standardization of the subject, entity-level and ontology-level standardization of the object, and relation-level standardization of the predicate. Different levels of standardization results can be obtained after clustering.
[0077] S306. Adjust the first data set according to the clustering results to obtain the second data set.
[0078] Specifically, the implementation process and underlying principles of S306 and S103 are the same, and will not be repeated here.
[0079] In this embodiment, the pre-trained language model is pre-trained before encoding text information containing structured data. This pre-training further improves the accuracy of the pre-trained language model, enabling it to output more accurate contextual representation vectors, thereby strengthening the structural information of the standardized knowledge graph. Furthermore, the pre-trained language model contains factual and common-sense knowledge to a certain extent, which is beneficial for ontology-level standardization.
[0080] It is understood that an open knowledge graph, i.e., a first data set, can be standardized to obtain a second data set as described above. This embodiment does not limit the degree of standardization of the second data set. For example, the second data set can be an ontology-based knowledge graph, which is a structured knowledge graph with a high degree of standardization or normalization. Ontological knowledge graphs include FreeBase and Wikidata, where nodes are entities and entities have ontologies. In e-commerce scenarios, ontology-based knowledge graphs have wide applications. For example, by querying an ontology-based knowledge graph, one can obtain the category of a product and its corresponding attribute information (model, brand), thereby facilitating the service platform to search for and recommend products to users. Typically, constructing an ontology-based knowledge graph is difficult and requires a lot of manpower to maintain. However, constructing an open knowledge graph is relatively easy. For example, Open Information Extraction (OpenIE) tools can be used to mine subject-verb-object triples from unstructured text, and then these triples can be used to construct an open knowledge graph. Therefore, the method described in this embodiment can standardize open knowledge graphs into ontology-based knowledge graphs. This process requires no manual intervention, thus reducing the difficulty of constructing ontology-based knowledge graphs. Furthermore, standardizing open knowledge graphs into ontology-based knowledge graphs includes entity-level standardization, relation-level standardization, and ontology-level standardization as described above. Using the method described in this embodiment, the degree of entity-level standardization can be improved by 5%, the degree of relation-level standardization by 55%, the degree of ontology-level standardization for subjects by 8%, and the degree of ontology-level standardization for objects by 2%.
[0081] Furthermore, because this embodiment adds a pre-training step for the pre-trained language model, the pre-trained language model becomes more accurate, thereby further improving the degree of entity-level standardization, relation-level standardization, and ontology-level standardization. For example... Figure 5 As shown, this embodiment statistically analyzes the improvement in standardization of the same pre-trained language model after pre-training with different pre-training methods compared to before pre-training, and the improvement in standardization of different pre-trained language models after pre-training with the same pre-training method compared to before pre-training. Figure 5 Bert, Roberta, SpanBert, and ERNIE2.0 represent different pre-trained language models. Figure 5The subword and triple shown represent different pre-training methods. For example, subword represents the masked pre-training of a common masked language model (MLM), such as randomly replacing or masking words with a 15% probability. Figure 5 The triple shown represents Triple Pretraining or Triple Pretraining as described above, that is, the pretraining method represented by triple is an embodiment of this disclosure. Figure 4 The pre-training method is shown below. `subj` represents the subject, and `obj` represents the object. NPC-E represents entity-level standardization, NPC-O represents ontology-level standardization, and RPC represents relation-level standardization. Therefore, NPC-E(subj) represents entity-level standardization for the subject. NPC-E(obj) represents entity-level standardization for the object. NPC-O(subj) represents one metric for ontology-level standardization for the subject, and NPC-O(subj)-Jac represents another metric for ontology-level standardization for the subject. NPC-O(obj) represents one metric for ontology-level standardization for the object, and NPC-O(obj)-Jac represents another metric for ontology-level standardization for the object. Figure 5 Each value shown represents the improvement in standardization after pre-training compared to before pre-training. For example, 1.72 in the first row and second column indicates that the entity-level standardization for objects improved by 1.72 after pre-training BERT using the subword representation method compared to before pre-training. This demonstrates that pre-training a language model using pre-training methods can lead to a more stable and significant improvement in standardization.
[0082] In addition, this embodiment also compares the degree of entity-level standardization, relation-level standardization, and ontology-level standardization achieved by BERT, pre-trained method + BERT, non-neural network method, and static word vector method, respectively, as shown in Table 1 below.
[0083] Table 1
[0084]
[0085] As shown in Table 1, using a pre-training method combined with BERT significantly improves the standardization level compared to using BERT alone. Furthermore, the standardization achieved by BERT and the pre-training method combined with BERT is higher than that achieved by non-neural network methods and static word vector methods. In other words, the pre-trained language model provided in this embodiment yields better standardization results than phenotypic-dependent non-neural network methods and static word vector methods, and the method provided in this embodiment shows a more significant improvement in relation-level standardization and ontology-level standardization.
[0086] Furthermore, in this embodiment, each component element of the triple can be input into a pre-trained language model to obtain a representation vector for each component element. Alternatively, the three components of the triple can be concatenated and input into the pre-trained language model, or the three components of the triple can be concatenated with preset characters added at the concatenation positions and then input into the pre-trained language model. Another option is to input the original sentence or the entire sentence providing the triple into the pre-trained language model. However, comparison shows that inputting the original sentence or the entire sentence providing the triple into the pre-trained language model and using mean pooling yields a more accurate context representation vector. This combination achieves the best results, highlighting the importance of context. In other words, the more accurate the context representation vector of each component element in the triple calculated by the pre-trained language model, the higher the degree of standardization for the open knowledge graph. Additionally, standardization of the open knowledge graph can better model the context. In addition, since the pre-trained language model contains factual and common-sense information, the pre-trained language model provided in this embodiment has better standardization results compared with non-neural network methods and static word vector methods that rely on phenotypes.
[0087] Figure 6 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of this disclosure. The data processing apparatus provided in this embodiment of the disclosure can execute the processing flow provided in the data processing method embodiment, such as... Figure 6 As shown, the data processing device 60 includes:
[0088] Encoding module 61 is used to encode text information containing structured data using a pre-trained language model to obtain context representation vectors corresponding to multiple components in the structured data in the text information.
[0089] Clustering processing module 62 is used to perform clustering processing on multiple constituent elements of the same type in the multiple structured data included in the first data set according to the context representation vectors corresponding to the multiple constituent elements of the same type, and obtain clustering results;
[0090] The adjustment module 63 is used to adjust the first data set according to the clustering results to obtain the second data set.
[0091] Optionally, the text information is unstructured text; the data processing device 60 further includes an acquisition module 64, which is used to acquire the structured data from the unstructured text before the encoding module 61 encodes the text information containing structured data using a pre-trained language model.
[0092] Optionally, the text information is text information composed of multiple constituent elements in the structured data.
[0093] Optionally, the text information is obtained by concatenating multiple components in the structured data and adding preset characters at the concatenation positions.
[0094] Optionally, when clustering multiple constituent elements of the same type, the clustering processing module 62 is specifically used for at least one of the following:
[0095] If the multiple constituent elements of the same type are multiple phenotypes corresponding to at least one entity, then at least two phenotypes among the multiple phenotypes that correspond to the same entity are determined;
[0096] If the multiple constituent elements of the same type are multiple entities, then the commonalities between at least two of the multiple entities are determined;
[0097] If multiple constituent elements of the same type are multiple relations, then at least two of the multiple relations are identified as identical relations.
[0098] Optionally, the clustering results include at least one of the following:
[0099] At least two phenotypes of the same entity, at least two commonalities between the entities, and at least two identical relations.
[0100] Optionally, the multiple components of the structured data include subject, predicate, and object;
[0101] The subject is either the phenotype corresponding to the entity, or the subject is the entity.
[0102] The object is the phenotype corresponding to the entity, or the object is the entity;
[0103] The predicate is the relationship between the subject and the object.
[0104] Optionally, the encoding module 61 uses a pre-trained language model to encode text information containing structured data, obtaining context representation vectors for the multiple components of the structured data in the text information, specifically for:
[0105] A pre-trained language model is used to encode text information containing structured data to obtain multiple representation vectors corresponding to any component element in the structured data in the text information.
[0106] Based on the multiple representation vectors corresponding to any one of the constituent elements in the text information, determine the context representation vector corresponding to any one of the constituent elements in the text information.
[0107] Optionally, the data processing device 60 further includes: a deletion module 65, an input module 66, and a training module 67; wherein, the deletion module 65 is used to delete any component element in the structured data in the text information before the encoding module 61 encodes the text information containing structured data using a pre-trained language model, to obtain the remaining text information; the input module 66 is used to input the remaining text information into the pre-trained language model, the pre-trained language model is used to predict any deleted component element based on the remaining text information; the training module 67 is used to train the pre-trained language model based on any deleted component element predicted by the pre-trained language model and any actually deleted component element.
[0108] Figure 6 The data processing apparatus of the illustrated embodiment can be used to execute the technical solutions of the above method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.
[0109] The above describes the internal functions and structure of a data processing device, which can be implemented as an electronic device. Figure 7 A schematic diagram illustrating the structure of an electronic device embodiment provided in this disclosure. (See attached diagram.) Figure 7 As shown, the electronic device includes a memory 71 and a processor 72.
[0110] Memory 71 is used to store programs. In addition to the programs described above, memory 71 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phone book data, messages, pictures, videos, etc.
[0111] The memory 71 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0112] The processor 72 is coupled to the memory 71 and executes the program stored in the memory 71 for:
[0113] A pre-trained language model is used to encode text information containing structured data to obtain context representation vectors corresponding to multiple components of the structured data in the text information.
[0114] Based on the context representation vectors corresponding to multiple components of the same type in the multiple structured data included in the first data set, clustering is performed on the multiple components of the same type to obtain clustering results;
[0115] The first data set is adjusted based on the clustering results to obtain the second data set.
[0116] Furthermore, such as Figure 7 As shown, the electronic device may also include other components such as a communication component 73, a power supply component 74, an audio component 75, and a display 76. Figure 7 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 7 The components shown.
[0117] Communication component 73 is configured to facilitate wired or wireless communication between electronic devices and other devices. The electronic devices can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 73 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 73 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0118] Power supply component 74 provides power to various components of the electronic device. Power supply component 74 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.
[0119] Audio component 75 is configured to output and / or input audio signals. For example, audio component 75 includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 71 or transmitted via communication component 73. In some embodiments, audio component 75 also includes a speaker for outputting audio signals.
[0120] Display 76 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.
[0121] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the data processing method described in the above embodiments.
[0122] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0123] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data processing method, wherein, The method includes: A pre-trained language model is used to encode text information containing structured data to obtain context representation vectors corresponding to multiple components of the structured data in the text information. Based on the context representation vectors corresponding to multiple components of the same type in the multiple structured data included in the first data set, clustering processing is performed on the multiple components of the same type to obtain clustering results. The first data set is used to represent an open knowledge graph, and the same type is used to represent at least one of subject, predicate, and object. The clustering processing includes entity-level standardization and ontology-level standardization of the subject, entity-level standardization and ontology-level standardization of the object, and relation-level standardization of the predicate. The clustering results include at least two phenotypes of the same entity as entity-level standardization results, commonalities between at least two entities as ontology-level standardization results, and at least two identical relations as relation-level standardization results. The first data set is adjusted based on the clustering results to obtain a second data set, wherein the second data set is used to represent the standardized knowledge graph and is used for data querying.
2. The method according to claim 1, wherein, The text information is unstructured text; Before encoding text information containing structured data using a pre-trained language model, the method further includes: The structured data is obtained from the unstructured text.
3. The method according to claim 1, wherein, The text information is composed of multiple elements from the structured data.
4. The method according to claim 1, wherein, The text information is obtained by concatenating multiple components of the structured data and adding preset characters at the concatenation positions.
5. The method according to claim 1, wherein, Clustering multiple constituent elements of the same type includes at least one of the following: If the multiple constituent elements of the same type are multiple phenotypes corresponding to at least one entity, then at least two phenotypes among the multiple phenotypes that correspond to the same entity are determined; If the multiple constituent elements of the same type are multiple entities, then the commonalities between at least two of the multiple entities are determined; If multiple constituent elements of the same type are multiple relations, then at least two of the multiple relations are identified as identical relations.
6. The method according to claim 1, wherein, The structured data contains multiple components, including subject, predicate, and object; The subject is either the phenotype corresponding to the entity, or the subject is the entity. The object is the phenotype corresponding to the entity, or the object is the entity; The predicate is the relationship between the subject and the object.
7. The method according to claim 1, wherein, A pre-trained language model is used to encode text information containing structured data, resulting in context representation vectors for multiple components of the structured data within the text information, including: A pre-trained language model is used to encode text information containing structured data to obtain multiple representation vectors corresponding to any component element in the structured data in the text information. Based on the multiple representation vectors corresponding to any one of the constituent elements in the text information, determine the context representation vector corresponding to any one of the constituent elements in the text information.
8. The method according to claim 1, wherein, Before encoding text information containing structured data using a pre-trained language model, the method further includes: Delete any element from the structured data in the text information to obtain the remaining text information; The remaining text information is input into the pre-trained language model, which is used to predict any of the deleted constituent elements based on the remaining text information. The pre-trained language model is trained based on any of the deleted components predicted by the pre-trained language model and any of the actually deleted components.
9. A data processing apparatus, wherein, include: The encoding module is used to encode text information containing structured data using a pre-trained language model to obtain context representation vectors corresponding to multiple components in the structured data in the text information. A clustering module is used to perform clustering processing on multiple components of the same type in a first data set according to the context representation vectors corresponding to the components of the same type in the multiple structured data included in the first data set, and to obtain a clustering result. The first data set is used to represent an open knowledge graph, and the same type is used to represent at least one of subject, predicate, and object. The clustering processing includes entity-level standardization and ontology-level standardization of the subject, entity-level standardization and ontology-level standardization of the object, and relation-level standardization of the predicate. The clustering result includes at least two phenotypes of the same entity as the entity-level standardization result, commonalities between at least two entities as the ontology-level standardization result, and at least two identical relations as the relation-level standardization result. The adjustment module is used to adjust the first data set according to the clustering results to obtain a second data set, wherein the second data set is used to represent the standardized knowledge graph and is used for data querying.
10. An electronic device, wherein, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-8.
11. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.