Operation and maintenance knowledge graph natural language retrieval method and device based on contrast learning

Abstract

The application provides an operation and maintenance knowledge graph natural language retrieval method and device based on contrast learning, equipment and a storage medium, comprising: using the contrast learning method to pre-train the original pre-training model, retaining the general knowledge learned by the original pre-training, and adding new features of the contrast learning that fit the target task; based on the contrast pre-training model, completing the vector representation of the retrieval sentence, recalling from the candidate sentences generated by the knowledge graph, and obtaining the retrieval answer. The method greatly reduces the time cost and material cost of manual labeling, simplifies the retrieval process, reduces the use of supervised learning models, uses the contrast pre-training model to meet the model requirements of the whole process, improves the retrieval efficiency while reducing the computing power consumption.

Description

Technical Field

[0001] This disclosure relates to the field of natural language retrieval of operation and maintenance knowledge graphs, and in particular to a method, apparatus, device and storage medium for natural language retrieval of operation and maintenance knowledge graphs based on contrastive learning. Background Technology

[0002] Current knowledge graph natural language retrieval methods involve named entity recognition models, question classification models, and path selection models, all of which are supervised learning methods. These require corpus annotation based on the task objective before model training, resulting in significant human resource consumption. Furthermore, the cost of constructing an entity link dictionary increases rapidly with the size of the knowledge graph. Moreover, as... Figure 1 The existing methods shown contain six stages in addition to the input-output stage, each with extremely high requirements for error control. However, these stages involve multiple supervised learning models, including named entity recognition, question classification, and path filtering, leading to unavoidable error accumulation that ultimately affects the accuracy of the retrieval results. Furthermore, the multi-stage models in the existing methods impact retrieval efficiency, requiring significant computational power and exhibiting bottlenecks in response time.

[0003] Current knowledge graph natural language retrieval methods involve supervised learning models at multiple stages of the process. However, when new entities, attributes, or graph structures are introduced into the knowledge graph, the effectiveness of the supervised learning models cannot be guaranteed. It is necessary to label and train the new data, which is a complex and time-consuming process. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method, apparatus, device, and storage medium for natural language retrieval of operation and maintenance knowledge graphs based on contrastive learning.

[0005] According to a first aspect of this disclosure, a natural language retrieval method for an operations and maintenance knowledge graph based on contrastive learning is provided, comprising: acquiring multiple retrieval statements used in a business scenario and constructing a semantic architecture; automatically generating candidate question-answer pairs on the operations and maintenance knowledge graph using the semantic architecture as rules, wherein each candidate question-answer pair includes multiple candidate retrieval statements and corresponding answers; using contrastive learning, inputting the candidate question-answer pairs into a pre-constructed original pre-trained model for contrastive pre-training, and updating the model to obtain a contrastive pre-trained model; inputting the multiple candidate retrieval statements from the candidate question-answer pairs into the contrastive pre-trained model to obtain multiple candidate output vectors, and saving the multiple candidate output vectors to a similarity vector retrieval library; inputting the retrieval statement into the contrastive pre-trained model to obtain a target output vector, and finding at least one candidate output vector with the highest similarity to the target output vector in the similarity vector retrieval library; and retrieving the answer corresponding to the retrieval statement based on the at least one candidate output vector.

[0006] According to embodiments of this disclosure, the step of obtaining multiple search statements used in a business scenario and constructing a semantic architecture includes: analyzing commonly used operation and maintenance search languages, summarizing the semantic reasoning structure of multiple search statements used in a business scenario, and constructing the semantic architecture based on the semantic reasoning structure.

[0007] According to embodiments of this disclosure, the semantic reasoning structure is derived by combining two dimensions: the number of entities appearing in the multiple search statements and the number of relation hops; the semantic reasoning structure is semantically identical to the search statements.

[0008] According to embodiments of this disclosure, the candidate search statements and their corresponding answers are assigned the same number to identify the record correspondence.

[0009] According to embodiments of this disclosure, the step of using contrastive learning to input the candidate question-answer pairs into a pre-constructed original pre-trained model for contrastive pre-training, and updating the contrastive pre-trained model, includes: acquiring backbone data of the candidate question-answer pairs and multiple candidate retrieval statements, wherein the backbone data includes entities and relations of the candidate retrieval statements; selecting one backbone data and a natural language question from the corresponding candidate question-answer pair, and randomly selecting to preprocess it with or without data augmentation methods to obtain preprocessed data; selecting N-2 natural language questions from the candidate question-answer pairs and the preprocessed data to form original training data, wherein the number of original training data is N; inputting the N original training data into the original pre-trained model, encoding them through the encoder in the original pre-trained model, and then entering the average pooling layer to obtain N sentence vectors; adjusting the N sentence vectors using a loss function to update the contrastive pre-trained model.

[0010] According to embodiments of this disclosure, the loss function is the NT-Xent loss function, and the expression for the NT-Xent loss function is:

[0011]

[0012] Where, sim(z) i z j The function is the cosine similarity function, r represents the sentence vector obtained after passing through the average pooling layer, and τ is a hyperparameter with a value range of 0.08-0.12.

[0013] According to an embodiment of this disclosure, the step of inputting multiple candidate retrieval statements from the candidate question-answer pair into the comparison pre-training model to obtain multiple candidate output vectors, and saving the multiple candidate output vectors to a similarity vector retrieval library, includes: sequentially inputting the candidate retrieval statements into the comparison pre-training model; passing all characters of the candidate retrieval statements through the average pooling layer of the comparison pre-training model to obtain candidate output vectors; saving the candidate output vectors to a vector similarity retrieval library, and simultaneously recording the number identifier of the representation vector.

[0014] According to embodiments of this disclosure, the retrieval method of the vector similarity retrieval library is the inverted product quantization method.

[0015] According to embodiments of this disclosure, inputting the statement to be retrieved into the comparison pre-training model to obtain a target output vector, and finding at least one candidate output vector with the highest similarity to the target output vector in the similarity vector retrieval library, includes: inputting the statement to be retrieved into the comparison pre-training model, passing all characters of the statement to be retrieved through the average pooling layer of the comparison pre-training model to obtain a target output vector; inputting the target input vector into a vector similarity retrieval library, and finding at least one candidate output vector with the highest similarity to the target output vector in the similarity vector retrieval library.

[0016] The second aspect of this disclosure provides a natural language retrieval device for an operations and maintenance knowledge graph based on contrastive learning, comprising: a question-answer pair generation module, which automatically generates candidate question-answer pairs on the operations and maintenance knowledge graph using the semantic architecture as a rule, wherein the candidate question-answer pairs include multiple candidate search statements and corresponding answers; a contrastive pre-training module, which uses contrastive learning to input the candidate question-answer pairs into a pre-constructed original pre-training model for contrastive pre-training, and updates the model to obtain a contrastive pre-training model; a database configuration module, which inputs multiple candidate search statements from the candidate question-answer pairs into the contrastive pre-training model to obtain multiple candidate output vectors, and saves the multiple candidate output vectors to a similarity vector retrieval library; a vector comparison module, which inputs the statement to be retrieved into the contrastive pre-training model to obtain a target output vector, and finds at least one candidate output vector with the highest similarity to the target output vector in the similarity vector retrieval library; and an answer retrieval module, which retrieves the answer corresponding to the statement to be retrieved based on the at least one candidate output vector.

[0017] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0018] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0019] The fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0020] This disclosure provides a method, apparatus, device, medium, and program product for natural language retrieval of operation and maintenance knowledge graphs based on contrastive learning. By leveraging the semantic reasoning structure characteristics of retrieval language in business scenarios, language rules describing entity nodes and relationships in the knowledge graph are summarized. A contrastive learning approach is then used to pre-train the original pre-trained model, retaining the general knowledge learned during pre-training and adding new contrastive learning features tailored to the target task. Based on the contrastive pre-trained model, a vector representation of the retrieval statement is generated, and the retrieval answer is obtained by recalling candidate statements generated from the knowledge graph. Since the retrieval language is automatically constructed using language rules derived from business scenarios as the backbone, it significantly reduces the time and material costs of manual annotation, creating conditions for full-process unsupervised retrieval in operation and maintenance knowledge graphs. Furthermore, by using contrastive learning, knowledge and features from the operation and maintenance knowledge graph are transferred to the pre-trained model, enabling the pre-trained model to achieve better results in handling tasks in the operation and maintenance domain. This disclosure allows for the entire process of relearning new domain knowledge by only changing the language rule configuration, enabling retrieval tasks to quickly adapt to new tasks, reducing the requirements for users, and allowing for flexible operation within a certain range, thus achieving a more open usage method. Attached Figure Description

[0021] Figure 1 The diagram illustrates the structure of knowledge graph natural language retrieval methods in the prior art.

[0022] Figure 2 The diagram illustrates the structure of a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning according to an embodiment of the present disclosure.

[0023] Figure 3 A flowchart illustrating a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning according to an embodiment of the present disclosure is shown.

[0024] Figure 4 The flowchart illustrating the construction of a semantic system for a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning according to an embodiment of the present disclosure is shown.

[0025] Figure 5The diagram illustrates a partial semantic reasoning structure within two hops for three entities in a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning, according to an embodiment of the present disclosure.

[0026] Figure 6 The flowchart illustrating the process of updating a contrastive pre-trained model using a contrastive learning-based natural language retrieval method for operational knowledge graphs according to an embodiment of the present disclosure is shown.

[0027] Figure 7 The diagram illustrates the structure of the raw training data for the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure.

[0028] Figure 8 The diagram schematically illustrates the structure of the original training model of the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure.

[0029] Figure 9 The flowchart illustrating the faiss database configuration of the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure is shown.

[0030] Figure 10 The flowchart illustrating the vector comparison of the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure is shown.

[0031] Figure 11 The diagram illustrates a structural block diagram of a natural language retrieval device for operation and maintenance knowledge graphs based on contrastive learning, according to an embodiment of the present disclosure.

[0032] Figure 12 A block diagram schematically illustrates an electronic device suitable for implementing a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning, according to embodiments of the present disclosure. Detailed Implementation

[0033] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0034] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0035] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0036] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0037] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of data (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0038] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0039] Supervised learning: The process of adjusting the parameters of a classifier using a set of samples of known classes to achieve the desired performance; also known as supervised training or teacher-guided learning. Supervised learning is a machine learning task that infers a function from labeled training data.

[0040] Unsupervised learning refers to learning the distribution of data or the relationships between data using unlabeled data.

[0041] Contrastive learning: a self-supervised learning method used to learn the general features of a dataset by having the model learn which data points are similar or different without labels.

[0042] Pre-trained model: An initial model is pre-trained on a raw task, and then the model is used on the target task. The initial model is fine-tuned according to the characteristics of the target task, thereby improving the performance of the target task.

[0043] Faiss, short for Facebook AI Similarity Search, is a tool developed by Facebook's AI team for large-scale similarity retrieval problems. Written in C++ with a Python interface, it achieves millisecond-level retrieval performance for indexes of up to one billion. Simply put, Faiss encapsulates candidate vector sets into an index database, accelerating the process of retrieving the Top K similar vectors. In this publication, it is presented as a similarity vector retrieval library.

[0044] (6) Mention: refers to named entities and attributes that appear in a question. Named entities and attributes that appear in a question are identified through named entity recognition technology, dictionary maximum matching recognition, and rule recognition.

[0045] Figure 1 This is a flowchart of existing knowledge graph natural language retrieval methods. For example... Figure 1 As shown, the flowchart includes: mention identification stage, entity linking stage, question classification stage, path generation stage, path sorting stage, and answer retrieval.

[0046] In current knowledge graph natural language retrieval methods, the named entity recognition model, question classification model, and path selection model are all supervised learning methods. These require corpus annotation based on the task objective before model training, resulting in significant human resource consumption. Furthermore, the cost of building an entity link dictionary increases rapidly with the expansion of the knowledge graph. Existing methods, excluding the input-output stage, have six other stages, each with extremely high requirements for error control. However, the involvement of multiple supervised learning models (named entity recognition, question classification, path selection, etc.) inevitably leads to error accumulation, ultimately affecting the accuracy of the retrieval results. Additionally, the multiple stages of existing methods impact retrieval efficiency, requiring high computational power and exhibiting bottlenecks in response time. When new entities, attributes, or graph structure updates are introduced into the knowledge graph, the effectiveness of the supervised learning models in multiple stages of the current knowledge graph natural language retrieval method cannot be guaranteed, necessitating annotation and training on new data, a complex and time-consuming process.

[0047] To address the aforementioned technical problems, this application proposes a method, apparatus, device, medium, and program product for natural language retrieval of operation and maintenance knowledge graphs based on contrastive learning. The technical solution of this application and how it solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0048] Figure 2 The diagram illustrates the structure of a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning according to an embodiment of the present disclosure.

[0049] Figure 3 A flowchart illustrating a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning according to an embodiment of the present disclosure is shown.

[0050] like Figure 2 , Figure 3 As shown, the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning in this embodiment includes operations S110 to S160.

[0051] In operation S110, multiple search statements used in the business scenario are obtained, and a semantic architecture is constructed.

[0052] Figure 4 The flowchart illustrating the construction of a semantic system for a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning according to an embodiment of the present disclosure is shown.

[0053] Specifically, multiple search queries used in the business scenario are obtained, and a semantic architecture is constructed, such as... Figure 4 As shown, the specific steps include:

[0054] In operating S1101, we analyzed commonly used operation and maintenance retrieval languages ​​and summarized the semantic reasoning structure of multiple retrieval statements used in business scenarios.

[0055] In operation S1102, a semantic architecture is constructed based on the semantic reasoning structure.

[0056] The semantic reasoning structure is derived by combining two dimensions: the number of entities appearing in multiple search statements and the number of relation hops. Furthermore, the semantic reasoning structure is semantically identical to the search statements.

[0057] Figure 5 The diagram illustrates a partial semantic reasoning structure within two hops for three entities in a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning, according to an embodiment of the present disclosure.

[0058] For example, such as Figure 5 As shown, the semantic reasoning structure obtained by combining the two dimensions of the number of entities and the number of relation hops can be in the form of single entity single hop, single entity double hop, single entity multi-hop, double entity single hop, double entity double hop, double entity multi-hop, three entity single hop, and multi entity multi-hop.

[0059] For example, when the search query is: "Who is the project manager of the system to which by18cupc2ap2001 belongs?", the semantic reasoning structure of this search query is a single-entity double-hop, where the entity is by18cupc2ap2001, and the relations are: system to which it belongs, managed. In this example, the system to which by18cupc2ap2001 belongs is also an intermediate entity, corresponding to... Figure 5 The middle element is a single-entity double jump.

[0060] Step S110 aims to use the semantic reasoning structure characteristics of the retrieval language in business scenarios to summarize the language rules described by entity nodes and relationships in the knowledge graph, and use these language rules as the backbone of the retrieval language to automatically construct natural language that is close to the business scenario, which greatly reduces the time and material costs of manual annotation and creates conditions for full-process unsupervised retrieval in the operation and maintenance knowledge graph.

[0061] When operating S120, using semantic architecture as the rule, candidate question-answer pairs are automatically generated on the operation and maintenance knowledge graph. Each candidate question-answer pair includes multiple candidate search statements and their corresponding answers.

[0062] In this embodiment, the same number identifier is generated between the candidate search statement and the corresponding answer to record the correspondence.

[0063] For example, in this embodiment, the semantic architecture is used as a rule to automatically generate candidate question-answer pairs on the operation and maintenance knowledge graph. For instance, a single-entity, single-hop semantic reasoning structure can be selected. The machine entity instance by18cupc2ap2001 in the operation and maintenance knowledge graph is selected, and all relations of this entity instance are traversed to generate corresponding retrieval languages ​​sequentially. For example, for the relation "belonging to system", the retrieval language "What system does by18cupc2ap2001 belong to?" can be constructed. "What system" can also be replaced with synonyms, such as "Which system does by18cupc2ap2001 belong to?". Using the semantic architecture as the subject structure of the retrieval statement adds descriptive language, making the retrieval language generated according to the semantic reasoning structure rules closer to the description methods in actual application scenarios. Finally, the question-answer pairs, candidate retrieval statement backbones (entities, relations), and their number identifiers are persistently saved to a database, such as MySQL or MongoDB.

[0064] In operation S130, a contrastive learning approach is used to perform contrastive pre-training on the original pre-trained model pre-built by inputting candidate question-answer pairs, thereby updating the contrastive pre-trained model.

[0065] Figure 6 The flowchart illustrating the process of updating a contrastive pre-trained model using a contrastive learning-based natural language retrieval method for operational knowledge graphs according to an embodiment of the present disclosure is shown.

[0066] Specifically, using contrastive learning, the candidate question-answer pairs are input into a pre-built, pre-trained model for comparative pre-training, resulting in an updated contrastive pre-trained model, such as... Figure 6 As shown, it may include steps S1301-S1305.

[0067] In step S1301, the backbone data of candidate question-answer pairs and multiple candidate search statements are obtained. The backbone data includes the entities and relationships of the candidate search statements.

[0068] In step S1302, a backbone data and the natural language question in the candidate question-answer pair corresponding to the backbone data are selected, and a data augmentation method is randomly selected to perform preprocessing to obtain preprocessed data.

[0069] The data augmentation methods mentioned above can be implemented using either synonym replacement or word order alteration.

[0070] Based on the above scheme, the data augmentation methods mentioned above in the pre-training stage can be replaced by implicitly generating augmented samples in the BERT Embedding layer, including: cropping, shuffling word order, and Dropout, which are obtained by modifying the Embedding matrix (or BERT's Position Encoding).

[0071] In step S1303, natural language questions from N-2 candidate question-answer pairs are selected and combined with preprocessed data to form the original training data, with the number of original training data being N.

[0072] Figure 7 The diagram illustrates the structure of the raw training data for the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure.

[0073] like Figure 7 As shown, the original training data for this input batch consists of preprocessed data and natural language questions from N-2 candidate question-answer pairs. The preprocessed data includes the selected retrieval statement backbone and the corresponding natural language questions in the candidate question-answer pairs.

[0074] In step S1304, N original training data are input into the original pre-trained model, encoded by the encoder in the original pre-trained model, and then entered into the average pooling layer to obtain N sentence vectors.

[0075] Figure 8 The diagram schematically illustrates the structure of the original training model of the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure.

[0076] like Figure 8 As shown, the original data, after data augmentation, forms a batch of data with a quantity of N. These N data points are all encoded by the BERT encoder and then passed through an average pooling layer to obtain N sentence vectors.

[0077] The original pre-trained model can be selected from, but is not limited to, the Roberta-wwm model, the Chinese-BERT-wwm model, the ERNIE model, the NEZHA model, the ALBERT model, or the XLNet model. Preferably, the Chinese-BERT-wwm model is used.

[0078] In step S1305, the loss function is used to adjust the N sentence vectors and update the comparison pre-trained model.

[0079] The loss function mentioned above is the NT-Xent loss function, and its expression is as follows:

[0080]

[0081] Where sim() is the cosine similarity function; r represents the sentence vector obtained after pooling; τ is a hyperparameter, typically taken as 0.08-0.12. i and z j Let z represent the search query i and its corresponding natural search query j, which are a pair of positive samples. i and z k Let i and k represent a pair of negative samples, respectively, which are the search query i and the natural search query k that is not a corresponding relationship.

[0082] The numerator represents the similarity between sample i and its corresponding positive sample j within the same batch. Higher similarity among positive samples is encouraged in the numerator. The denominator represents the similarity between sample i and all its corresponding negative samples k within the same batch. Lower similarity among negative samples is encouraged in the denominator.

[0083] The purpose of the loss function is to make two positive samples as consistent as possible in the representation space, while keeping them as far away as possible from other negative samples in the same batch.

[0084] Step S130 aims to use contrastive learning to transfer knowledge and features from the operations and maintenance knowledge graph to the pre-trained model. The contrastive pre-training retains the general knowledge learned in the original pre-training and adds new features that are tailored to the target task, enabling the pre-trained model to achieve better results in handling operations and maintenance tasks.

[0085] In operation S140, multiple candidate retrieval statements from the candidate question-answer pair are input into the pre-trained model to obtain multiple candidate output vectors, and these multiple candidate output vectors are saved to the similarity vector retrieval library.

[0086] Figure 9 The flowchart illustrating the faiss database configuration of the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure is shown.

[0087] Specifically, multiple candidate retrieval statements from the candidate question-answer pair are input into the pre-trained model for comparison to obtain multiple candidate output vectors, which are then saved to a similarity vector retrieval database. For example... Figure 9 As shown, it specifically includes steps S1401-S1402.

[0088] In operation S1401, candidate search statements are sequentially input into the comparison pre-trained model. All characters of the candidate search statements are passed through the average pooling layer of the comparison pre-trained model to obtain the candidate output vector.

[0089] In operation S1402, the candidate output vectors are saved to the vector similarity retrieval database, and the number identifier representing the vector is recorded at the same time.

[0090] The retrieval method of the aforementioned vector similarity retrieval library is the inverted product quantization (IVFxPQy) method. By using the product quantization method, the k-means of IVF is improved. The dimension of a vector is cut into x segments, and each segment is searched using k-means.

[0091] In operation S150, the query to be retrieved is input into the pre-trained model to obtain the target output vector. At least one candidate output vector with the highest similarity to the target output vector is found in the similarity vector retrieval library.

[0092] Figure 10 The flowchart illustrating the vector comparison of the operation and maintenance knowledge graph natural language retrieval method based on contrastive learning according to an embodiment of the present disclosure is shown.

[0093] Specifically, the query statement is input into the pre-trained model to obtain the target output vector. Then, at least one candidate output vector with the highest similarity to the target output vector is found in the similarity vector retrieval database. Figure 10 As shown, the specific steps include S1501-S1502.

[0094] In operation S1501, the statement to be retrieved is input into the comparison pre-trained model. All characters of the statement to be retrieved are passed through the average pooling layer of the comparison pre-trained model to obtain the target output vector.

[0095] In operation S1502, the target input vector is input into the vector similarity retrieval library, and at least one candidate output vector with the highest similarity to the target output vector is found in the similarity vector retrieval library.

[0096] In operation S160, the answer corresponding to the search query is found based on at least one candidate output vector.

[0097] Through the above embodiments, this disclosure utilizes an unsupervised learning method, eliminating the need for labeled training data throughout the entire process. Even in the semantic architecture construction stage, it involves summarizing existing commonly used retrieval language structures, significantly reducing labor and time costs. Furthermore, this disclosure streamlines the retrieval process, reducing the use of supervised learning models and leveraging a comparative pre-trained model to meet the model requirements throughout the entire process, improving retrieval efficiency while reducing computational consumption. When new entities, attributes, or graph structure updates are introduced into the knowledge graph, the candidate set and the unsupervised comparative pre-trained model are automatically updated, enabling the retrieval function to quickly adapt to data changes.

[0098] Based on the aforementioned natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning, this disclosure also provides a natural language retrieval device for operation and maintenance knowledge graphs based on contrastive learning. The following will combine... Figure 11 The device is described in detail.

[0099] Figure 11 The diagram illustrates a structural block diagram of a natural language retrieval device for operation and maintenance knowledge graphs based on contrastive learning, according to an embodiment of the present disclosure.

[0100] like Figure 11 As shown, the operation and maintenance knowledge graph natural language retrieval device 1100 based on contrastive learning in this embodiment includes a semantic system construction module 1110, a question-answer pair generation module 1120, a contrastive pre-training module 1130, a database configuration module 1140, a vector comparison module 1150, and an answer retrieval module 1160.

[0101] The semantic system construction module 1110 is used to obtain multiple search statements used in the business scenario and construct a semantic system architecture. In one embodiment, the semantic system construction module 1110 can be used to perform the operation S110 described above, which will not be repeated here.

[0102] The question-answer pair generation module 1120 is used to automatically generate candidate question-answer pairs on the operation and maintenance knowledge graph using the semantic architecture as rules. The candidate question-answer pairs include multiple candidate search statements and their corresponding answers. In one embodiment, the question-answer pair generation module 1120 can be used to perform the operation S120 described above, which will not be repeated here.

[0103] The contrastive pre-training module 1130 is used to perform contrastive pre-training on the original pre-trained model pre-built by the candidate question-answer pair input using contrastive learning, thereby updating the contrastive pre-trained model. In one embodiment, the contrastive pre-training module 1130 can be used to perform the operation S130 described above, which will not be repeated here.

[0104] The database configuration module 1140 is used to input multiple candidate retrieval statements from the candidate question-answer pairs into the comparison pre-trained model to obtain multiple candidate output vectors, and save the multiple candidate output vectors to the similarity vector retrieval library. In one embodiment, the database configuration module 1140 can be used to perform the operation S140 described above, which will not be repeated here.

[0105] The vector comparison module 1150 is used to input the query statement to be retrieved into the comparison pre-trained model to obtain the target output vector, and to find at least one candidate output vector with the highest similarity to the target output vector in the similarity vector retrieval library. In one embodiment, the vector comparison module 1150 can be used to perform the operation S150 described above, which will not be repeated here.

[0106] The answer retrieval module 1160 is used to find the answer corresponding to the query statement based on the at least one candidate output vector. In one embodiment, the answer retrieval module 1160 can be used to perform the operation S160 described above, which will not be repeated here.

[0107] According to embodiments of this disclosure, any and multiple modules among the semantic system construction module 1110, question-answer pair generation module 1120, comparison pre-training module 1130, database configuration module 1140, vector comparison module 1150, and answer retrieval module 1160 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functionality of one or more of these modules can be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the semantic system construction module 1110, question-answer pair generation module 1120, comparison pre-training module 1130, database configuration module 1140, vector comparison module 1150, and answer retrieval module 1160 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or any other reasonable means of integrating or packaging circuits, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the semantic system construction module 1110, question-answer pair generation module 1120, comparison pre-training module 1130, database configuration module 1140, vector comparison module 1150, and answer retrieval module 1160 can be at least partially implemented as computer program modules, which can perform corresponding functions when the computer program module is run.

[0108] Figure 12 A block diagram schematically illustrates an electronic device suitable for implementing a natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning, according to embodiments of the present disclosure.

[0109] like Figure 12 As shown, an electronic device 1200 according to an embodiment of the present disclosure includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0110] RAM 1203 stores various programs and data required for the operation of electronic device 1200. Processor 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. Processor 1201 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 1202 and / or RAM 1203. It should be noted that the programs may also be stored in one or more memories other than ROM 1202 and RAM 1203. Processor 1201 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0111] According to embodiments of this disclosure, the electronic device 1200 may further include an input / output (I / O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1210 as needed so that computer programs read from it can be installed into the storage section 1208 as needed.

[0112] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0113] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1202 and / or RAM 1203 and / or one or more memories other than ROM 1202 and RAM 1203 described above.

[0114] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the item recommendation method provided in the embodiments of this disclosure.

[0115] When the computer program is executed by the processor 1201, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0116] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1209, and / or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0117] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0118] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0120] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0121] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A natural language retrieval method for operation and maintenance knowledge graphs based on contrastive learning, comprising: Obtain multiple search statements used in the business scenario and construct a semantic architecture; Using the semantic architecture as a rule, candidate question-answer pairs are automatically generated on the operation and maintenance knowledge graph. Each candidate question-answer pair includes multiple candidate search statements and their corresponding answers. By using contrastive learning, the candidate question-answer pairs are compared and pre-trained with the pre-built original pre-trained model to update the contrastive pre-trained model. Multiple candidate retrieval statements from the candidate question-answer pair are input into the comparison pre-trained model to obtain multiple candidate output vectors, and the multiple candidate output vectors are saved to the similarity vector retrieval library; The search query is input into the comparison pre-training model to obtain the target output vector, and at least one candidate output vector with the highest similarity to the target output vector is found in the similarity vector retrieval library. Based on the at least one candidate output vector, find the answer corresponding to the search statement; The step of using contrastive learning to perform contrastive pre-training on the original pre-trained model pre-built by the candidate question-answer pair as input, and updating the contrastive pre-trained model, includes: Obtain the backbone data of the candidate question-answer pairs and multiple candidate search statements, wherein the backbone data includes the entities and relationships of the candidate search statements; Select one of the backbone data and the natural language question in the candidate question-answer pair corresponding to the backbone data, and randomly select to preprocess the data with or without data augmentation to obtain preprocessed data; the data augmentation method adopts one of the two data augmentation methods, namely synonym replacement or word order change, to perform data augmentation; Select N-2 natural language questions from the candidate question-answer pairs and combine them with the preprocessed data to form the original training data, where the number of original training data is N. The N original training data are input into the original pre-trained model, encoded by the encoder in the original pre-trained model, and then entered into the average pooling layer to obtain N sentence vectors. The N sentence vectors are adjusted using a loss function to update and obtain the comparison pre-trained model; The loss function is the NT-Xent loss function, and the expression for the NT-Xent loss function is as follows: Where, sim( The function is the cosine similarity function, and τ is a hyperparameter with a value range of 0.08-0.12; z i and z j These represent search query i and its corresponding natural search query j, respectively, and are a pair of positive samples; z i and z k Let i and k represent a pair of negative samples, which are search terms and natural search terms that are not corresponding.

2. The method according to claim 1, characterized in that, The process of acquiring multiple search statements used in the business scenario and constructing a semantic architecture includes: Analyze commonly used operation and maintenance retrieval languages ​​and summarize the semantic reasoning structure of multiple retrieval statements used in business scenarios; Based on the semantic reasoning structure, construct the semantic architecture.

3. The method according to claim 2, characterized in that: The semantic reasoning structure is derived by combining the number of entities and the number of relation hops in the multiple search statements. The semantic reasoning structure is semantically identical to the statement to be retrieved.

4. The method according to claim 1, characterized in that, The candidate search statements and their corresponding answers are assigned the same number to identify the corresponding records.

5. The method according to claim 4, characterized in that, The step of inputting multiple candidate retrieval statements from the candidate question-answer pairs into the comparison pre-trained model to obtain multiple candidate output vectors, and saving the multiple candidate output vectors to the similarity vector retrieval library, includes: The candidate search statements are sequentially input into the comparison pre-training model, and all characters of the candidate search statements are passed through the average pooling layer of the comparison pre-training model to obtain the candidate output vector; The candidate output vectors are saved to the similar vector retrieval database, and the number identifier of the candidate output vectors is recorded at the same time.

6. The method according to claim 5, characterized in that, The retrieval method of the similarity vector retrieval library is the inverted product quantization method.

7. The method according to claim 5, characterized in that, The query to be retrieved is input into the comparison pre-trained model to obtain the target output vector. At least one candidate output vector with the highest similarity to the target output vector is found in the similarity vector retrieval library, including: The search query is input into the comparison pre-training model, and all characters of the search query are passed through the average pooling layer of the comparison pre-training model to obtain the target output vector; The target output vector is input into a similar vector retrieval library, and at least one candidate output vector with the highest similarity to the target output vector is found in the similar vector retrieval library.

8. A natural language retrieval device for an operation and maintenance knowledge graph based on contrastive learning, comprising: The semantic system construction module acquires multiple search statements used in the business scenario and constructs a semantic system architecture. The question-answer pair generation module uses the semantic architecture as a rule to automatically generate candidate question-answer pairs on the operation and maintenance knowledge graph. The candidate question-answer pairs include multiple candidate search statements and their corresponding answers. The contrastive pre-training module is used to perform contrastive pre-training on the original pre-trained model pre-built by the candidate question-answer pair input using contrastive learning, and update the contrastive pre-trained model. The database configuration module is used to input multiple candidate retrieval statements from the candidate question-answer pair into the comparison pre-trained model to obtain multiple candidate output vectors, and save the multiple candidate output vectors to the similarity vector retrieval library; The vector comparison module is used to input the statement to be retrieved into the comparison pre-training model to obtain the target output vector, and to find at least one candidate output vector with the highest similarity to the target output vector in the similarity vector retrieval library; The answer retrieval module is used to find the answer corresponding to the search statement based on the at least one candidate output vector; The contrastive pre-training module is used for: Obtain the backbone data of the candidate question-answer pairs and multiple candidate search statements, wherein the backbone data includes the entities and relationships of the candidate search statements; Select one of the backbone data and the natural language question in the candidate question-answer pair corresponding to the backbone data, and randomly select to preprocess the data with or without data augmentation to obtain preprocessed data; the data augmentation method adopts one of the two data augmentation methods, namely synonym replacement or word order change, to perform data augmentation; Select N-2 natural language questions from the candidate question-answer pairs and combine them with the preprocessed data to form the original training data, where the number of original training data is N. The N original training data are input into the original pre-trained model, encoded by the encoder in the original pre-trained model, and then entered into the average pooling layer to obtain N sentence vectors. The N sentence vectors are adjusted using a loss function to update and obtain the comparison pre-trained model; The loss function is the NT-Xent loss function, and the expression for the NT-Xent loss function is as follows: Where, sim( The function is the cosine similarity function, and τ is a hyperparameter with a value range of 0.08-0.12; z i and z j These represent search query i and its corresponding natural search query j, respectively, and are a pair of positive samples; z i and z k Let i and k represent a pair of negative samples, which are search terms and natural search terms that are not corresponding.

9. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images