Industrial knowledge graph question answering method and computer-readable medium based on large language model

Abstract

The embodiments of the present disclosure disclose an industrial knowledge graph question-answering method and a computer-readable medium based on a large language model. A specific implementation of the method includes: inputting at least one question information in the question information set into the initial question embedding network; inputting at least one question information into the initial question relationship chain feature extraction network; determining the semantic similarity between at least one question information feature vector and the corresponding initial question relationship chain feature vector; constructing at least one question information sample based on at least one question information and at least one initial question relationship chain feature vector; constructing at least one question information annotation sample based on at least one question information sample and the question-answering industrial knowledge graph; training the initial industrial knowledge graph question-answering model to obtain a trained industrial knowledge graph question-answering model. This implementation can better utilize existing knowledge and semantic information, thereby improving the accuracy and effect of answer reasoning.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the computer field, and specifically to an industrial knowledge graph question-answering method based on a large language model and a computer-readable medium. Background Art

[0002] The question-answering system based on the industrial knowledge graph is the most intelligent method for knowledge acquisition at present. It integrates cross-research fields such as semantic web, cognitive computing, data mining and machine learning, and is an important research topic in the field of artificial intelligence. Since the industrial knowledge graph was proposed in 2012, combined with the development of artificial intelligence technology in the past decade, the traditional intelligent question-answering system based on the industrial knowledge graph has supported people to obtain the desired knowledge or answers to questions only through simple language expressions or natural language question texts. The system will automatically understand people's intentions and directly give multiple answers sorted by the degree of matching.

[0003] However, when the above method is used to give answers, the following technical problems usually exist: Traditional industrial knowledge graph intelligent question answering mainly relies on keyword matching and simple pattern recognition, which makes it difficult to deeply understand the complex intentions and implicit semantic information in user queries, resulting in the inability to effectively handle fuzzy or ambiguous queries.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose industrial knowledge graph question-answering methods, devices, electronic devices, and computer-readable media based on a large language model to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide an industrial knowledge graph question-answering method based on a large language model, the method comprising: obtaining a question information set; determining a pre-trained large language model, wherein the pre-trained large language model comprises: an initial question embedding network, an initial question relationship chain feature extraction network; inputting at least one question information in the question information set into the initial question embedding network to obtain at least one question information feature vector; inputting the at least one question information into the initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector; determining whether the at least one question information feature vector corresponds to the corresponding initial question relationship chain feature vector. semantic similarity between vectors; in response to at least one semantic similarity being greater than or equal to a preset similarity, constructing at least one question information sample based on the at least one question information and the at least one initial question relationship chain feature vector; constructing at least one question information annotation sample based on the at least one question information sample and a preset question and answer industry knowledge graph; training the initial industry knowledge graph question and answer model based on the at least one question information annotation sample to obtain a trained industry knowledge graph question and answer model; in response to receiving the target question information, inputting the target question information into the industry knowledge graph question and answer model to obtain the target question question and answer result.

[0008] In a second aspect, some embodiments of the present disclosure provide an industrial knowledge graph question-answering device based on a large language model, the device comprising: an acquisition unit, configured to acquire a question information set; a first determination unit, configured to determine a pre-trained large language model, wherein the pre-trained large language model comprises: an initial question embedding network, an initial question relationship chain feature extraction network; a first input unit, configured to input at least one question information in the question information set into the initial question embedding network, to obtain at least one question information feature vector; a second input unit, configured to input the at least one question information into the initial question relationship chain feature extraction network, to obtain at least one initial question relationship chain feature vector; a second determination unit, configured to determine whether the at least one question information feature vector corresponds to the corresponding initial question relationship chain feature vector. The semantic similarity between the linked feature vectors; the first construction unit is configured to, in response to at least one semantic similarity being greater than or equal to a preset similarity, construct at least one question information sample according to the above-mentioned at least one question information and the above-mentioned at least one initial question relationship chain feature vector; the second construction unit is configured to construct at least one question information annotation sample according to the above-mentioned at least one question information sample and a preset question and answer industry knowledge graph; the training unit is configured to train the above-mentioned initial industry knowledge graph question and answer model according to the above-mentioned at least one question information annotation sample to obtain a trained industry knowledge graph question and answer model; the third input unit is configured to, in response to receiving the target question information, input the above-mentioned target question information into the above-mentioned industry knowledge graph question and answer model to obtain the target question and answer result.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0011] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the industrial knowledge graph question-answering method based on the large language model of some embodiments of the present disclosure, the relationship chain generated by the large language model is introduced to constrain and prompt the reasoning path of answer prediction. The relationship chain can mainly help us retrieve the answer to the question in the limited industrial knowledge graph according to the question subject entity and the number of question jumps. The relationship chain can reasonably expand the correct answer options and the entity information associated with the question on the complete industrial knowledge graph. It is proposed to use the Euclidean distance between the two in the vector space to represent the similarity between the relationship chain and the question, so as to select the relationship chain with the highest similarity as the reasoning path, and the entity it points to as the final answer. Thus, the accuracy of semantic recognition of question information can be improved, and the accuracy of answer query can be improved. Specifically, the reason why fuzzy or ambiguous queries cannot be effectively processed is that the traditional industrial knowledge graph intelligent question-answering mainly relies on keyword matching and simple pattern recognition, and it is difficult to deeply understand the complex intentions and implicit semantic information in the user query. Based on this, the industrial knowledge graph question-answering method based on the large language model of some embodiments of the present disclosure first obtains the question information set. Determine the pre-trained large language model. Among them, the above-mentioned pre-trained large language model includes: an initial question embedding network, an initial question relationship chain feature extraction network. Therefore, it is convenient to determine the entity relationship chain of the question information based on the pre-trained initial question embedding network and the initial question relationship chain feature extraction network. Secondly, at least one question information in the above-mentioned question information set is input into the initial question embedding network to obtain at least one question information feature vector. The above-mentioned at least one question information is input into the initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector. Determine the semantic similarity between the above-mentioned at least one question information feature vector and the corresponding initial question relationship chain feature vector. Therefore, the relationship between the question and the industry knowledge graph can be better understood, so as to perform more accurate answer reasoning. Then, in response to at least one semantic similarity being greater than or equal to the preset similarity, at least one question information sample is constructed based on the above-mentioned at least one question information and the above-mentioned at least one initial question relationship chain feature vector.

[0012] Afterwards, at least one question information annotation sample is constructed based on the above-mentioned at least one question information sample and the pre-set question and answer industry knowledge graph. This can help us capture the contextual information in the relationship chain. Then, based on the above-mentioned at least one question information annotation sample, the initial industry knowledge graph question and answer model is trained to obtain a trained industry knowledge graph question and answer model. Finally, in response to receiving the target question information, the above-mentioned target question information is input into the above-mentioned industry knowledge graph question and answer model to obtain the target question and answer result. Therefore, by using the existing industry knowledge graph relationship embedding representation, our model can better understand and represent the semantic information in the relationship chain. By applying these pre-trained embedding representations to the relationship chain, we can better utilize the existing knowledge and semantic information, thereby improving the accuracy and effectiveness of answer reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 It is a flowchart of some embodiments of the industrial knowledge graph question answering method based on a large language model according to the present disclosure;

[0015] Figure 2 It is a structural schematic diagram of some embodiments of the industrial knowledge graph question-answering device based on a large language model according to the present disclosure;

[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 The flowchart of some embodiments of the industrial knowledge graph question-answering method based on a large language model according to the present disclosure is shown. The industrial knowledge graph question-answering method based on a large language model comprises the following steps:

[0024] Step 101, obtaining a question information set.

[0025] In some embodiments, the execution subject of the industry knowledge graph question-answering method based on a large language model can obtain a question information set through a wired connection or a wireless connection. The question information set can be various questions submitted by various users. For example, the question information can be "In which year was the movie released that included the actor who starred in the movie "Pili Pili"?"

[0026] It should be noted that the above wireless connection methods may include but are not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0027] Step 102: determine a pre-trained large language model.

[0028] In some embodiments, the above-mentioned execution entity may determine a pre-trained large language model. The above-mentioned pre-trained large language model includes: an initial question embedding network and an initial question relationship chain feature extraction network. The pre-trained large language model may refer to a pre-trained large language model (LLM). The initial question embedding network may be an embedding network for generating a question information feature vector corresponding to the question information. For example, the initial question embedding network may be a RoBERTa model. The initial question relationship chain feature extraction network may be a feature extraction network for generating a question relationship chain feature vector corresponding to the question information. For example, the initial question relationship chain feature extraction network may include: a bidirectional GRU (Gate Recurrent Unit) gated recurrent unit and an attention mechanism layer. The bidirectional GRU (Gate Recurrent Unit) gated recurrent unit is used to obtain the embedding results of each entity relationship in the question information. The attention mechanism layer is used to generate a question relationship chain feature vector from each entity relationship embedding result.

[0029] Step 103: input at least one question information in the question information set into the initial question embedding network to obtain at least one question information feature vector.

[0030] In some embodiments, the execution entity may input at least one question information in the question information set into the initial question embedding network to obtain at least one question information feature vector, wherein one question information corresponds to a question information feature vector.

[0031] Step 104: input the at least one question information into an initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector.

[0032] In some embodiments, the execution subject may input the at least one question information into an initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector. One question information corresponds to one initial question relationship chain feature vector.

[0033] Step 105: Determine the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector.

[0034] In some embodiments, the execution entity may determine the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector.

[0035] For example, the semantic similarity between each question information feature vector and the initial question relationship chain feature vector corresponding to the question information feature vector can be determined by the following formula: .

[0036] in, Represents the feature vector of the problem information The corresponding initial problem relationship chain feature vector The semantic similarity between them, i.e., the 2-norm (Euclidean distance). ‌exp() represents an exponential function with the natural constant e as the base.

[0037] Step 106: in response to at least one semantic similarity being greater than or equal to a preset similarity, construct at least one question information sample according to the at least one question information and the at least one initial question relationship chain feature vector.

[0038] In some embodiments, the execution subject may construct at least one question information sample according to the at least one question information and the at least one initial question relationship chain feature vector in response to at least one semantic similarity being greater than or equal to a preset similarity, wherein the preset similarity may be pre-set.

[0039] For example, for each piece of question information, the question information and the corresponding initial question relationship chain feature vector may be combined into a question information sample.

[0040] Step 107: construct at least one question information annotation sample based on the at least one question information sample and the pre-set question and answer industry knowledge graph.

[0041] In some embodiments, the execution subject may construct at least one question information annotation sample based on the at least one question information sample and the pre-set question-answering industry knowledge graph. The question-answering industry knowledge graph may be an industry knowledge graph of each question entity node (entity relationship chain) established based on different question information. For example, let mathcalE and mathcalR be the entity set and relationship set in the industry knowledge graph KG, respectively. The domain space of the industry knowledge graph is regarded as the interaction domain of these entities and relationships. mathcalK⊆mathcalK×mathcalK×mathcalK represents all facts and knowledge in the knowledge base space. The way to use knowledge graph embedding to answer multi-hop natural language questions is to first learn the embedding representation of KG in an embedding space, then learn its embedding representation given a question, and finally use it in combination with the KG embedding method to predict the answer. The question-answering process can be abstractly described as given a natural language question q and a subject entity eh∈mathcalE in the question, the goal is to extract an entity et∈mathcalE that correctly answers the question, and the goal is to use mathematical methods to quantify the question embedding eq as the answer prediction from eh to et. For example, the question-and-answer industry knowledge graph can be a knowledge graph in the field of intelligent question-and-answering applied in the automotive industry, logistics industry, etc.

[0042] In practice, the above execution entity can construct at least one question information annotation sample through the following steps:

[0043] The first step is to determine the annotation label corresponding to the question information sample and the question-answering industry knowledge graph for each question information sample. For example, first, an entity relationship chain with the same characteristics as the question relationship chain included in the question information sample can be found from the question-answering industry knowledge graph. Then, the question label corresponding to the entity relationship chain is determined as the annotation label of the question information sample. The annotation label can represent the entity answer label corresponding to the question information.

[0044] In the second step, each question information sample is combined with the corresponding annotation label into a question information annotated sample.

[0045] Step 108: Train the initial industry knowledge graph question-answering model based on the at least one question information labeled sample to obtain a trained industry knowledge graph question-answering model.

[0046] In some embodiments, the execution subject may train the initial industry knowledge graph question-answering model based on the at least one question information labeling sample to obtain a trained industry knowledge graph question-answering model. The initial industry knowledge graph question-answering model may be an industry knowledge graph question-answering model that includes a pre-trained large language model. For example, the initial industry knowledge graph question-answering model may be a KGQA model.

[0047] In practice, the above-mentioned execution entity can train the initial industry knowledge graph question-answering model through the following steps to obtain a trained industry knowledge graph question-answering model:

[0048] In the first step, a target question information annotation sample is selected from the at least one question information annotation sample. For example, a question information annotation sample can be randomly selected from the at least one question information annotation sample as the target question information annotation sample.

[0049] In the second step, based on the above target question information, labeled samples are generated to generate entity alignment loss information and entity alignment information corresponding to the initial entity alignment network. Among them, the above initial industry knowledge graph question-answering model includes: initial entity alignment network, initial question semantic parsing network, answer filtering and relationship chain reasoning network. The initial entity alignment network may refer to an entity alignment network that has not yet been trained, and may be a neural network model that realizes the alignment of entities between the industry knowledge graph and the question relationship chain. The entity alignment network may be a weakly supervised model. For example, the entity alignment network may be a generative adversarial network (GAN) model. The entity alignment loss information may be the loss value output by the corresponding loss function of the initial entity alignment network.

[0050] The second step may include the following sub-steps:

[0051] In the first sub-step, each entity and entity relationship in the above-mentioned question-and-answer industry knowledge graph is vectorized to generate entity vectors and entity relationship vectors, and an entity vector set and an entity relationship vector set are obtained. Among them, the entity relationship can represent the association relationship between two entities in the question-and-answer industry knowledge graph. For example, the entity relationship can be an entity belonging relationship. There is a one-to-one correspondence between each entity in the question-and-answer industry knowledge graph and the entity vector in the entity vector set. There is a one-to-one correspondence between each entity relationship in the question-and-answer industry knowledge graph and the entity relationship vector in the entity relationship vector set. The entity vector can represent the corresponding feature semantics of the entity. The entity relationship vector can represent the corresponding feature semantics of the entity relationship. Each entity and entity relationship in the question-and-answer industry knowledge graph can be input into the pre-trained model PTM (Pre-Trained Model, such as BERT) to generate entity vectors and entity relationship vectors, and obtain an entity vector set and an entity relationship vector set.

[0052] In the second sub-step, the entity vector set and the entity relationship vector set are input into the initial entity alignment network to obtain entity alignment information. The initial entity alignment network includes an initial coding layer and an initial alignment reasoning layer, and the initial coding layer includes an initial position coding layer and an initial attention mechanism layer. The initial coding layer is a coding layer that has not yet been trained. The initial coding layer is a coding layer for combining the feature semantics of the graph anchor position, the entity vector set and the entity relationship vector set. The initial alignment reasoning layer may be an alignment reasoning layer that has not yet been trained. The alignment reasoning layer is a network layer used to determine the probability of alignment relationships between entities in the question-answering industry knowledge graph based on the feature semantics of each entity, the feature semantics of each entity relationship and the feature semantics of the entity at the anchor position. For example, the alignment reasoning layer may be a fully connected layer. The initial position coding layer may be a network layer that has not yet been trained and is used to encode the position. The initial attention mechanism layer may be a model that has not yet been trained and is used to better capture the information about the influence of the anchor position on entity alignment. For example, the initial position coding layer may be a multi-layer concatenated convolutional neural network. The initial attention mechanism layer can be a multi-head attention mechanism model.

[0053] The step of inputting the entity vector set and the entity relationship vector set into the initial entity alignment network comprises:

[0054] First, the graph anchor point position corresponding to the target question information annotation sample is obtained. The graph anchor point position can represent the position of the entity in the target question information annotation sample in the corresponding question-answering industry knowledge graph.

[0055] Second, the above-mentioned graph anchor point position, the above-mentioned entity vector set and the above-mentioned entity relationship vector set are input into the above-mentioned initial coding layer to obtain the graph fusion matrix. First, the above-mentioned graph anchor point position is position-encoded using the above-mentioned initial position coding layer to generate anchor point position coding information. Then, based on the above-mentioned anchor point position coding information, the above-mentioned initial attention mechanism layer is used to generate an initial attention mechanism vector. Then, based on the above-mentioned initial attention mechanism vector, the above-mentioned entity vector set and the above-mentioned entity relationship vector set, a graph fusion matrix is ​​generated.

[0056] Third, the graph fusion matrix is ​​input into the initial alignment reasoning layer to obtain the alignment matrix as entity alignment information.

[0057] The above-mentioned graph fusion matrix is ​​input into the above-mentioned initial alignment reasoning layer to obtain the alignment matrix, including:

[0058] 1. Using the above initial alignment reasoning layer, generate a similarity matrix corresponding to the graph fusion matrix. First, the above execution subject can use the initial alignment reasoning layer to perform matrix transposition processing on the graph fusion matrix to generate a transposed matrix. Then, the above execution subject can use the initial alignment reasoning layer to multiply the transposed matrix and the graph fusion matrix by cosine similarity to generate a similarity matrix.

[0059] 2. Perform alignment processing on the similarity matrix to generate an alignment matrix. The Sinkhorn operator can be used to perform alignment transformation on the similarity matrix to generate an alignment matrix.

[0060] In the third sub-step, entity alignment loss information is generated according to the target question information labeled sample and the entity alignment information. The target question information labeled sample and the entity alignment information can be vectorized respectively to obtain a first vectorization result and a second vectorization result. Finally, the first vectorization result and the second vectorization result are input into the target loss function to generate entity alignment loss information. For example, the target loss function can be a cross entropy loss function.

[0061] Therefore, by labeling samples with question information, weakly supervised learning of the initial entity alignment network can be achieved accurately and effectively.

[0062] The third step is to generate a reconstructed label based on the above entity alignment information. Among them, the reconstructed label set can be a label set that is re-selected based on the entity alignment information and represents the entity alignment relationship with a higher probability. As an example, first, the above execution subject can filter out elements whose corresponding element values ​​are greater than the target probability value from the probability matrix corresponding to the entity alignment information to obtain at least one element. The entity alignment information can be information in the form of a matrix in which the corresponding element value is between 0 and 1. The element value represents the entity content matching relationship between the corresponding multiple entities. The larger the element value, the more matching the corresponding entities. Then, determine at least one entity pair corresponding to the above at least one element. Based on the above at least one entity pair, generate a reconstructed label.

[0063] In the fourth step, in response to determining that the reconstructed label is consistent with the corresponding annotation label, the question information sample included in the target question information annotation sample is input into the initial question semantic parsing network included in the initial industry knowledge graph question answering model to obtain the initial question semantic vector. The initial question semantic parsing network can be a natural language question encoder, which can include: two fully connected layers, a ReLU activation function and a DropOut layer.

[0064] In the fifth step, the above-mentioned initial question semantic vector is input into the answer filtering and relationship chain reasoning network included in the initial industry knowledge graph question and answer model to obtain the initial industry knowledge graph question and answer result. The answer filtering and relationship chain reasoning network may include: an answer filtering network model and a relationship chain reasoning network model. The answer filtering network model is a model for multi-hop intelligent question and answer based on the relationship path of the industry knowledge graph. It converts the multi-hop intelligent question and answer problem into the problem of finding the optimal relationship path in the industry knowledge graph in a low-dimensional vector space. The model uses representation learning to embed the industry knowledge graph and user questions into a low-dimensional vector space at the same time, realizing a unified representation of the knowledge space and the problem space. Then, the candidate entities are semantically scored in combination with the subject entity vector representation and the question vector representation to generate a set of candidate answers. Taking the question entity as the starting node and the candidate answer entity as the ending node, a set of relationship paths related to the question is extracted from the industry knowledge graph. The relationship chain reasoning network model can refer to forming a structured graph that facilitates reasoning by constructing a series of reasoning nodes and the dependencies between these nodes. This graph can be a chain (CoT), tree (ToT) structure, or a more complex graph (GoT) structure.

[0065] Step 6: Determine the loss value between the above-mentioned initial industry knowledge graph question-answering result and the corresponding annotation label. For example, the loss value between the above-mentioned initial industry knowledge graph question-answering result and the corresponding annotation label can be determined by a preset loss function. The preset loss function can refer to a cross entropy loss function or a mean square error (MSE) loss function.

[0066] In the seventh step, in response to determining that the above loss value is less than or equal to the preset loss value, the initial industry knowledge graph question and answer model is determined as the trained industry knowledge graph question and answer model.

[0067] In the eighth step, in response to determining that the above-mentioned loss value is greater than the preset loss value, the model parameters of the initial industry knowledge graph question and answer model are adjusted, and the target question information annotation samples are reselected from the unselected question information annotation samples to perform model training on the adjusted initial industry knowledge graph question and answer model.

[0068] Therefore, it is proposed to use the Euclidean distance between the two in the vector space to represent the similarity between the relationship chain and the question, so as to select the relationship chain with the highest similarity as the reasoning path, and the entity it points to as the final answer. This method can avoid entities with high scores but low recall rates from being selected as answers, and ensure that the model is based on the correct logical relationship chain of the question to infer and predict the answer.

[0069] Step 109, in response to receiving the target question information, input the above target question information into the above industry knowledge graph question and answer model to obtain the target question question and answer result.

[0070] In some embodiments, the execution subject may, in response to receiving the target question information, input the target question information into the industry knowledge graph question-answering model to obtain the target question question-answering result. The industry knowledge graph question-answering model may be a pre-trained question-answering model, which may include: FiD model‌, (Knowledge-based QA, KBQA) model. The target question information may refer to newly proposed question information. The target question question-answering result may refer to the question-answering result corresponding to the target question information.

[0071] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an industrial knowledge graph question-answering device based on a large language model. These industrial knowledge graph question-answering device embodiments based on a large language model are similar to Figure 1 Corresponding to the method embodiments shown, the industrial knowledge graph question-answering device based on a large language model can be specifically applied to various electronic devices.

[0072] like Figure 2As shown, in some embodiments, the industrial knowledge graph question-answering device 200 based on the large language model includes: an acquisition unit 201, a first determination unit 202, a first input unit 203, a second input unit 204, a second determination unit 205, a first construction unit 206, a second construction unit 207, a training unit 208 and a third input unit 209. The acquisition unit 201 is configured to acquire a question information set; the first determination unit 202 is configured to determine a pre-trained large language model, wherein the pre-trained large language model includes: an initial question embedding network, an initial question relationship chain feature extraction network; the first input unit 203 is configured to input at least one question information in the question information set into the initial question embedding network to obtain at least one question information feature vector; the second input unit 204 is configured to input the at least one question information into the initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector; the second determination unit 205 is configured to determine the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector; the first construction unit 206 is configured to determine the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector; the first construction unit 206 is configured to determine the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector; the first construction unit 207 ...7 is configured to determine the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector; the first construction unit 207 is configured to determine the semantic similarity between The construction unit 206 is configured to construct at least one question information sample according to the at least one question information and the at least one initial question relationship chain feature vector in response to at least one semantic similarity being greater than or equal to a preset similarity; the second construction unit 207 is configured to construct at least one question information annotation sample according to the at least one question information sample and a preset question and answer industry knowledge graph; the training unit 208 is configured to train the initial industry knowledge graph question and answer model according to the at least one question information annotation sample to obtain a trained industry knowledge graph question and answer model; the third input unit 209 is configured to input the target question information into the industry knowledge graph question and answer model in response to receiving the target question information to obtain the target question and answer result.

[0073] It can be understood that the units recorded in the industrial knowledge graph question-answering device 200 based on the large language model and the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the industrial knowledge graph question-answering device 200 based on a large language model and the units contained therein, and will not be repeated here.

[0074] Reference below Figure 3, which shows a schematic diagram of the structure of an electronic device 300 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0075] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0076] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0077] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0078] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0079] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0080] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being installed in the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a question information set; determines a pre-trained large language model, wherein the above-mentioned pre-trained large language model includes: an initial question embedding network, an initial question relationship chain feature extraction network; inputs at least one question information in the above-mentioned question information set into the initial question embedding network to obtain at least one question information feature vector; inputs the above-mentioned at least one question information into the initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector; determines the above-mentioned at least one question information feature vector and the corresponding initial question relationship In response to at least one semantic similarity being greater than or equal to a preset similarity, at least one question information sample is constructed according to the at least one question information and the at least one initial question relationship chain feature vector; at least one question information annotation sample is constructed according to the at least one question information sample and a preset question and answer industry knowledge graph; according to the at least one question information annotation sample, an initial industry knowledge graph question and answer model is trained to obtain a trained industry knowledge graph question and answer model; in response to receiving target question information, the target question information is input into the industry knowledge graph question and answer model to obtain a target question and answer result.

[0081] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0082] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising: an acquisition unit, a first determination unit, a first input unit, a second input unit, a second determination unit, a first construction unit, a second construction unit, a training unit, and a third input unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring a set of problem information".

[0084] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0085] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. An industrial knowledge graph question answering method based on a large language model, comprising: Get the problem information set; Determine a pre-trained large language model, wherein the pre-trained large language model includes: an initial question embedding network and an initial question relationship chain feature extraction network; Inputting at least one question information in the question information set into an initial question embedding network to obtain at least one question information feature vector; Inputting the at least one question information into an initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector; Determining the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector; In response to at least one semantic similarity being greater than or equal to a preset similarity, constructing at least one question information sample according to the at least one question information and the at least one initial question relationship chain feature vector; Constructing at least one question information annotation sample according to the at least one question information sample and a preset question-answering industry knowledge graph; According to the at least one question information labeled sample, the initial industry knowledge graph question-answering model is trained to obtain a trained industry knowledge graph question-answering model; In response to receiving the target question information, the target question information is input into the industry knowledge graph question answering model to obtain the target question question answering result; The step of constructing at least one question information annotation sample according to the at least one question information sample and a preset question-answering industry knowledge graph includes: For each question information sample, determine the annotation label corresponding to the question information sample and the question-answering industry knowledge graph; Combine each question information sample with the corresponding annotation label to form a question information annotated sample; Among them, determining the annotation labels corresponding to the question information sample and the question-answering industry knowledge graph includes: Find the entity relationship chain with the same characteristics as the question relationship chain included in the question information sample from the question answering industry knowledge graph; Determine the question label corresponding to the entity relationship chain as the annotation label of the question information sample; The step of training the initial industry knowledge graph question-answering model according to the at least one question information labeling sample to obtain a trained industry knowledge graph question-answering model includes: Selecting a target question information annotation sample from the at least one question information annotation sample; According to the target question information labeled samples, entity alignment loss information and entity alignment information corresponding to the initial entity alignment network are generated, wherein the initial industry knowledge graph question answering model includes: an initial entity alignment network, an initial question semantic parsing network, an answer filtering and relationship chain reasoning network; generating a reconstruction label according to the entity alignment information; In response to determining that the reconstructed label is consistent with the corresponding annotation label, inputting the question information sample included in the target question information annotation sample into the initial question semantic parsing network included in the initial industry knowledge graph question answering model to obtain an initial question semantic vector; Inputting the initial question semantic vector into the answer filtering and relationship chain reasoning network included in the initial industry knowledge graph question-answering model to obtain the initial industry knowledge graph question-answering result; Determine the loss value between the initial industry knowledge graph question answering result and the corresponding annotation label; In response to determining that the loss value is less than or equal to a preset loss value, determining the initial industry knowledge graph question-answering model as a trained industry knowledge graph question-answering model; Among them, according to the above target problem information, the sample is labeled to generate the entity alignment loss information and entity alignment information corresponding to the initial entity alignment network, including: Vectorizing each entity and entity relationship in the question-answering industry knowledge graph to generate entity vectors and entity relationship vectors, and obtaining entity vector sets and entity relationship vector sets; Inputting the entity vector set and the entity relationship vector set into the initial entity alignment network to obtain entity alignment information, wherein the initial entity alignment network includes: an initial encoding layer and an initial alignment reasoning layer, and the initial encoding layer includes: an initial position encoding layer and an initial attention mechanism layer; Annotate samples according to the target question information and the entity alignment information to generate entity alignment loss information.

2. The method according to claim 1, wherein: The method further comprises: In response to determining that the loss value is greater than a preset loss value, the model parameters of the initial industry knowledge graph question and answer model are adjusted, and the target question information annotation samples are reselected from the unselected question information annotation samples to perform model training on the adjusted initial industry knowledge graph question and answer model.

3. An industrial knowledge graph question-answering device based on a large language model, applied to the method according to claim 1, comprising: an acquisition unit, configured to acquire a problem information set; A first determining unit is configured to determine a pre-trained large language model, wherein the pre-trained large language model includes: an initial question embedding network and an initial question relationship chain feature extraction network; A first input unit is configured to input at least one question information in the question information set into an initial question embedding network to obtain at least one question information feature vector; A second input unit is configured to input the at least one question information into an initial question relationship chain feature extraction network to obtain at least one initial question relationship chain feature vector; A second determining unit is configured to determine the semantic similarity between the at least one question information feature vector and the corresponding initial question relationship chain feature vector; A first construction unit is configured to construct at least one question information sample according to the at least one question information and the at least one initial question relationship chain feature vector in response to at least one semantic similarity being greater than or equal to a preset similarity; A second construction unit is configured to construct at least one question information annotation sample according to the at least one question information sample and a preset question-answering industry knowledge graph; A training unit is configured to train the initial industry knowledge graph question-answering model according to the at least one question information labeling sample, to obtain a trained industry knowledge graph question-answering model; The third input unit is configured to input the target question information into the industrial knowledge graph question and answer model in response to receiving the target question information, so as to obtain the target question and answer result.

4. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-2.

5. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images