Model distillation method and device for knowledge graph unknown entity reasoning, equipment and medium

Abstract

The application discloses a model distillation method and device for knowledge graph unknown entity reasoning, equipment and medium, relates to the field of artificial intelligence and machine learning technology, including: by converting the knowledge graph query request into a text prompt and a structured query, using a large language model to generate semantic embedding and fuse with structured embedding, the student model predicts the initial candidate entity. The large language model further evaluates and feeds back the newly added candidate entity and its confidence, forms a soft label distribution for optimizing the student model, and obtains an optimized knowledge graph distillation model. The optimized knowledge graph distillation model makes up for the shortcomings of traditional methods in handling unknown entities, improves the accuracy and generalization ability of knowledge graph reasoning, enhances the prediction ability of unknown entities, and provides reliable support for intelligent decision-making in complex scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and machine learning, and particularly relates to a model distillation method, device and equipment for knowledge graph unknown entity reasoning and a medium. BACKGROUND

[0002] The current application of knowledge distillation in knowledge graph reasoning generally adopts a single imitation framework of "teacher-student". The existing method usually assumes that the teacher model and the student model have consistent structures, and only differ in the embedding vector dimension. The knowledge transfer is achieved by letting the student model learn the output distribution (soft label) or hard label of the teacher model. In terms of entity and relationship encoding, the traditional method relies on random digital identifiers to initialize them, and then trains them based on the triple (static) or quadruple (time sequence) structure of the knowledge graph, so that the model can master the topological relationship pattern between entities. For time sequence knowledge graph reasoning, the existing method mainly predicts future facts by modeling time dependence based on historical snapshot sequences, but still does not break away from the closed world training paradigm that relies on known entity embeddings.

[0003] The existing method has two major defects: first, the semantic information is severely missing. Since the training process relies on randomly initialized entity / relationship encoding, a large amount of semantic information is discarded, making it difficult for the compressed student model to capture the implicit concept of the entity, cross-relation dependence and counterfactual structure clues, and limiting the reasoning accuracy. At the same time, the traditional distillation target only repeats the structure pattern learning of the original training, without supplementing additional semantic knowledge, further exacerbating the semantic poverty problem. Second, the unknown entity processing capability is insufficient. The existing method is trained in a closed world scenario, and only the entities observed during parameterized training are parameterized. When the query involves unknown entities, the extrapolation blind spot is caused due to the lack of corresponding embedding vectors, and the posterior probability collapses to zero. The parameter restriction caused by distillation compression makes the student model lack the ability to model long-distance path constraints and global ontology constraints, and cannot infer unknown entities through deep logical reasoning, resulting in a significant decline in generalization performance.

[0004] Therefore, how to make up for the lack of semantic information in the knowledge distillation process and improve the reasoning and prediction ability of the model for unknown entities has become a problem to be solved. SUMMARY

[0005] The main purpose of the present application is to provide a model distillation method, device and equipment for knowledge graph unknown entity reasoning and a medium, aiming to solve the technical problem of how to improve the reasoning and prediction ability of the model for unknown entities in the knowledge graph and retain semantic information.

[0006] To achieve the above purpose, the present application provides a model distillation method for knowledge graph unknown entity reasoning, comprising:

[0007] receiving a knowledge graph query request, converting the knowledge graph query request into a corresponding text prompt and a structured query;

[0008] calling a large language model to perform semantic encoding on a subject entity and a predicate relationship in the text prompt, to generate a semantic entity embedding and a semantic relationship embedding;

[0009] fusing the semantic entity embedding and the semantic relationship embedding with a structured embedding in the structured query to obtain a fused query representation;

[0010] using a student model to predict the fused query representation to obtain an initial candidate entity set and corresponding prediction scores;

[0011] constructing the text prompt, the initial candidate entity set and the corresponding prediction scores into reasoning instructions, and sending the reasoning instructions to the large language model to enable the large language model to perform rationality evaluation on the initial candidate entity set and feed back new candidate entities and corresponding confidence, wherein the new candidate entities are unknown candidate entities that do not appear in a known entity set of a knowledge graph and are consistent with the semantic logic of the knowledge graph query request;

[0012] merging the initial candidate entity set, the prediction scores, the new candidate entities and the confidence to form a soft label distribution;

[0013] jointly optimizing the student model based on the soft label distribution and a preset loss function to obtain an optimized knowledge graph distillation model.

[0014] In an embodiment, the step of receiving a knowledge graph query request and converting the knowledge graph query request into a corresponding text prompt and a structured query comprises:

[0015] parsing the knowledge graph query request to extract a subject entity and a predicate relationship;

[0016] constructing a first structured query according to the subject entity and the predicate relationship, wherein the first structured query is in the form of a triple;

[0017] if there is a timestamp in the knowledge graph query request, constructing a second structured query, wherein the second structured query is in the form of a quadruple including the timestamp;

[0018] organizing the subject entity, the predicate relationship and / or the timestamp according to a natural language order to form a text prompt.

[0019] In an embodiment, the step of calling the large language model to semantically encode the subject entity and the predicate relation in the text prompt to generate semantic entity embedding and semantic relation embedding comprises:

[0020] extracting a subject entity segment and a predicate relation segment from the text prompt;

[0021] packaging the subject entity segment and the predicate relation segment into an entity encoding request and a relation encoding request respectively, and sending them to a large language model for processing to make the large language model feedback original entity encoding vectors and original relation encoding vectors;

[0022] aligning the dimensions of the original entity encoding vectors and the original relation encoding vectors through a preset projection matrix to obtain dimension-aligned encoding vectors;

[0023] normalizing the dimension-aligned encoding vectors to obtain semantic entity embedding and semantic relation embedding.

[0024] In an embodiment, the step of fusing the semantic entity embedding and the semantic relation embedding with a structured embedding in the structured query to obtain a fused query representation comprises:

[0025] extracting a structured embedding corresponding to the structured query, wherein the structured embedding is generated based on a historical fact set of a knowledge graph;

[0026] performing dimension-uniform processing on the semantic entity embedding, the semantic relation embedding, and the structured embedding through a preset learnable projection matrix to obtain a first embedding, a second embedding, and a third embedding, wherein the first embedding corresponds to the semantic entity embedding, the second embedding corresponds to the semantic relation embedding, and the third embedding corresponds to the structured embedding;

[0027] calculating a first similarity and a second similarity based on the semantic entity embedding, the semantic relation embedding, and the structured embedding, wherein the first similarity is a semantic structure similarity between the semantic entity embedding and the structured embedding, and the second similarity is a relation structure similarity between the semantic relation embedding and the structured embedding;

[0028] assigning a dynamic weight according to the first similarity and the second similarity;

[0029] performing element-level weighted fusion on the first embedding, the second embedding, and the third embedding according to the dynamic weight to obtain a fused query representation.

[0030] In an embodiment, the step of using a student model to predict the fused query representation to obtain an initial candidate entity set and corresponding prediction scores comprises:

[0031] inputting the fused query representation into an embedding layer of the student model to generate a high-dimensional feature vector;

[0032] inputting the high-dimensional feature vector into an inference layer of the student model to perform correlation calculation in combination with known factual relationships of the knowledge graph to obtain a similarity score;

[0033] based on the similarity score, using a Softmax function of an output layer of the student model to calculate a prediction probability of all candidate entities in the knowledge graph;

[0034] sorting the candidate entities according to a preset rule, and selecting a preset number of entities at the top to form an initial candidate entity set;

[0035] extracting a prediction probability corresponding to each entity in the initial candidate entity set, and taking the prediction probability as a prediction score of the corresponding entity.

[0036] In an embodiment, the step of constructing the text prompt, the initial candidate entity set and the corresponding prediction score into inference instructions, and sending the inference instructions to the large language model, so that the large language model performs rationality evaluation on the initial candidate entity set and feeds back newly added candidate entities and corresponding confidence, comprises:

[0037] extracting core query elements in the text prompt, and associating and binding the initial candidate entity set and the corresponding prediction score to obtain associated and bound information, wherein the core query elements include subject entities, relationship types and query gaps;

[0038] constructing inference instructions based on the associated and bound information according to a preset logic;

[0039] performing syntax normalization processing on the inference instructions to obtain normalized inference instructions;

[0040] sending the normalized inference instructions to the large language model through a preset data transmission channel to make the large language model feed back a response result;

[0041] separating an initial candidate entity rationality evaluation list and an initial newly added candidate entity set from the response result;

[0042] inputting the initial newly added candidate entity set returned by the large language model into the student model to obtain a corresponding prediction score, and feeding back the prediction score to the large language model to trigger a second round of inference update to obtain an updated newly added candidate entity set.

[0043] analyzing the rationality evaluation list to obtain a screened initial candidate entity;

[0044] The confidence scores in the updated set of newly added candidate entities are validated to obtain the newly added candidate entities and their corresponding confidence scores.

[0045] In one embodiment, the step of jointly optimizing the student model based on the soft label distribution and a preset loss function to obtain the optimized knowledge graph distillation model includes:

[0046] The composition of the preset loss function is determined, wherein the preset loss function includes distillation loss, structural loss and semantic matching loss;

[0047] The Hubel loss function is used to calculate the distillation loss between the current prediction result of the student model and the soft label distribution.

[0048] The structural loss of the student model based on knowledge graph-based structured knowledge reasoning is calculated using the binary cross-entropy loss function.

[0049] The semantic matching loss between the semantic entity embedding, the semantic relation embedding, and the initial embedding of the student model is calculated using cosine similarity.

[0050] The distillation loss, the structural loss, and the semantic matching loss are weighted and summed using preset weighting coefficients to obtain the total loss function.

[0051] The total loss function is minimized using the Adam optimizer, and the parameters of the student model are iteratively updated according to a preset learning rate until the maximum number of iterations is reached, resulting in the optimized knowledge graph distillation model.

[0052] Furthermore, to achieve the above objectives, this application also proposes a model distillation apparatus for reasoning about unknown entities in a knowledge graph, the model distillation apparatus comprising:

[0053] The conversion module is used to receive knowledge graph query requests and convert the knowledge graph query requests into corresponding text prompts and structured queries;

[0054] The encoding module is used to call the large language model to perform semantic encoding on the subject entities and predicate relations in the text prompt, and generate semantic entity embeddings and semantic relation embeddings;

[0055] The fusion module is used to fuse the semantic entity embedding, the semantic relation embedding, and the structured embedding in the structured query to obtain the fused query representation;

[0056] The prediction module is used to predict the fused query representation using the student model to obtain an initial set of candidate entities and corresponding prediction scores.

[0057] receive a text prompt, an initial candidate entity set and a corresponding prediction score, and send the inference instruction to the large language model to make the large language model perform rationality evaluation on the initial candidate entity set, and feed back new candidate entities and corresponding confidence, wherein the new candidate entities are unknown candidate entities that do not appear in the known entity set of the knowledge graph and are consistent with the semantic logic of the knowledge graph query request;

[0058] a merging module configured to merge the initial candidate entity set, the prediction score, the new candidate entity and the confidence to form a soft label distribution;

[0059] an optimization module configured to jointly optimize the student model based on the soft label distribution and a preset loss function to obtain an optimized knowledge graph distillation model.

[0060] In addition, to achieve the above-mentioned purposes, the present application also provides a storage medium, which is a computer readable medium, and a computer program is stored on the storage medium. The computer program is executed by a processor to implement the steps of the model distillation method for knowledge graph unknown entity reasoning as described above.

[0061] In addition, to achieve the above-mentioned purposes, the present application also provides a computer program product, which includes a computer program. The computer program is executed by a processor to implement the steps of the model distillation method for knowledge graph unknown entity reasoning as described above.

[0062] The present application converts the knowledge graph query request into a text prompt and a structured query, generates semantic embeddings using a large language model, and fuses them with structured embeddings. The student model predicts the initial candidate entity. The large language model further evaluates and feeds back the new candidate entity and its confidence, forms a soft label distribution for optimizing the student model, and obtains an optimized knowledge graph distillation model. The optimized knowledge graph distillation model makes up for the shortcomings of traditional methods in handling unknown entities, improves the accuracy and generalization ability of knowledge graph reasoning, and enhances the prediction ability of unknown entities, providing reliable support for intelligent decision-making in complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor.

[0064] Figure 1A flowchart of a first embodiment of a model distillation method for unknown entity reasoning of a knowledge graph according to the present application;

[0065] Figure 2 A flowchart of a second embodiment of a model distillation method for unknown entity reasoning of a knowledge graph according to the present application;

[0066] Figure 3 A flowchart of a third embodiment of a model distillation method for unknown entity reasoning of a knowledge graph according to the present application;

[0067] Figure 4 A module structure diagram of a model distillation device for unknown entity reasoning of a knowledge graph according to the present application for the first embodiment of a model distillation method for unknown entity reasoning of a knowledge graph;

[0068] Figure 5 A device structure diagram of a hardware running environment involved in the model distillation method for unknown entity reasoning of a knowledge graph according to the embodiments of the present application.

[0069] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0070] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0071] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0072] The current application of knowledge distillation in knowledge graph reasoning generally adopts a single imitation framework of "teacher-student". The existing method usually assumes that the teacher model and the student model have consistent structure, and only the difference exists in the embedding vector dimension. By letting the student model learn the output distribution (soft label) or hard label of the teacher model, knowledge transfer is achieved. In terms of entity and relationship encoding, traditional methods rely on random digital identifiers to initialize them, and then train based on the triple (static) or quadruple (time sequence) structure of the knowledge graph, so that the model can master the topological relationship pattern between entities. For time sequence knowledge graph reasoning, the existing method mainly based on historical snapshot sequence predicts future facts by modeling time dependence, but still does not break away from the closed world training paradigm relying on known entity embedding.

[0073] Therefore, the application provides a model distillation method for knowledge graph unknown entity reasoning to improve the reasoning and prediction ability of the model for the knowledge graph unknown entity and retain semantic information. The main solution of the embodiment of the application is that a knowledge graph query request is received, and the knowledge graph query request is converted into a corresponding text prompt and a structured query; a large language model is called to perform semantic coding on the subject entity and the predicate relationship in the text prompt to generate semantic entity embedding and semantic relationship embedding; the semantic entity embedding and the semantic relationship embedding are fused with the structured embedding in the structured query to obtain a fused query representation; a student model is used to predict the fused query representation to obtain an initial candidate entity set and corresponding prediction scores; the text prompt, the initial candidate entity set and the corresponding prediction scores are constructed into reasoning instructions, and the reasoning instructions are sent to the large language model to enable the large language model to reasonably evaluate the initial candidate entity set and feed back new candidate entities and corresponding confidence, wherein the new candidate entities are unknown candidate entities that do not appear in a knowledge graph known entity set and are consistent with the semantic logic of the knowledge graph query request; the initial candidate entity set, the prediction scores, the new candidate entities and the confidence are merged to form a soft label distribution; and the student model is jointly optimized based on the soft label distribution and a preset loss function to obtain an optimized knowledge graph distillation model.

[0074] Based on the above, the embodiment of the application provides a model distillation method for knowledge graph unknown entity reasoning, which is described below with reference to Figure 1 , Figure 1 FIG. 1 is a flowchart of a model distillation method for knowledge graph unknown entity reasoning according to an embodiment of the application. In a multi-source data integration and decision assistance scenario, the method can process structured and unstructured data from multiple channels such as sensor observation, open source information summary, historical record archive, etc., to construct a dynamically updated knowledge graph. For information query requests proposed by users (such as “which objects are associated with a certain subject under certain conditions” “which associated features may the unrecorded objects have” etc.), a lightweight reasoning model after model distillation is used to quickly analyze the query intent, combine the existing structured knowledge and rich semantic information, accurately identify the known associated entities, and predict the unknown associated objects not appearing in the existing knowledge graph and their credibility.

[0075] In the embodiment, the model distillation method for knowledge graph unknown entity reasoning includes steps S10-S70:

[0076] In step S10, a knowledge graph query request is received, and the knowledge graph query request is converted into a corresponding text prompt and a structured query.

[0077] It should be noted that the knowledge graph query request refers to a request for obtaining specific information from the knowledge graph, and the core is to query the relationship or related attributes of a certain entity and other entities, which can be divided into static knowledge graph query request and time sequence knowledge graph query request. The former only involves the association query of entities and relationships, and the latter also contains the constraint condition of time dimension. The text prompt refers to a sentence formed by organizing the core elements in the knowledge graph query request in the natural language order. Its role is to enable the large language model to directly understand the query intention, and to provide a clear input for subsequent semantic coding. The structured query refers to a standardized query form formed by converting the core elements in the knowledge graph query request into the data format specification of the knowledge graph. In the static scenario, it is in the form of triple, and in the time sequence scenario, it is in the form of four-tuple containing timestamp, which is convenient for subsequent extraction of structured embedding.

[0078] Further, step S10 includes: first, parsing the knowledge graph query request to extract the subject entity and the predicate relationship. It should be noted that the subject entity is the entity element located at the first position in the knowledge graph query request, representing the initiator of the action or relationship, which will be mapped to a vector to participate in similarity calculation in the subsequent embedding process. In this embodiment, when the request is (A, support,?, 2025-08-16), A is identified as the subject entity, and its unique identifier is used for embedding lookup. In addition, the predicate relationship is the action or attribute description connecting the subject and the object in the query request, used to define the semantic association between the two, which can be an attribute association or an interactive association between entities. It is the key part of the semantic logic embodied in the knowledge graph triple or quadruple. In this embodiment, "support" is identified as the predicate relationship, and its number or name is extracted and sent to the embedding layer to capture the action semantics. Further, the parsing process is a process of tokenization and slot recognition on the original request, which takes the complete query as input and outputs the standardized representation of the subject entity and the predicate relationship, while retaining additional information such as timestamp. In this embodiment, the process splits the string into four slots, where the content of the subject slot and the predicate slot is recorded separately for subsequent parallel generation of text prompts and structured queries.

[0079] Secondly, a first structured query is constructed according to the subject entity and the predicate relationship, wherein the first structured query is in the form of a triple; if there is a timestamp in the knowledge graph query request, a second structured query is constructed, wherein the second structured query is in the form of a quadruple including the timestamp. It should be noted that the first structured query is a triple representation containing only the subject entity, the predicate relationship and the to-be-completed object, and does not contain time information, and is used for a static knowledge graph reasoning task. In the embodiment, when the input request is (A, support,?), the generated first structured query is (A, support,?), which can be directly read by a static student model and used for embedding calculation. In addition, the timestamp is a discrete time point or interval identifier attached to the query, which is used to distinguish the sequence of fact occurrence, and whether it exists or not determines whether the query is upgraded to a timing form. In the embodiment, 2025-08-16 is the timestamp, and when the parsing module detects this field, the quadruple construction logic is triggered. Further, the second structured query is a quadruple representation after the timestamp is appended to the triple, and the form is subject-predicate-object-time, which is used for a timing knowledge graph reasoning task. In the embodiment, the same request is upgraded to (A, support,?, 2025-08-16), wherein the time element is recorded by an independent slot, and the timing student model performs historical matching according to the snapshot sequence. It can be understood that it is first judged whether the query request carries a timestamp, if not, the first structured query is output, if yes, the second structured query is output, so that the static and timing two reasoning requirements are compatible in the same process, and it is ensured that the subsequent embedding and prediction steps can be executed according to the correct form, and the coverage ability of unknown entities is improved.

[0080] Finally, the subject entity, predicate relation, and / or timestamp are organized according to natural language word order to form a text prompt. It should be noted that natural language word order refers to the word arrangement that conforms to human reading habits, placing the subject at the beginning of the sentence, the predicate after the subject, and the time adverbial after the predicate or at the end of the sentence, making the sentence fluent and readable. In this embodiment, when the element is (A, support, ?, 2025-08-16), after being organized according to natural language word order, it becomes "A supported whom on August 16, 2025," thus allowing it to be directly encoded by the large language model. Additionally, the timestamp in the text prompt is converted into a complete date phrase containing year, month, and day to maintain the integrity and readability of the time information, and its format is consistent with everyday expressions. In this embodiment, 2025-08-16 is expanded to August 16, 2025, and inserted after the predicate to form a time adverbial, making the sentence semantically clear. Furthermore, the text prompts are the sole natural language input for subsequent semantic encoding, and their quality directly affects the accuracy of semantic embedding. Therefore, it is necessary to ensure that the subject-verb-time elements are complete and the word order is correct. In this embodiment, the organized text prompts do not contain numbering symbols and are presented entirely in Chinese characters and numbers, ensuring that the large language model can capture the subject of the action, the action itself, and the time point of the action.

[0081] Step S20: Call the large language model to perform semantic encoding on the subject entity and predicate relationship in the text prompt, and generate semantic entity embedding and semantic relationship embedding.

[0082] It should be noted that the Large Language Model (LLM) is a deep learning model with powerful natural language understanding and generation capabilities. It can capture deep semantic information in text and encode input text fragments to generate corresponding vector representations. In this embodiment, it is used to extract semantic features from the core elements of the text prompt. Semantic encoding is the process of converting natural language text content into high-dimensional vectors that can be processed by computers. This process maps the semantic information of the text into a vector space, allowing the distance between vectors to reflect the degree of semantic similarity. Semantic entity embedding is a high-dimensional vector representation of the subject entity obtained after semantic encoding. This vector contains the semantic features and contextual information of the subject entity and can be used for subsequent embedding and fusion operations. Semantic relation embedding is a high-dimensional vector representation of the predicate relation obtained after semantic encoding. This vector contains the semantic features and logical association information of the predicate relation and can be used in conjunction with semantic entity embedding to complete subsequent feature fusion.

[0083] Further, step S20 includes: first, from the text prompt Extracting subject entity fragments Relationship between the verb and the predicate It should be noted that the subject entity fragment is the text content in the text prompt for expressing the query core subject, which corresponds to the subject entity in the knowledge graph query request, is the core part of the text prompt with clear direction, and does not contain redundant modifying information. The predicate relationship fragment is the text content in the text prompt for expressing the association type of the subject entity and other entities, which corresponds to the predicate relationship in the knowledge graph query request, is the key part of the semantic logic of the text prompt, and only retains the core semantics of the association relationship.

[0084] Then, the subject entity fragment and the predicate relationship fragment are respectively encapsulated as entity encoding requests and relationship encoding requests, and are sent to the large language model for processing to make the large language model feedback original entity encoding vectors and original relationship encoding vectors. It should be noted that the subject entity fragment is the shortest continuous word sequence containing only the subject entity intercepted from the text prompt, which does not contain predicates or other components, and is used to obtain the subject side semantics alone. In this embodiment, when the text prompt is "A supported who on August 16, 2025", the subject entity fragment is truncated to A, and then encapsulated into an entity encoding request. In addition, the predicate relationship fragment is a word sequence containing a predicate and its directly associated time adverbial, which is used to capture the action itself and the background of the action, forming the relationship side semantic input. In this embodiment, "on August 16, 2025 supported who" is combined into a predicate relationship fragment, and is encapsulated into a relationship encoding request. Further, the original entity encoding vector is the hidden state output by the large language model after forward calculation of the entity encoding request, which has the same dimension as the internal hidden layer of the model, and retains all the context features of the subject. It can be understood that the subject entity fragment and the predicate relationship fragment are first encapsulated into independent requests, and then sent to the large language model in turn, and finally the returned original entity encoding vector and original relationship encoding vector are collected, so as to obtain decoupled semantic representation, providing high-fidelity input for subsequent dimension alignment and fusion. Specifically, the semantic encoder implementation layer of LLM respectively encodes the subject and the predicate:

[0085]

[0086] wherein represents the semantic encoding layer of LLM.

[0087] The original entity encoding vector and the original relation encoding vector are dimensionally aligned by a preset projection matrix to obtain a dimensionally aligned encoding vector. The preset projection matrix is a learnable matrix that is pre-trained and stored, and its function is to map vectors of different dimensions to the same dimensional space. The dimension of this matrix is determined by the target dimension of subsequent embedding fusion, and it is the core tool for realizing the unification of vector dimensions. In addition, the original entity encoding vector is a high-dimensional vector directly output by the large language model after semantic encoding of the subject entity fragment. This vector contains the deep semantic features of the subject entity, but the dimension may not match the target dimension of subsequent processing. In addition, the original relation encoding vector is a high-dimensional vector directly output by the large language model after semantic encoding of the predicate relation fragment. This vector contains the semantic logical features of the predicate relation, and its dimension is usually consistent with that of the original entity encoding vector, but may differ from the dimension required for subsequent fusion. In addition, dimension alignment is an operation that maps vectors of different sources or different dimensions to the same dimensional space through linear transformation, aiming to eliminate the influence of vector dimension difference on subsequent element-level operations and ensure the smooth progress of embedding fusion. It can be understood that the original entity encoding vector is multiplied by the preset projection matrix to obtain a dimensionally transformed entity vector, and the original relation encoding vector is multiplied by the same preset projection matrix to obtain a dimensionally transformed relation vector. The above dimensionally transformed entity vector and relation vector are the dimensionally aligned encoding vector. The specific formula is:

[0088]

[0089]

[0090] wherein and are projection matrices for dimension alignment, , , and represent the corresponding learnable parameters.

[0091] Finally, the dimension-aligned encoding vectors are normalized to obtain semantic entity embeddings and semantic relation embeddings. It should be noted that normalization is a data preprocessing operation, the core of which is to map the value range of the vector to a specific interval while keeping the direction of the vector unchanged, which can eliminate the dimensional differences of different dimensional features and avoid interference with subsequent calculations due to too large or too small feature value ranges. The dimension-aligned encoding vector is a vector obtained by linearly transforming the original entity encoding vector and the original relation encoding vector by a pre-set projection matrix. The semantic entity embedding is a standardized vector obtained by normalizing the dimension-aligned entity encoding vector, which retains the deep semantic features of the subject entity and has a unified numerical scale. In addition, the semantic relation embedding is a standardized vector obtained by normalizing the dimension-aligned relation encoding vector, which retains the semantic logic features of the predicate relation and has a numerical scale consistent with the semantic entity embedding, which can be used for collaborative operation with the semantic entity embedding. It can be understood that the dimension-aligned entity encoding vector and the relation encoding vector are obtained, and the normalization method is used to process the two types of vectors respectively to eliminate the dimensional differences within the vectors. The entity vector obtained after processing is the semantic entity embedding, and the relation vector obtained after processing is the semantic relation embedding.

[0092] Step S30, the semantic entity embedding and the semantic relation embedding are fused with the structured embedding in the structured query to obtain a fused query representation.

[0093] It should be noted that the structured embedding is a vector representation generated based on a historical fact set of a knowledge graph, which contains topological structure features of the knowledge graph and structured constraint information of entity relations, and is a direct carrier of structured knowledge. The fused query representation is a comprehensive vector obtained by integrating the feature information of the semantic entity embedding, the semantic relation embedding and the structured embedding, which contains deep association information at the semantic level and topological constraint information at the structure level, and can be directly used as input of the student model.

[0094] Further, the step S30 comprises: extracting a structured embedding corresponding to the structured query first. It is needed that the structured embedding is generated based on a historical fact set of the knowledge graph, and the historical fact set is a set of existing fact triples or quadruples in the knowledge graph, which is used to train the student model to learn the structural rules of entities and relations. In the embodiment, the historical fact set contains all entities supported by A in the past, and these facts are used to train the student model to enable it to predict unknown entities based on historical patterns. The corresponding initial vectors are obtained by looking up the numbered table of entities and relations, which are then used to fuse with the embedding layer of the student model. In the embodiment, the extraction process of the structured embedding is: according to the subject, predicate and timestamp in the structured query, the corresponding vectors are looked up from the entity embedding table and the relation embedding table respectively, and then these vectors are spliced or added to form the structured embedding.

[0095] Then, the semantic entity embedding, the semantic relation embedding and the structured embedding are respectively subjected to dimension uniform processing through a preset learnable projection matrix to obtain a first embedding, a second embedding and a third embedding, wherein the first embedding corresponds to the semantic entity embedding, the second embedding corresponds to the semantic relation embedding, and the third embedding corresponds to the structured embedding. It should be noted that the dimension uniform processing is an operation of mapping vectors of different dimensions to the same target dimension space through operation with the projection matrix, and the purpose is to eliminate the hindrance of dimension difference to subsequent fusion calculation and to ensure that various embedding vectors can be operated cooperatively. It can be understood that the semantic entity embedding is subjected to matrix multiplication operation with the preset learnable projection matrix to obtain the vector after dimension uniform, which is the first embedding; the semantic relation embedding is subjected to matrix multiplication operation with the same preset learnable projection matrix to obtain the vector after dimension uniform, which is the second embedding; and the structured embedding is subjected to matrix multiplication operation with the above preset learnable projection matrix to obtain the vector after dimension uniform, which is the third embedding.

[0096] Then, the first similarity and the second similarity are calculated based on the semantic entity embedding, the semantic relation embedding, and the structured embedding. It should be noted that the first similarity is a semantic structure similarity of the semantic entity embedding and the structured embedding, and the numerical size of the first similarity can reflect the matching degree of the semantic feature and the structured feature of the subject entity. The second similarity is a relation structure similarity of the semantic relation embedding and the structured embedding, and the numerical size of the second similarity can reflect the matching degree of the semantic feature and the structured feature of the predicate relation. It can be understood that the cosine similarity method is used for calculation in the embodiment. The core logic of the cosine similarity is to determine the similarity degree of two vectors by calculating the cosine value of the included angle between the two vectors, and the value range is between -1 and 1. The closer the numerical value is to 1, the higher the similarity degree of the two vectors, and the closer the numerical value is to -1, the greater the difference degree of the two vectors. In addition, the second similarity is a quantitative index for measuring the similarity degree between the semantic relation embedding and the structured embedding, and the numerical size of the second similarity can reflect the matching degree of the semantic feature and the structured feature of the predicate relation. The same calculation method, i.e., the cosine similarity, is used for the first similarity to ensure that the quantitative standards of the two types of similarities are consistent, which facilitates the subsequent dynamic weight distribution.

[0097] Then, the dynamic weight is distributed according to the first similarity and the second similarity. It should be noted that the dynamic weight is a weight coefficient that is adjusted in real time according to the numerical size of the first similarity and the second similarity, and the numerical value of the dynamic weight is updated with the change of the similarity. The purpose is to make the feature fusion process more suitable for the current semantic and structural matching of the query, which is different from the static distribution mode of the fixed weight.

[0098] Finally, the element-level weighted fusion is performed on the first embedding, the second embedding, and the third embedding according to the dynamic weight to obtain the fused query representation. It can be understood that the first similarity and the second similarity are normalized to obtain the first similarity weight and the second similarity weight, and the sum of the three weights is 1. Therefore, the third similarity weight, i.e., the weight corresponding to the structured embedding, is calculated according to the first similarity weight and the second similarity weight. The first embedding is multiplied by the first similarity weight, the second embedding is multiplied by the second similarity weight, and the third embedding is multiplied by the third similarity weight. The element-level summation operation is performed on the above three weighted vectors to obtain the fused query representation.

[0099] In step S40, the student model is used to predict the fused query representation to obtain an initial candidate entity set and a corresponding prediction score.

[0100] It should be noted that the student model is a lightweight knowledge graph reasoning model, and its embedding layer and scorer have been pre-trained through historical facts, and can quickly evaluate the likelihood of each candidate entity according to the query representation. In this embodiment, the model receives the fused query representation and outputs the score vector of all entities through one forward propagation. The initial candidate entity set is a fixed number of entity list selected by the student model according to the score from high to low, which represents the known and highest scoring potential answer in the graph. In this embodiment, the set usually takes the top one thousand entities, which is used for subsequent screening and expansion to ensure coverage while controlling the amount of calculation. The prediction score is a scalar value given by the student model for each entity, and the larger the value, the higher the probability that the entity becomes the correct answer, which can be directly used for sorting or probability normalization. In this embodiment, the score is calculated by the scorer through dot product or bilinear transformation, and can be used in subsequent soft label construction without normalization.

[0101] Further, step S40 includes: first input the fused query representation into the embedding layer of the student model to generate a high-dimensional feature vector. It should be noted that the embedding layer is one of the core components of the student model, and is a key structure connecting the input features and the internal reasoning layer of the model. Its role is to perform nonlinear transformation on the input fused feature vector, mine the potential association between features, and generate a more discriminative feature representation. In addition, the high-dimensional feature vector is the feature vector output by the embedding layer after nonlinear transformation of the fused query representation. The dimension of this vector is usually higher than that of the input fused query representation, contains more rich feature association information, and can provide sufficient feature support for the calculation of the subsequent reasoning layer. It can be understood that the fused query representation is input into the embedding layer of the student model, and the embedding layer performs nonlinear mapping on the fused query representation through the preset learnable parameters, completes the deep mining and association reconstruction of the features, and finally outputs a high-dimensional feature vector containing multi-source feature association information.

[0102] Then the high-dimensional feature vector is input into the inference layer of the student model, and the known fact relationship of the knowledge graph is combined for correlation calculation to obtain a similarity score. It should be noted that the inference layer is based on the input high-dimensional feature vector, and the structured constraint information of the knowledge graph is combined to complete the correlation reasoning, and the matching degree quantization value of each candidate entity and the query intent is output. In addition, the known fact relationship of the knowledge graph is the sum of the correlation information between the verified entities in the knowledge graph, which is in the form of a triple in a static scenario and a four-tuple with a timestamp in a time sequence scenario, and is a structured basis for correlation calculation by the inference layer. In addition, the similarity score is a quantization index obtained by the inference layer after correlating the high-dimensional feature vector and the feature vector of each candidate entity in the knowledge graph, and is used to measure the matching degree of the candidate entity and the query intent. The higher the score, the higher the matching degree. It can be understood that the high-dimensional feature vector is input into the inference layer of the student model, the known fact relationship of the knowledge graph is retrieved, and the structured feature vector corresponding to each candidate entity is extracted; the high-dimensional feature vector and the structured feature vector of each candidate entity are correlated to calculate the matching degree between them; and the quantization result of the matching degree is taken as the similarity score of the corresponding candidate entity.

[0103] Then the similarity score is input into the output layer of the student model based on the similarity score, and a Softmax function is used to calculate the prediction probability of all candidate entities in the knowledge graph. It should be noted that the output layer is the end operation structure of the student model, and its core function is to normalize the similarity score output by the inference layer, map the score to the probability interval, and output the prediction result with probability significance, providing an intuitive quantization basis for subsequent candidate entity screening. In addition, the Softmax function is a commonly used normalization function, which can convert a set of arbitrary real scores into probability values between 0 and 1, and the sum of all probability values is 1. This function can amplify the probability proportion of high-score candidate entities while compressing the probability proportion of low-score candidate entities, and is suitable for probability calculation in multi-classification scenarios. In addition, the prediction probability is a value obtained by processing the similarity score with the Softmax function in the output layer, representing the probability of the matching of the corresponding candidate entity and the query intent. The closer the value is to 1, the higher the matching probability, which is the core basis for screening the initial candidate entity set. It can be understood that the similarity scores corresponding to all candidate entities output by the inference layer of the student model are obtained, and then the similarity scores are input into the output layer of the student model. The output layer uses the Softmax function to normalize the similarity scores, and then maps the similarity score of each candidate entity to the probability interval of 0 to 1 to obtain the prediction probability of all candidate entities in the knowledge graph.

[0104] Then the candidate entities are sorted according to the preset rule, and the top pre-set number of entities are selected to form an initial candidate entity set. It should be noted that the preset rule is a pre-defined candidate entity sorting basis. In this embodiment, the rule takes the predicted probability corresponding to the candidate entity as the core sorting index, and arranges the candidate entities in order from high to low according to the predicted probability. In addition, the candidate entity is a set of entities in the knowledge graph that have a potential matching relationship with the query intent. The entities in this set all come from the known entity library of the knowledge graph and are the screening source of the initial candidate entity set. In addition, the pre-set number is a pre-set entity size threshold of the initial candidate entity set. This threshold is determined according to factors such as the total number of entities in the knowledge graph and the query accuracy requirement, and can balance the coverage range and screening efficiency of the candidate entity. In addition, the initial candidate entity set is an entity list obtained by screening according to the preset rule. The entities in the list have a higher predicted probability and are the core objects for further verification and optimization. It can be understood that all candidate entities in the knowledge graph and their corresponding predicted probabilities are obtained, and then all candidate entities are sorted in order from high to low according to the preset rule, i.e. the predicted probability. Then the top pre-set number of candidate entities are selected, and these entities are integrated to form the initial candidate entity set.

[0105] Finally, the predicted probability corresponding to each entity in the initial candidate entity set is extracted, and the predicted probability is taken as the predicted score of the corresponding entity. It should be noted that the predicted probability is a value obtained by normalizing the similarity score using the Softmax function in the output layer of the student model. It represents the probability of matching the corresponding candidate entity with the query intent. The closer the value is to 1, the higher the matching probability, and it has a clear quantitative reference significance. In addition, the predicted score is a quantitative index for intuitively measuring the matching degree of each entity in the initial candidate entity set with the query intent. Its value is completely consistent with the predicted probability, and it is a key basis for subsequent reasonable evaluation and screening of candidate entities.

[0106] Step S50, the text prompt, the initial candidate entity set and the corresponding predicted score are constructed into reasoning instructions, and the reasoning instructions are sent to the large language model to make the large language model perform reasonable evaluation on the initial candidate entity set and feed back the newly added candidate entity and the corresponding confidence.

[0107] It should be noted that the reasoning instruction is a standardized instruction text formed by integrating the text prompt, the initial candidate entity set and the prediction score, has a clear task orientation and complete evaluation basis, and can guide the large language model to carry out targeted rationality evaluation. The new candidate entity is an unknown candidate entity that does not appear in the known entity set of the knowledge graph and is consistent with the semantic logic of the knowledge graph query request. It is an unknown candidate entity obtained by the large language model based on semantic logic reasoning. The confidence is a quantitative index assigned by the large language model for the new candidate entity, which is used to measure the matching degree of the new candidate entity and the query intent. The higher the value, the higher the semantic logic fit degree.

[0108] It can be understood that according to the preset instruction template, the text prompt, all entity information of the initial candidate entity set and the prediction score corresponding to each entity are integrated to generate a reasoning instruction with clear evaluation task requirements. The constructed reasoning instruction is input into the large language model to trigger the semantic reasoning and rationality evaluation functions of the large language model. The large language model understands the query intent based on the text prompt, analyzes the rationality of the initial candidate entity set in combination with the prediction score, and performs expansion reasoning based on semantic logic. The large language model outputs the evaluation result of the initial candidate entity set and feeds back the new candidate entity obtained by reasoning and the confidence corresponding to each new candidate entity.

[0109] In step S60, the initial candidate entity set, the prediction score, the new candidate entity and the confidence are merged to form a soft label distribution.

[0110] It should be noted that the soft label distribution is a probability distribution set formed by integrating the known candidate entity, the unknown candidate entity and the corresponding weight, which is different from the hard label of this or that. It contains the matching weight information of different entities, which can provide more delicate reference basis for model training or query result optimization.

[0111] It can be understood that each entity in the initial candidate entity set is associated with its corresponding prediction score to form a known entity-score key-value pair, and each entity in the new candidate entity is associated with its corresponding confidence to form a new entity-confidence key-value pair. Then, the two types of key-value pairs are merged and integrated into a set containing all candidate entities and their corresponding weights. This set is the soft label distribution.

[0112] In step S70, the student model is jointly optimized based on the soft label distribution and the preset loss function to obtain an optimized knowledge graph distillation model.

[0113] It should be noted that the preset loss function is a predefined quantitative function for measuring the difference between the prediction result of the student model and the soft label distribution, and is the core criterion for driving model parameter updating. The knowledge graph distillation model is a student model with better reasoning ability after soft label supervision and loss optimization. It retains the efficient characteristics of the student model, while absorbing the semantic generalization ability of the large language model, and adapts to the deployment needs of knowledge graph reasoning.

[0114] It can be understood that the soft label distribution is preprocessed first, and the known and newly added candidate entities and corresponding weights are integrated to form a standard supervision signal after normalization and weight balancing; the fused query representation is input into the student model to obtain the prediction probability distribution, and the loss value of the distribution and the soft label distribution is calculated through the preset loss function; based on the loss value, the parameters of each layer of the model are updated through back propagation, and the iterative training is performed until the loss converges; finally, the model is lightened to obtain the optimized knowledge graph distillation model.

[0115] Further, after step S70: reasoning and prediction are performed using the optimized knowledge graph distillation model. After the model receives the query input, the complete process of feature processing, model operation and output of matching results is adapted to multiple application scenarios such as intelligent question answering, knowledge retrieval, semantic analysis, etc., and the core goal is to quickly output accurate entity matching results. Taking the intelligent question answering application scenario as an example, the process is as follows: first, the user's natural language question, i.e. the natural language form of the knowledge graph query request, is received, and the subject entity and predicate relationship are extracted and converted into a structured query through semantic analysis. Then the semantic encoding module converts the subject entity and predicate relationship into semantic entity embedding and semantic relationship embedding, and the structured embedding generation module generates structured embedding according to the structured query. The embedding fusion module unifies the dimensions of the three types of embeddings and dynamically weights and fuses them to obtain the fused query representation, which is then input into the optimized knowledge graph distillation model. After nonlinear transformation by the embedding layer, correlation calculation by the reasoning layer, and probability conversion by the output layer, a set of candidate entities and corresponding prediction scores are obtained, and the result output module sorts the results from high to low according to the prediction scores, selects the top preset number of entities as the answer, and feeds back to the user.

[0116] The optimized knowledge graph distillation model has fast reasoning speed and high accuracy, and is suitable for scenarios such as intelligent question answering that require high real-time performance. The query representation that integrates multiple source features ensures the matching degree of the results, improves the user experience of the application scenarios, and the modularized process design facilitates the adjustment of parameters according to different application requirements, such as increasing the number of filtered entities to ensure coverage in the knowledge retrieval scenario, enhancing the flexibility of the process.

[0117] The embodiment converts a knowledge graph query request into a text prompt and a structured query, generates semantic embeddings using a large language model, and fuses the semantic embeddings with structured embeddings. A student model predicts initial candidate entities. The large language model further evaluates and feeds back newly added candidate entities and their confidence, forms a soft label distribution for optimizing the student model, and obtains an optimized knowledge graph distillation model. The optimized knowledge graph distillation model compensates for the shortcomings of traditional methods in handling unknown entities, improves the accuracy and generalization ability of knowledge graph reasoning, and enhances the prediction ability of unknown entities, providing reliable support for intelligent decision-making in complex scenarios.

[0118] Based on the first embodiment of the present application, the same or similar contents as the above embodiment one can be referred to the above introduction, and will not be repeated hereinafter. On this basis, please refer to Figure 2 , the model distillation method for knowledge graph unknown entity reasoning step S50 further comprises steps S201-S208:

[0119] Step S201, extracting the core query elements in the text prompt, and associating and binding the initial candidate entity set and the corresponding prediction score to obtain the associated and bound information.

[0120] It should be noted that the core query elements are key information extracted from the text prompt that has a clear query direction. The core query elements include the subject entity, the relationship type, and the query gap. The subject entity is the main part of the core query elements, is an existing known entity in the knowledge graph, and is the starting point of the entire query requirement, which clearly defines the core object of the reasoning process. The relationship type is information used to describe the association attribute between the subject entity and the target entity in the core query elements, which defines the semantic association logic between the two. The query gap is the target information gap part to be supplemented in the core query elements, which is the core problem that needs to be solved in the entire query requirement, and clearly defines the final goal of the reasoning prediction, that is, to find an entity that can fill the gap. The initial candidate entity set is a high matching degree entity list predicted by the student model based on the fused query representation, which is derived from the known entity set of the knowledge graph, and is the target object for association and binding with the core query elements. In addition, the prediction score is a matching degree quantitative index corresponding to each entity in the initial candidate entity set, the numerical value is consistent with the prediction probability output by the student model, and is used to represent the matching weight of the candidate entity and the query intent, which is an important part of the associated and bound information. The associated and bound information is a structured information set formed by corresponding association of the core query elements, the initial candidate entity set and the corresponding prediction score, which integrates the query direction information and the candidate matching information.

[0121] It can be understood that, firstly, the text prompt is semantically parsed to extract subject entities, relationship types, and query gaps, three core query elements, and then an association mapping relationship between the core query elements and the initial candidate entity set is established, each candidate entity is corresponded to the complete core query element, and finally the prediction score corresponding to the candidate entity is synchronized and bound to the association relationship to form structured data containing core query elements, candidate entities, and prediction scores, that is, the information after association and binding is obtained.

[0122] In step S202, the reasoning instruction is constructed based on the information after association and binding according to a preset logic.

[0123] It should be noted that the preset logic is a predefined reasoning instruction construction rule, which clearly defines the composition structure, content arrangement order, and expression specification of the reasoning instruction. In this embodiment, the preset logic generally includes three core components, which are query requirement description, candidate entity display, and evaluation task requirement content. The three contents are combined in a fixed order to form a complete reasoning instruction. The query requirement description clearly describes the core query elements extracted from the text prompt, which specifically includes the subject entity, the relationship type, and the query gap. The expression form is a natural language declarative sentence. The candidate entity display presents the initial candidate entity set and the corresponding prediction score in a unified format, usually in the form of an "entity-score" list. The evaluation task instruction clearly informs the specific task that the large language model needs to perform. The reasoning instruction is a standardized text generated based on the information after association and binding and the preset logic, which includes query requirement description, candidate entity display, and evaluation task requirement content, and is an input carrier for driving the large language model to carry out rationality evaluation.

[0124] In step S203, the reasoning instruction is subjected to syntax normalization processing to obtain a normalized reasoning instruction.

[0125] It should be noted that the grammar normalization processing is a text optimization operation on the constructed reasoning instruction, and its core purpose is to correct grammatical errors in the instruction, adjust the order of the sentence, unify the expression format, ensure that the instruction language is smooth and logical, and enable the large language model to unambiguously understand the evaluation task and related information. In this embodiment, the processing mainly develops around three dimensions of sentence fluency, expression consistency, and information integrity. The sentence fluency optimization is to correct grammatical problems such as incomplete components, improper collocation, and chaotic order in the instruction, convert colloquial expressions into written and standardized expressions, and avoid language ambiguity affecting the understanding and judgment of the large language model. The expression consistency optimization is to unify the expression of the same concept in the instruction, ensure the consistency of the name of the key information such as the candidate entity prediction score, and unify the format specification such as the list punctuation, thereby improving the readability of the instruction. The information integrity optimization is to check whether the instruction contains the three core modules of query requirement description, candidate entity display, and evaluation task requirement, and supplement the missing key information.

[0126] It can be understood that first, the constructed reasoning instruction is called, grammatical errors are corrected according to the requirement of sentence fluency, and colloquial expressions are converted into written expressions, then the name of the key information and the format specification are unified according to the requirement of expression consistency, and finally it is checked whether the core modules of the instruction are complete, and the missing information is supplemented, to obtain the normalized reasoning instruction.

[0127] In step S204, the normalized reasoning instruction is sent to the large language model through a preset data transmission channel to make the large language model feedback a response result.

[0128] It should be noted that the preset data transmission channel is a communication link pre-configured for data interaction, has stable, reliable and safe transmission characteristics, and is specially used for transmitting instructions and data between different functional modules, and is a key carrier for ensuring that the normalized reasoning instruction is accurately delivered to the large language model. In this embodiment, the channel supports efficient transmission of text format data, and can ensure that the instruction does not lose or tamper information during transmission. The normalized reasoning instruction is a standardized text after grammar normalization processing. The large language model is an artificial intelligence model with strong semantic understanding and logical reasoning ability, which can complete complex text analysis and reasoning tasks based on the input instruction, and its core function is to analyze the normalized reasoning instruction, perform the corresponding evaluation operation and generate a response result. It can be understood that first, the preset data transmission channel is started, and it is confirmed that the channel is in a usable state, then the normalized reasoning instruction is packaged in the format required by the channel, and finally the packaged reasoning instruction is sent to the large language model through the channel, triggering the evaluation process of the large language model, and waiting for the large language model to feedback the response result.

[0129] Step S205, separating the initial candidate entity rationality evaluation list and the initial new candidate entity set from the response result.

[0130] It should be noted that the response result is the feedback data output by the large language model based on the normalized reasoning instruction, including the rationality judgment information of the initial candidate entity, the new candidate entity information and the corresponding confidence. In this embodiment, the response result is presented in the form of structured text. In addition, the initial candidate entity rationality evaluation list is a judgment list recording whether each initial candidate entity meets the query semantic logic, which contains the initial candidate entity name and the corresponding rationality judgment result. It can be understood that the response result output by the large language model is first called, the boundary identifier for distinguishing different information modules in the result is identified, then the initial candidate entity rationality judgment information divided by the boundary identifier is extracted, and the initial candidate entity rationality evaluation list is formed by arranging the entity name and the judgment result one by one. Finally, the new candidate entity and the corresponding confidence information in the response result are extracted and integrated into the initial new candidate entity set, and the separation of the two types of information is completed.

[0131] Step S206, inputting the initial new candidate entity set returned by the large language model into the student model to obtain the corresponding prediction score, and feeding back the prediction score to the large language model to trigger the second round of reasoning update to obtain the updated new candidate entity set.

[0132] It should be noted that the initial new candidate entity set is a list of candidate entities that do not appear in the known entity set of the knowledge graph obtained by the large language model based on semantic logic reasoning. In this embodiment, this set is an expansion and supplement to the initial candidate entity set, providing a new reference sample for the iterative optimization of the model. The prediction score is a quantitative index output by the student model after feature extraction and correlation calculation on the input initial new candidate entity set, which is used to measure the matching degree of the new candidate entity and the query intent. The second round of reasoning update is a secondary optimization process of the initial new candidate entity set by the large language model after receiving the prediction score feedback from the student model, combining its own semantic reasoning ability. The purpose is to filter out new entities with low matching degree and improve the accuracy of new candidate entities. The updated new candidate entity set is an optimized version of the new entity list output by the large language model after the second round of reasoning update. The entities in the set have both semantic logic rationality and model prediction matching degree, and are an important part of building a soft label distribution. It can be understood that the initial new candidate entity set is first input into the optimized knowledge graph distillation model, and the prediction score corresponding to each entity is obtained through the embedding layer, reasoning layer and output layer operation. Then the prediction score is fed back to the large language model, and then the large language model combines the prediction score to perform secondary evaluation and screening on the initial new candidate entity set, completes the second round of reasoning update, and obtains the updated new candidate entity set.

[0133] Through the bidirectional interaction iteration of the student model and the large language model, the accurate screening of the new candidate entity is realized, which not only ensures the semantic logic rationality of the new entity, but also improves the matching degree of the new entity with the query intention. The mechanism of bidirectional feedback makes the advantages of the two types of models complementary, providing reliable data support for the subsequent construction of high-quality soft label distribution, and further improving the optimization effect of the model.

[0134] In step S207, the rationality evaluation list is parsed to obtain the screened initial candidate entity.

[0135] It should be noted that the screened initial candidate entity is obtained by removing the entities that do not conform to the query semantic logic from the initial candidate entity set. The entities in this list have both high prediction scores and semantic logic rationality. It can be understood that first, the rationality evaluation list of the initial candidate entity is called, and the rationality judgment result corresponding to each initial candidate entity in the list is checked one by one. Then, the entities with a judgment result that conforms to the semantic logic are retained, and the entities with a judgment result that does not conform to the semantic logic are removed. Finally, the retained entities are integrated to obtain the screened initial candidate entity.

[0136] In step S208, the effectiveness of the confidence in the updated new candidate entity set is verified to obtain the new candidate entity and the corresponding confidence.

[0137] It should be noted that the effectiveness verification is a rationality verification operation carried out on the confidence in the updated new candidate entity set. The core is to filter out the reliable confidence and the corresponding entity through the preset standard, and remove the low-quality data that does not meet the requirements, so as to avoid the influence of unreliable information on the construction quality of the subsequent soft label distribution. The new candidate entity is an entity that is retained after effectiveness verification and does not appear in the known entity set of the knowledge graph. The corresponding confidence is a quantitative index that is confirmed to be reliable after effectiveness verification, and corresponds to the new candidate entity one by one.

[0138] It can be understood that first, the updated new candidate entity set is called, and the confidence value corresponding to each entity is determined. Then, according to the application scenario, the confidence threshold is set. Specifically, the threshold value of the high-risk scenario is usually set to 0.85 to 0.95, and the threshold value of the low-risk scenario can be set to 0.6 to 0.75. Then, the entities and the corresponding confidence whose confidence is higher than the set threshold are retained, and the entities whose confidence is lower than the threshold are removed to obtain the new candidate entity and the corresponding confidence.

[0139] Through structured processing and bidirectional iteration, the candidate entity has both quantitative matching degree and semantic rationality, which improves the quality of soft label distribution, provides accurate supervision signal for student model optimization, and enhances the comprehensiveness and reliability of knowledge graph reasoning.

[0140] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as the above embodiment one can be referred to the above introduction, and the subsequent will not be described. On this basis, please refer to Figure 3 , the model distillation method for knowledge graph unknown entity reasoning step S70 further comprises steps S301-S306:

[0141] Step S301, determine the composition of the preset loss function.

[0142] It should be noted that the preset loss function is a quantitative function for measuring the difference between the prediction results of the student model and the supervision signal, and the preset loss function includes distillation loss, structure loss and semantic matching loss. The distillation loss is a loss term for measuring the difference between the prediction probability distribution of the student model and the soft label distribution. In addition, the structure loss is a loss term for measuring the fitting degree between the entity structured embedding output by the student model and the known fact relationship of the knowledge graph. In addition, the semantic matching loss is a loss term for measuring the matching degree between the entity semantic embedding output by the student model and the query intent semantic feature.

[0143] Step S302, calculate the distillation loss between the current prediction result of the student model and the soft label distribution by Huber loss function.

[0144] It should be noted that Huber loss function is a segmented loss function, and its core is to distinguish the calculation method of different error intervals by a preset error threshold. The preset error threshold is the core hyperparameter of Huber loss function, which is used to divide different calculation intervals, and the preset error threshold is usually set to 1 according to the weight range of soft label distribution, which can be fine-tuned according to actual optimization requirements. The weight of soft label distribution is the normalized weight corresponding to each candidate entity in the soft label distribution.

[0145] It can be understood that first, the error threshold of Huber loss function is set to 1 , and then the error value of the prediction probability of the student model and the soft label weight corresponding to the prediction probability is calculated , when , the loss value is calculated in the form of square:

[0146]

[0147] When , the loss value is calculated in the form of linearity:

[0148]

[0149] The average value of the loss values of all candidate entities is taken to obtain the final distillation loss, and the formula is:

[0150]

[0151] wherein, denotes the total number of candidate entities, denotes the current number of candidate entities. The segmentation calculation formula can effectively reduce the interference of outliers on the distillation loss and avoid overfitting of the model to low-confidence entities.

[0152] In step S303, a binary cross-entropy loss function is used to calculate the structure loss of the student model based on the knowledge graph structured knowledge reasoning.

[0153] It should be noted that the binary cross-entropy loss function is a loss function used to measure the difference between the predicted probability and the true label in a binary classification task. The structure loss is one of the sub-loss items of the preset loss function, which functions to constrain the reasoning process of the student model to conform to the inherent structural logic of the knowledge graph. The predicted probability is the entity association probability output by the student model based on the knowledge graph structured embedding, and the value range is 0 to 1. The closer the value is to 1, the more the entity association relationship conforms to the structural constraints of the knowledge graph. The true label is a binary label generated based on the known facts of the knowledge graph. When the entity association relationship predicted by the student model exists in the knowledge graph, the label value is 1, and when it does not exist, the label value is 0, which is a supervision standard for measuring the accuracy of the structured reasoning result.

[0154] It can be understood that first, the calculation object of the binary cross-entropy loss function is the structured reasoning probability output by the student model and the corresponding structure true label Then, for the reasoning result of each entity association pair, the single-sample loss is calculated using the basic formula of the binary cross-entropy loss:

[0155]

[0156] wherein, denotes the single-sample loss, denotes the logarithmic function. Finally, the single-sample losses of all entity association pairs participating in the calculation are averaged to obtain the final structure loss, and the calculation formula is:

[0157]

[0158] wherein, denotes the total number of entity association pairs participating in the structured reasoning verification, denotes the current entity association pair.

[0159] In step S304, the semantic matching loss between the semantic entity embedding, the semantic relationship embedding and the initial embedding of the student model is calculated by the cosine similarity.

[0160] It should be noted that the cosine similarity is a quantitative index for measuring the degree of similarity between two vectors, and the value range is -1 to 1. The closer the value is to 1, the more consistent the direction of the two vectors, and the closer the value is to -1, the more opposite the direction. The semantic matching loss is one of the sub-loss items of the preset loss function, which functions to constrain the embedding space of the student model to be consistent with the semantic feature space, so that the embedding generated by the model can accurately represent the semantic information of entities and relationships. In addition, the semantic entity embedding is a low-dimensional dense vector converted from the subject entity in the query, which contains the deep semantic features of the entity and is one of the core input vectors for semantic matching. The semantic relationship embedding is a low-dimensional dense vector converted from the relationship type in the query, which contains the deep semantic features of the relationship and is one of the core input vectors for semantic matching. The student model initial embedding is a basic embedding vector generated by the student model based on the pre-training of the knowledge graph, which contains the initial entity embedding and the initial relationship embedding in the knowledge graph.

[0161] It can be understood that the cosine similarity between the semantic entity embedding and the student model initial entity embedding is first calculated respectively , the cosine similarity between the semantic relationship embedding and the student model initial relationship embedding . Then based on the value characteristics of the cosine similarity, the semantic matching loss is defined as the complementary form of the similarity, and the matching loss formula of the single-class embedding is:

[0162]

[0163] Finally, the weight coefficients and are assigned to the two types of embedding losses, and , and the final semantic matching loss calculation formula is:

[0164] .

[0165] Step S305, introducing a preset weighting coefficient to weight and sum the distillation loss, the structure loss and the semantic matching loss to obtain a total loss function.

[0166] It should be noted that the preset weighting coefficient is a weight parameter preset to balance the contribution proportion of the distillation loss, the structure loss and the semantic matching loss in the total loss function. The value range of each coefficient is 0 to 1, and the sum of all coefficients is 1. Weighted summation is a calculation method of multiplying each sub-loss item by the corresponding weighting coefficient and then adding them up, which is a common method for constructing a multi-objective optimization total loss function, and can allow the model to consider multiple performance optimization objectives at the same time.

[0167] It can be understood that first, the corresponding preset weighting coefficients are set for the distillation loss, the structure loss and the semantic matching loss, denoted as , which satisfies Then, each loss term is multiplied by the corresponding weighting coefficient, and finally the results of the above three items are accumulated to obtain the total loss function, and the calculation formula is:

[0168]

[0169] Through the flexible configuration of the preset weighting coefficient, the core performance dimension of the model can be optimized according to the needs of different knowledge graph reasoning scenarios, such as increasing the weight in scenarios with high structured requirements, and increasing the weight in scenarios with high semantic generalization requirements.

[0170] Step S306, the Adam optimizer is used to minimize the total loss function, and the parameters of the student model are updated iteratively according to the preset learning rate until the maximum number of iterations is reached, and the optimized knowledge graph distillation model is obtained.

[0171] It should be noted that the Adam optimizer is an optimization algorithm that combines momentum gradient descent and adaptive learning rate, which can adaptively adjust the learning rate of each parameter, has fast convergence speed and strong stability, and is suitable for parameter optimization of the knowledge graph distillation model. The total loss function is the final loss function formed after integrating the distillation loss, the structure loss and the semantic matching loss, and is a quantitative function that measures the comprehensive deviation between the prediction results of the student model and the supervision target. The preset learning rate is a hyperparameter that controls the step size of parameter update, which is used to adjust the convergence speed of the model, and the value needs to be pre-set according to the model training requirements. The parameters of the student model are various adjustable variables in the student model used to realize reasoning calculation, including weights and biases of different modules such as embedding layer, reasoning layer and output layer, etc. The maximum number of iterations is the termination condition of model training, which is the upper limit of the number of parameter updates pre-set, and needs to be determined according to the size of the data set and the complexity of the model.

[0172] It can be understood that first, the related parameters of the Adam optimizer are initialized, including the first moment estimation, the second moment estimation, the numerical stability term and the preset learning rate, etc., and then the iteration process is entered. In each iteration, the gradient of the total loss function with respect to the parameters of the student model is calculated, the first moment and the second moment are updated based on the gradient, and the moment estimation is corrected, then the parameters of the student model are updated according to the corrected moment estimation and the preset learning rate, and the above iteration process is repeated until the number of iterations reaches the preset maximum number of iterations, and the update is stopped. At this time, the student model obtained is the optimized knowledge graph distillation model.

[0173] The embodiment first determines a preset loss function containing distillation, structure, and semantic matching loss; calculates the three types of loss by using Huber loss, binary cross-entropy loss, and cosine similarity, respectively, and then introduces a preset weighting coefficient to perform weighted summation to obtain a total loss function; finally, the student model parameters are iteratively updated by using an Adam optimizer with a preset learning rate until a maximum number of iterations is reached, and an optimized knowledge graph distillation model is obtained, thereby finally improving the comprehensive reasoning ability and generalization of the knowledge graph distillation model.

[0174] Based on the first embodiment of the present application, the present application also provides a model distillation device for knowledge graph unknown entity reasoning, please refer to Figure 4 The device comprises: a conversion module 10 configured to receive a knowledge graph query request and convert the knowledge graph query request into a corresponding text prompt and a structured query. An encoding module 20 is configured to call a large language model to perform semantic encoding on the subject entity and the predicate relationship in the text prompt, and generate semantic entity embedding and semantic relationship embedding. A fusion module 30 is configured to fuse the semantic entity embedding and the semantic relationship embedding with the structured embedding in the structured query to obtain a fused query representation. A prediction module 40 is configured to predict the fused query representation by using a student model to obtain an initial candidate entity set and a corresponding prediction score. A receiving module 50 is configured to construct a reasoning instruction from the text prompt, the initial candidate entity set, and the corresponding prediction score, and send the reasoning instruction to the large language model, so that the large language model evaluates the rationality of the initial candidate entity set and feeds back new candidate entities and corresponding confidence, wherein the new candidate entities are unknown candidate entities that do not appear in the known entity set of the knowledge graph and are consistent with the semantic logic of the knowledge graph query request. A merging module 60 is configured to merge the initial candidate entity set, the prediction score, the new candidate entity, and the confidence to form a soft label distribution. An optimization module 70 is configured to jointly optimize the student model based on the soft label distribution and a preset loss function to obtain an optimized knowledge graph distillation model.

[0175] The model distillation device for knowledge graph unknown entity reasoning provided by the present application adopts the model distillation method for knowledge graph unknown entity reasoning in the above embodiments, and can solve the technical problem of how to improve the reasoning and prediction ability of the model for knowledge graph unknown entities while preserving semantic information.

[0176] The present application provides a model distillation device for knowledge graph unknown entity reasoning, which comprises at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the model distillation method for knowledge graph unknown entity reasoning in the above embodiment one.

[0177] Reference will now be made to Figure 5 FIG. 1 illustrates a structural diagram of a model distillation apparatus for knowledge graph unknown entity reasoning, according to an embodiment of the present application. The model distillation apparatus for knowledge graph unknown entity reasoning according to an embodiment of the present application can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook, a digital broadcasting receiver, a PDA (Personal Digital Assistant), a PAD (Portable Application Description), a PMP (Portable Media Player), a car terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. Figure 5 The model distillation apparatus for knowledge graph unknown entity reasoning illustrated is merely one example, and should not bring about any limitation on the function and use range of the embodiments of the present application.

[0178] As illustrated in Figure 5 The model distillation apparatus for knowledge graph unknown entity reasoning can include a processing device 1001 (e.g., a central processing unit, a graphic processing unit, or the like) that can perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the model distillation apparatus for knowledge graph unknown entity reasoning are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An I / O (Input / Output) interface 1006 is also connected to the bus. In general, the following can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, and the like; an output device 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, and the like; the storage device 1003 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 1009. The communication device 1009 can allow the model distillation apparatus for knowledge graph unknown entity reasoning to communicate with other devices wirelessly or by wire to exchange data. Although a model distillation apparatus for knowledge graph unknown entity reasoning having various components is illustrated in the drawing, it should be understood that all of the illustrated components are not required to be implemented or possessed. More or less components can be alternatively implemented or possessed.

[0179] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present application are performed.

[0180] The present application provides a computer readable medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the model distillation method for knowledge graph unknown entity reasoning in the above embodiments. The computer readable medium provided by the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor, or device, or any combination of the above. The above computer readable medium can be included in the model distillation device for knowledge graph unknown entity reasoning; or can exist separately and not be assembled into the model distillation device for knowledge graph unknown entity reasoning.

[0181] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A model distillation method for knowledge graph unknown entity reasoning, the method is applied to an intelligent question and answer application scenario, characterized in that, The method comprises: receiving a knowledge graph query request, converting the knowledge graph query request into a corresponding text prompt and a structured query; calling a large language model to perform semantic encoding on the subject entity and the predicate relationship in the text prompt, to generate semantic entity embedding and semantic relationship embedding; fusing the semantic entity embedding and the semantic relationship embedding with the structured embedding in the structured query to obtain a fused query representation; using a student model to predict the fused query representation to obtain an initial candidate entity set and corresponding prediction scores; constructing the text prompt, the initial candidate entity set and the corresponding prediction scores into reasoning instructions, and sending the reasoning instructions to the large language model to enable the large language model to perform rationality evaluation on the initial candidate entity set and feed back new candidate entities and corresponding confidence, wherein the new candidate entities are unknown candidate entities that do not appear in the known entity set of the knowledge graph and are consistent with the semantic logic of the knowledge graph query request; merging the initial candidate entity set, the prediction scores, the new candidate entities and the confidence to form a soft label distribution; optimizing the student model based on the soft label distribution and a preset loss function to obtain an optimized knowledge graph distillation model; After the step of optimizing the student model based on the soft label distribution and the preset loss function to obtain the optimized knowledge graph distillation model, the method comprises: inputting the fused query representation into the optimized knowledge graph distillation model, performing nonlinear transformation through an embedding layer, correlation calculation through an inference layer, and probability conversion through an output layer to obtain a candidate entity set and corresponding prediction scores, sorting the candidate entity set according to the prediction scores from high to low and selecting the top preset number of entities as answers to feed back to the user; The step of merging the initial candidate entity set, the prediction scores, the new candidate entities and the confidence to form a soft label distribution comprises: associating each entity in the initial candidate entity set with its corresponding prediction score to form a known entity-score key-value pair; associating each entity in the new candidate entity set with its corresponding confidence to form a new entity-confidence key-value pair; merge the known entity-score key-value pair and the new entity-confidence key-value pair into a set containing all candidate entities and their corresponding weights, which is a soft label distribution.

2. The method of claim 1, wherein, The step of receiving a knowledge graph query request and converting the knowledge graph query request into a corresponding text prompt and a structured query comprises: parsing the knowledge graph query request to extract the subject entity and the predicate relationship; constructing a first structured query according to the subject entity and the predicate relationship, wherein the first structured query is in the form of a triple; if there is a timestamp in the knowledge graph query request, constructing a second structured query, wherein the second structured query is in the form of a quadruple including the timestamp; organizing the subject entity, the predicate relationship and or the timestamp according to the natural language order to form a text prompt.

3. The method of claim 1, wherein, The step of calling the large language model to perform semantic coding on the subject entity and the predicate relationship in the text prompt to generate semantic entity embedding and semantic relationship embedding includes: extracting a subject entity segment and a predicate relationship segment from the text prompt; packaging the subject entity segment and the predicate relationship segment into an entity coding request and a relationship coding request, respectively, and sending them to the large language model for processing to enable the large language model to feed back an original entity coding vector and an original relationship coding vector; aligning the dimensions of the original entity coding vector and the original relationship coding vector through a preset projection matrix to obtain a dimension-aligned coding vector; normalizing the dimension-aligned coding vector to obtain semantic entity embedding and semantic relationship embedding.

4. The method of claim 1, wherein, The step of fusing the semantic entity embedding and the semantic relationship embedding with the structured embedding in the structured query to obtain a fused query representation includes: extracting a structured embedding corresponding to the structured query, wherein the structured embedding is generated based on a historical fact set of a knowledge graph; performing dimension-uniform processing on the semantic entity embedding, the semantic relationship embedding, and the structured embedding through a preset learnable projection matrix to obtain a first embedding, a second embedding, and a third embedding, wherein the first embedding corresponds to the semantic entity embedding, the second embedding corresponds to the semantic relationship embedding, and the third embedding corresponds to the structured embedding; calculating a first similarity and a second similarity based on the semantic entity embedding, the semantic relationship embedding, and the structured embedding, wherein the first similarity is a semantic structure similarity between the semantic entity embedding and the structured embedding, and the second similarity is a relationship structure similarity between the semantic relationship embedding and the structured embedding; assigning a dynamic weight according to the first similarity and the second similarity; performing element-level weighted fusion on the first embedding, the second embedding, and the third embedding according to the dynamic weight to obtain a fused query representation.

5. The method of claim 1, wherein, The step of using a student model to predict the fused query representation to obtain an initial candidate entity set and corresponding prediction scores includes: inputting the fused query representation into the embedding layer of the student model to generate a high-dimensional feature vector; inputting the high-dimensional feature vector into the reasoning layer of the student model to perform correlation calculation in combination with known fact relationships of the knowledge graph to obtain a similarity score; calculating prediction probabilities of all candidate entities in the knowledge graph through the output layer of the student model using a Softmax function based on the similarity score; sorting the candidate entities according to a preset rule to select the top pre-set number of entities to form an initial candidate entity set; extracting the prediction probability corresponding to each entity in the initial candidate entity set and taking the prediction probability as the prediction score of the corresponding entity.

6. The method of claim 1, wherein, The step of constructing the text prompt, the initial candidate entity set and the corresponding prediction score into inference instructions, and sending the inference instructions to the large language model, so that the large language model performs rationality evaluation on the initial candidate entity set and feeds back newly added candidate entities and corresponding confidence, comprises: extracting core query elements in the text prompt and associating and binding the initial candidate entity set and the corresponding prediction score to obtain associated and bound information, wherein the core query elements include subject entities, relationship types and query gaps; constructing inference instructions based on the associated and bound information according to a preset logic; performing syntax normalization processing on the inference instructions to obtain normalized inference instructions; sending the normalized inference instructions to the large language model through a preset data transmission channel to make the large language model feed back a response result; separating an initial candidate entity rationality evaluation list and an initial newly added candidate entity set from the response result; inputting the initial newly added candidate entity set returned by the large language model into a student model to obtain corresponding prediction scores, and feeding back the prediction scores to the large language model to trigger a second round of inference update to obtain an updated newly added candidate entity set; analyzing the rationality evaluation list to obtain screened initial candidate entities; performing effectiveness verification on the confidence in the updated newly added candidate entity set to obtain newly added candidate entities and corresponding confidence.

7. The method of claim 1, wherein, The step of jointly optimizing the student model based on the soft label distribution and a preset loss function to obtain an optimized knowledge graph distillation model comprises: determining the composition of the preset loss function, wherein the preset loss function comprises a distillation loss, a structure loss and a semantic matching loss; calculating the distillation loss between the current prediction result of the student model and the soft label distribution using a Huber loss function; calculating the structure loss of the student model based on knowledge graph structured knowledge reasoning using a binary cross-entropy loss function; calculating the semantic matching loss between the semantic entity embedding, the semantic relationship embedding and the initial embedding of the student model by cosine similarity; introducing a preset weighting coefficient to weight and sum the distillation loss, the structure loss and the semantic matching loss to obtain a total loss function; minimizing the total loss function using an Adam optimizer, iteratively updating the parameters of the student model at a preset learning rate until a maximum number of iterations is reached, to obtain an optimized knowledge graph distillation model.

8. An apparatus for model distillation for knowledge graph unknown entity reasoning, comprising: The device comprises: a conversion module configured to receive a knowledge graph query request and convert the knowledge graph query request into a corresponding text prompt and a structured query; an encoding module configured to call a large language model to perform semantic encoding on subject entities and predicate relationships in the text prompt to generate semantic entity embeddings and semantic relationship embeddings; a fusion module configured to fuse the semantic entity embeddings and the semantic relationship embeddings with structured embeddings in the structured query to obtain a fused query representation; The prediction module is configured to predict the fused query representation by using the student model to obtain an initial candidate entity set and corresponding prediction scores. The receiving module is configured to construct the text prompt, the initial candidate entity set and the corresponding prediction scores into reasoning instructions, and send the reasoning instructions to the large language model, so that the large language model performs rationality evaluation on the initial candidate entity set and feeds back new candidate entities and corresponding confidence, wherein the new candidate entities are unknown candidate entities that do not appear in the known entity set of the knowledge graph and are consistent with the semantic logic of the knowledge graph query request. The merging module is configured to merge the initial candidate entity set, the prediction scores, the new candidate entities and the confidence to form a soft label distribution; and is further configured to associate each entity in the initial candidate entity set with the corresponding prediction score to form a known entity-score key-value pair; associate each entity in the new candidate entity set with the corresponding confidence to form a new entity-confidence key-value pair; and merge the known entity-score key-value pair and the new entity-confidence key-value pair to integrate them into a set containing all candidate entities and corresponding weights, which is a soft label distribution. The optimization module is configured to jointly optimize the student model based on the soft label distribution and a preset loss function to obtain an optimized knowledge graph distillation model; and is further configured to input the fused query representation into the optimized knowledge graph distillation model, perform nonlinear transformation by an embedding layer, correlation calculation by an inference layer and probability conversion by an output layer, and obtain a candidate entity set and corresponding prediction scores, sort the candidate entity set in descending order of the prediction scores and select the top pre-set number of entities as answers to feed back to the user.

9. A model distillation device for knowledge graph unknown entity reasoning, comprising: The device comprises a memory, a processor and a model distillation program for knowledge graph unknown entity reasoning stored on the memory and running on the processor, and the model distillation program for knowledge graph unknown entity reasoning is configured to implement the steps of the model distillation method for knowledge graph unknown entity reasoning according to any one of claims 1-7.

10. A storage medium, characterized by The storage medium stores a model distillation program for knowledge graph unknown entity reasoning, and the model distillation program for knowledge graph unknown entity reasoning implements the steps of the model distillation method for knowledge graph unknown entity reasoning according to any one of claims 1-7 when executed by a processor.

Images