Response information generation method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a reply information generation method and device, equipment, a medium and a program product. A specific embodiment of the method comprises the steps of determining knowledge query information corresponding to a target question; querying associated knowledge related to the knowledge query information from the dynamic knowledge graph; generating first initial reply information according to the associated knowledge; in response to first evaluation information corresponding to the first initial reply information representing that reply quality evaluation is not passed, executing a generation step: according to the first evaluation information, generating reflection information corresponding to the first initial reply information; generating second initial reply information; and determining the second initial reply information as reply information corresponding to the target question in response to the second evaluation information representing that the reply quality evaluation is passed. The implementation mode is related to artificial intelligence, through knowledge retrieval of the dynamic knowledge graph and use of an error correction mechanism, the knowledge retrieval quality can be improved, and the reply content corresponding to the target question can be accurately generated.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of artificial intelligence, and in particular, to a reply information generation method, device, equipment, medium and program product. BACKGROUND

[0002] At present, with the continuous development of large models, intelligent question and answer systems are applied to more and more fields. For example, the demand for efficient and accurate fault diagnosis and maintenance consultation is increasing in the field of automobile after-sales service. The traditional mode is challenged by high labor costs, slow response speed, limited professional knowledge coverage and other problems. Intelligent question and answer systems can automatically solve common problems, reduce labor costs and provide 24-hour service, becoming an important solution. The existing technology often uses retrieval augmented generation (RAG, Retrieval-Augmented Generation) to combine information retrieval and large language models (LLM), which improves the generation of reply content by integrating external data sources (such as databases, documents, the Internet, etc.). However, traditional RAG relies on a fixed number of retrieval documents, and there are problems such as incomplete retrieval information, noise interference, and lack of verification mechanism. Especially in the field of automobile after-sales, its “static” nature makes it difficult to capture deep semantic associations, and the retrieval quality is limited, which cannot meet the high-precision demand. SUMMARY

[0003] The summary part of the present disclosure is used to introduce the concepts in a brief form, which will be described in detail in the specific embodiments part. The summary part of the present disclosure is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0004] Some embodiments of the present disclosure provide a reply information generation method, device, equipment, medium and program product to solve the technical problems mentioned in the background part.

[0005] In a first aspect, some embodiments of the present disclosure provide a reply information generation method, comprising: determining knowledge query information corresponding to a target question; querying associated knowledge related to the knowledge query information from a dynamic knowledge graph; generating first initial reply information according to the associated knowledge; in response to the first evaluation information corresponding to the first initial reply information indicating that the reply quality evaluation fails, performing the following steps: generating reflection information corresponding to the first initial reply information according to the first evaluation information; generating second initial reply information for the reflection information according to the dynamic knowledge graph; in response to the second evaluation information corresponding to the second initial reply information indicating that the reply quality evaluation passes, determining the second initial reply information as the reply information corresponding to the target question.

[0006] Optionally, the method further comprises: in response to the second evaluation information representing a failure in the reply quality evaluation, continuing to perform the generating step by taking the second initial reply information as the first initial reply information and taking the second evaluation information as the first evaluation information.

[0007] Optionally, the knowledge query information comprises entity information and entity relationship information, and the querying associated knowledge from the dynamic knowledge graph comprises: querying first context information related to the entity information and the entity relationship information from the dynamic knowledge graph by using a multi-hop retrieval manner; and generating the associated knowledge according to the first context information.

[0008] Optionally, the knowledge query information further comprises intention information, the dynamic knowledge graph further comprises case knowledge, and the generating the associated knowledge according to the first context information comprises: recalling case knowledge associated with the target question to a degree satisfying a target association condition as target case knowledge according to the intention information and the first context information; and generating the associated knowledge according to the first context information and the target case knowledge.

[0009] Optionally, the dynamic knowledge graph further comprises fault information in a multi-modal manner, and the generating the associated knowledge according to the first context information and the target case knowledge comprises: querying fault information related to the entity information and the entity relationship information from the dynamic knowledge graph as target fault information by using a multi-hop retrieval manner; and generating the associated knowledge according to the first context information, the target case knowledge and the target fault information.

[0010] Optionally, the method further comprises: packaging second context information referenced in the dynamic knowledge graph, corresponding case events, corresponding evaluation information and corresponding referenced article information corresponding to the reply information to obtain a packaged file; and displaying the reply information and the packaged file on a target page to support displaying each content in the packaged file on the target page.

[0011] Optionally, the generating the second initial reply information for the reflection information comprises: generating adjustment strategy information corresponding to the reflection information, wherein the adjustment strategy information comprises at least one of the following: strategy information for adjusting the knowledge query information, and strategy information for adjusting a query path corresponding to the associated knowledge; and regenerating the second initial reply information according to the adjustment strategy information.

[0012] Optionally, before the determining the knowledge query information corresponding to the target question, the method further includes: determining problem complexity information corresponding to the target question; in response to the problem complexity information satisfying a target complexity condition, performing single-reasoning reply to the target question to obtain third initial reply information; and generating reply information according to the third initial reply information.

[0013] Optionally, the generating the reply information according to the third initial reply information includes: in response to determining that the third initial reply information corresponds to third evaluation information representing failure in reply quality evaluation, continuing the generating step by taking the third initial reply information as first initial reply information and the third evaluation information as first evaluation information.

[0014] Optionally, the dynamic knowledge graph is updated by the following steps: in response to determining to perform update processing on the dynamic knowledge graph, obtaining external material information; extracting entity information, entity relationship information and event information corresponding to the external material information by using a pre-trained large language model; updating the dynamic knowledge graph according to the entity information, the entity relationship information and the event information to obtain an updated dynamic knowledge graph; and replacing the stored dynamic knowledge graph with the updated dynamic knowledge graph.

[0015] Optionally, the method further includes: obtaining user feedback information for the reply information; and optimizing at least one of a generation manner corresponding to the first initial reply information, an information query manner corresponding to the dynamic knowledge graph, and an adjustment manner of an adjustment strategy for the reflection information according to the user feedback information to obtain at least one optimization manner; and applying the at least one optimization manner online to perform subsequent reply.

[0016] In a second aspect, some embodiments of the present disclosure provide a reply information generation apparatus, including: a determination unit configured to determine knowledge query information corresponding to a target question; a query unit configured to query associated knowledge related to the knowledge query information from a dynamic knowledge graph; a generation unit configured to generate first initial reply information according to the associated knowledge; and an execution unit configured to, in response to first evaluation information corresponding to the first initial reply information representing failure in reply quality evaluation, perform a generation step of: generating reflection information corresponding to the first initial reply information according to the first evaluation information; generating second initial reply information for the reflection information; and in response to second evaluation information corresponding to the second initial reply information representing passing in reply quality evaluation, determining the second initial reply information as reply information corresponding to the target question.

[0017] Optionally, the apparatus further includes: in response to the second evaluation information indicating that the response quality assessment has not been passed, using the second initial response information as the first initial response information, using the second evaluation information as the first evaluation information, and continuing to perform the above generation steps.

[0018] Optionally, the aforementioned knowledge query information includes entity information and entity relationship information; and the query unit can be further configured to: use a multi-hop retrieval method to query first context information related to the aforementioned entity information and entity relationship information from the aforementioned dynamic knowledge graph; and generate the aforementioned associated knowledge based on the aforementioned first context information.

[0019] Optionally, the aforementioned knowledge query information further includes: intent information; the aforementioned dynamic knowledge graph further includes: case knowledge; and the query unit can be further configured to: based on the aforementioned intent information and the aforementioned first context information, recall case knowledge whose degree of association with the aforementioned target question meets the target association condition, as target case knowledge; and generate associated knowledge based on the aforementioned first context information and the aforementioned target case knowledge.

[0020] Optionally, the aforementioned dynamic knowledge graph also includes: fault information in a multimodal context; and the query unit can be further configured to: use a multi-hop retrieval method to query fault information related to the aforementioned entity information and the aforementioned entity relationship information from the aforementioned dynamic knowledge graph as target fault information; and generate the aforementioned associated knowledge based on the aforementioned first context information, the aforementioned target case knowledge and the target fault information.

[0021] Optionally, the device further includes: packaging the second context information, the corresponding case event, the corresponding evaluation information, and the corresponding cited article information in the dynamic knowledge graph corresponding to the above-mentioned response information to obtain a packaged file; and displaying the above-mentioned response information and the above-mentioned packaged file on a target page to support the display of various contents in the above-mentioned packaged file on the target page.

[0022] Optionally, the execution unit can be configured to: package the second context information, the corresponding case event, the corresponding evaluation information, and the corresponding cited article information in the dynamic knowledge graph corresponding to the above response information to obtain a packaged file; and display the above response information and the above packaged file on the target page to support the display of various contents in the above packaged file on the target page.

[0023] Optionally, the execution unit can be configured to: generate adjustment strategy information corresponding to the above-mentioned reflection information, wherein the adjustment strategy information includes at least one of the following: strategy information for adjusting knowledge query information, strategy information for adjusting the query path corresponding to related knowledge; and regenerate the second initial response information based on the above-mentioned adjustment strategy information.

[0024] Optionally, the apparatus further includes: determining the problem complexity information corresponding to the target problem; in response to the problem complexity information satisfying the target complexity condition, performing a single reasoning response for the target problem to obtain third initial response information; and generating response information based on the third initial response information.

[0025] Optionally, the apparatus further includes: in response to determining that the third initial response information corresponds to the third evaluation information indicating that the response quality assessment has not been passed, using the third initial response information as the first initial response information and the third evaluation information as the first evaluation information, and continuing to perform the above generation steps.

[0026] Optionally, the device further includes: acquiring user feedback information regarding the above-mentioned response information; optimizing at least one of the generation method corresponding to the first initial response information, the information query method corresponding to the dynamic knowledge graph, and the adjustment method of the adjustment strategy for the above-mentioned reflection information based on the user feedback information, to obtain at least one optimized method; and applying the at least one optimized method online for subsequent responses.

[0027] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0028] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0029] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0030] The above embodiments of this disclosure have the following beneficial effects: By using the knowledge retrieval and error correction mechanism of dynamic knowledge graphs through the response information generation method of some embodiments of this disclosure, the quality of knowledge retrieval can be improved and the response content corresponding to the target question can be accurately generated. Specifically, the reason why the relevant response content is not accurate enough is that traditional RAGs rely on a fixed number of retrieval documents, resulting in problems such as incomplete retrieval information, noise interference, and lack of verification mechanisms. Especially in the automotive aftermarket, its "static" characteristics make it difficult to capture deep semantic relationships, limiting retrieval quality and failing to meet high-precision requirements. Based on this, the response information generation method of some embodiments of this disclosure first determines the knowledge query information corresponding to the target question to facilitate high-quality knowledge retrieval later. Then, it queries the associated knowledge related to the above knowledge query information from the dynamic knowledge graph. Here, based on the real-time updating characteristic of the knowledge corresponding to the dynamic knowledge graph, the drawback of relying on a fixed number of retrieval documents can be avoided, and the problem of incomplete retrieval information can be solved. In addition, based on the knowledge query information, accurate querying from the dynamic knowledge graph can avoid the problem of noise interference. Then, based on the above associated knowledge, the first initial response information can be generated accurately. Next, in response to the first evaluation information representation corresponding to the first initial response information failing the response quality assessment, the following generation steps are executed: First, based on the first evaluation information, reflective information corresponding to the first initial response information is generated. Here, when it is determined that the evaluation result corresponding to the first initial response information does not meet the response quality requirements, reflective information is generated to achieve self-reflection and error correction of the corresponding defects in the response content, so as to facilitate continuous learning and evolution of the response content in the future. This allows for the generation of more accurate response information in the future. Second, based on the dynamic knowledge graph, a second initial response information is generated for the reflective information. Here, the dynamic knowledge graph is used again for further learning of related knowledge to effectively process the reflective information, resulting in a second initial response information with higher accuracy. Third, in response to the second evaluation information representation corresponding to the second initial response information passing the response quality assessment, the second initial response information is determined as the response information corresponding to the target question. In summary, based on the real-time updating characteristics of knowledge query information and dynamic knowledge graph, accurate querying and positioning of related knowledge can be achieved, providing sufficient material for the subsequent generation of response information. In addition, the reflection mechanism enables the relearning of knowledge and the re-analysis of responses. Through multiple rounds of iterative and dynamic strategy adjustments, high-quality responses are generated. This embedded self-reflection and error-correction capability significantly improves the robustness and accuracy of the question-and-answer system, allowing it to autonomously learn from mistakes and optimize its problem-solving process. Attached Figure Description

[0031] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0032] Figure 1 This is a schematic diagram illustrating an application scenario of a response information generation method according to some embodiments of the present disclosure; Figure 2 This is a flowchart of some embodiments of the response information generation method according to this disclosure; Figure 3 These are flowcharts of other embodiments of the response information generation method according to this disclosure; Figure 4 This diagram illustrates the modules of a question-and-answer system in the automotive aftermarket sector. Figure 5 These are schematic diagrams illustrating the structure of some embodiments of the response information generation apparatus according to this disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0033] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0034] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0035] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0036] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0037] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0038] Before performing any of the operations involving the collection, storage, or use of user personal information (such as target questions) as disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0039] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] Figure 1 This is a schematic diagram illustrating an application scenario of a response information generation method according to some embodiments of the present disclosure.

[0041] exist Figure 1 In this application scenario, firstly, the electronic device 101 can determine the knowledge query information 103 corresponding to the target question 102. In this application scenario, the target question 102 could be "Why do cars of brand A always malfunction?" The knowledge query information 103 could be "brand A", "car engine", "malfunction". Then, the electronic device 101 can query the related knowledge 105 from the dynamic knowledge graph 104. In this application scenario, the related knowledge 105 could be "20% battery depletion, 10% fuel pump failure, 5% crankshaft seizure", "potential high-temperature malfunction". Next, the electronic device 101 can generate a first initial response information 106 based on the related knowledge 105. In this application scenario, the first initial response information 106 could be "Frequent malfunctions due to high temperatures". Furthermore, in response to the first evaluation information characterizing the first initial response information 106 failing the response quality assessment, the electronic device 101 can execute the generation steps: First, based on the first evaluation information, generate reflection information 107 corresponding to the first initial response information 106. In this application scenario, the reflection information 107 could be "The temperature in the past few months has not been very high, which conflicts with the characteristic of high temperature." The second step is to generate a second initial response information 108 for the reflection information 107 based on the dynamic knowledge graph 104. In this application scenario, the second initial response information 108 could be "Region A often experiences frequent malfunctions due to high temperatures, while region B has a serious problem with battery depletion." The third step is to determine the second initial response information 108 as the response information corresponding to the second evaluation information representation through response quality evaluation, thus confirming it as the response information for the target problem.

[0042] It should be noted that the aforementioned electronic device 101 can be either hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0043] It should be understood that Figure 1 The number of electronic devices shown is merely illustrative. Any number of electronic devices can be used depending on the implementation requirements.

[0044] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a response information generation method according to the present disclosure. This response information generation method includes the following steps: Step 201: Determine the knowledge query information corresponding to the target question.

[0045] In some embodiments, the entity executing the above-described response information generation method (e.g.) Figure 1 The electronic device 101 shown can determine the knowledge query information corresponding to the target question. The target question can be a question to be answered. In practice, for automotive after-sales Q&A scenarios, the target question can be a question related to automotive after-sales service. For example, the target question could be "What causes the vehicle mileage to regress?". In practice, the target question can be a question input by a relevant technician received at the user interaction layer. In practice, the user interaction layer supports multimodal question input, and the user interaction layer automatically converts multimodal questions into text-based questions to facilitate subsequent knowledge query and question answering. The knowledge query information can be query content used to retrieve related knowledge. That is, for the knowledge query information, knowledge with contextual relationships is retrieved. In practice, the knowledge query information can be keywords in the target question. Contextual knowledge can be queried based on the keywords in the target question to obtain related knowledge.

[0046] As an example, the aforementioned executing entity can use keyword extraction methods to extract keywords from the target question and use them as knowledge query information.

[0047] In some optional implementations of certain embodiments, before step 201, the steps further include: The first step is to determine the problem complexity information corresponding to the target problem. Problem complexity information characterizes the degree of complexity of the content corresponding to the target problem. In practice, problem complexity information can be in numerical form or in label form. When it is in numerical form, the larger the value, the more complex the content of the target problem.

[0048] As an example, the aforementioned implementing entity can utilize a large language model to determine the problem complexity information corresponding to the target problem.

[0049] The second step involves responding to the aforementioned problem complexity information satisfying the target complexity condition by performing a single reasoning response to the target problem, thus obtaining the third initial response information. The target complexity condition can be that the problem complexity information represents the target problem as a simple problem. In practice, the target complexity condition can be that the value corresponding to the problem complexity information is less than a predetermined value, or it can be that the label content corresponding to the problem complexity information is "simple problem".

[0050] Here, a single-reason response can be an execution of a single reasoning step to answer a question.

[0051] The third step is to generate response information based on the initial response information mentioned above.

[0052] As an example, the aforementioned implementing entity can directly identify the third initial response information as the response information.

[0053] Optionally, the aforementioned implementing entity may generate response information based on the aforementioned third initial response information, including the following steps: In response to the determination that the third initial response information corresponds to the third evaluation information indicating that the response quality assessment has not been passed, the implementing entity may use the third initial response information as the first initial response information and the third evaluation information as the first evaluation information, and continue to perform the above generation steps.

[0054] Step 202: Query related knowledge from the dynamic knowledge graph that is related to the knowledge query information mentioned above.

[0055] In some embodiments, the aforementioned executing entity can query related knowledge from a dynamic knowledge graph that is relevant to the knowledge query information. The dynamic knowledge graph can be a graph whose content is updated in real time. For example, real-time cases can be searched externally using internet communication technology to extract knowledge from these cases and add it to the dynamic knowledge graph, thereby achieving real-time updates of the knowledge graph's content. Relevant knowledge can be knowledge that has a contextual relationship with the knowledge query information.

[0056] As an example, the aforementioned executing entity can utilize the hybrid retrieval-inference framework (RAG-KG) to query related knowledge from the dynamic knowledge graph that is relevant to the knowledge query information.

[0057] In some optional implementations of certain embodiments, the dynamic knowledge graph described above is updated through the following steps: The first step, in response to the determination to update the dynamic knowledge graph, involves acquiring external data information. Updating the dynamic knowledge graph can involve updating the content within the graph. In practice, this update process can be triggered periodically or manually by the manipulated entity. External data information can be unstructured text data obtained from external sources. In the automotive aftermarket, this external data information can include updated repair manuals, fault reports, technician experience sharing, online forum Q&A, and other similar materials.

[0058] The second step involves using a pre-trained large language model to extract entity information, entity relationship information, and event information corresponding to the aforementioned external data. Entity information can be entities mentioned in the external data. Entity relationship information can be the relationship between two related entities. Event information can be the content of events occurring within the corresponding domain.

[0059] The third step is to update the dynamic knowledge graph based on the entity information, entity relationship information, and event information mentioned above, to obtain the updated dynamic knowledge graph.

[0060] As an example, the aforementioned executing entity can integrate the aforementioned entity information, entity relationship information, and event information increments into the dynamic knowledge graph to obtain the updated dynamic knowledge graph.

[0061] The fourth step is to replace the stored dynamic knowledge graph with the updated dynamic knowledge graph described above.

[0062] In some optional implementations of certain embodiments, the aforementioned knowledge query information includes: entity information and entity relationship information. Entity information may be the various entities mentioned in the target problem. Entity relationship information may be the entity associations corresponding to the various entities in the target problem.

[0063] Optionally, the aforementioned executing entity can query related knowledge from the dynamic knowledge graph that is relevant to the knowledge query information, including the following steps: The first step involves using a multi-hop retrieval method to query the first contextual information related to the aforementioned entity information and entity relationship information from the dynamic knowledge graph. In practice, in specific automotive aftermarket scenarios, the dynamic knowledge graph can guide the system to retrieve knowledge related to specific car models, engine models, common abnormal noise types, and their corresponding repair solutions, rather than simply using literal matching. DynaGRAG's query-aware retrieval mechanism and dynamic traversal algorithm can retrieve relevant and diverse subgraphs, discovering deeper connections. KG^2RAG also utilizes the knowledge graph to provide fact-level relationships, improving the diversity and coherence of retrieval results. This deep integration can go beyond simple keyword or vector matching, achieving deep association of knowledge and true semantic understanding.

[0064] The second step is to generate the aforementioned related knowledge based on the first context information.

[0065] As an example, the aforementioned executing entity can directly identify the first context information as related knowledge.

[0066] In some optional implementations of certain embodiments, the aforementioned knowledge query information further includes: intent information. Intent information can be the intent recognition result corresponding to the target question. For example, intent information can include: intent category and intent confidence. The aforementioned dynamic knowledge graph also includes: case knowledge. Case knowledge can be the case content of event cases within the target domain. In practice, case knowledge can be represented in the form of knowledge nodes in the dynamic knowledge graph. For example, case knowledge can include: case object information, case object relationships, case solution methods, and case questions.

[0067] Optionally, the aforementioned executing entity may generate the aforementioned related knowledge based on the aforementioned first context information, including the following steps: The first step is to retrieve, based on the aforementioned intent information and the first context information, case knowledge from the dynamic knowledge graph that meets the target association condition in terms of relevance to the target question, and use this as target case knowledge. The relevance degree can be the similarity between the case knowledge and the intent information and the first context information. The relevance degree can be a value between 0 and 1. The higher the value, the higher the similarity. The target association condition can be case knowledge with a relevance degree higher than a target value. The target value can be a pre-set similarity value. Case knowledge with a relevance degree higher than the target value can be represented as knowledge with a high degree of relevance. In practice, for... As an example, the aforementioned executing entity uses a large language model to recall case knowledge from a dynamic knowledge graph that has a higher degree of correlation with the intent information and the aforementioned first context information than the target value, and uses it as target case knowledge.

[0068] The second step is to generate related knowledge based on the first context information and the target case knowledge mentioned above.

[0069] As an example, the aforementioned executing entity can identify the first context information and the target case information as related knowledge.

[0070] In some optional implementations of certain embodiments, the aforementioned dynamic knowledge graph further includes: fault information in multiple modalities. The multiple modalities may include, but are not limited to, at least one of the following: image modality, text modality, video modality, and audio modality. Fault information can be knowledge in different modalities. In practice, fault information in multiple modalities may include, but is not limited to, at least one of the following: fault information in image modality, fault information in text modality, fault information in video modality, and fault information in audio modality. For example, fault information in audio modality may be a recording of abnormal engine noise. Fault information in image modality may be fault images uploaded by technicians.

[0071] Optionally, the aforementioned executing entity may generate related knowledge based on the aforementioned first context information and the aforementioned target case knowledge, including the following steps: The first step is to query the fault information related to the above entity information and entity relationship information from the dynamic knowledge graph, and use this information as the target fault information.

[0072] As an example, the aforementioned execution entity can utilize a large language model to query fault content associated with the aforementioned entity information and entity relationship information from a dynamic knowledge graph, using this as the target fault information. Based on the target fault information, more comprehensive diagnoses and solutions can be provided. The supplementation of multimodal fault information will further broaden the input dimensions of the question-answering system, enabling it to handle more complex real-world problems, such as identifying component damage through image recognition or determining fault types through sound analysis.

[0073] The second step is to generate the aforementioned related knowledge based on the first context information, the target case knowledge, and the target fault information.

[0074] As an example, the aforementioned executing entity can directly identify the aforementioned first context information, the aforementioned target case knowledge, and the target fault information as associated knowledge.

[0075] Step 203: Generate the first initial response information based on the aforementioned related knowledge.

[0076] In some embodiments, the executing entity may generate first initial response information based on the aforementioned related knowledge. The first initial response information may be a preliminarily generated response content to be assessed for quality.

[0077] As an example, the aforementioned executing entity can generate the first initial response information based on the aforementioned related knowledge and the aforementioned target question, using a pre-trained large language model.

[0078] Step 204: In response to the first evaluation information corresponding to the first initial response information indicating that the response quality assessment was not passed, the generation step is executed: Step 2041: Based on the first assessment information mentioned above, generate reflection information corresponding to the first initial response information mentioned above.

[0079] In some embodiments, the executing entity can generate reflection information corresponding to the first initial response information based on the first evaluation information. The first evaluation information can be an evaluation result assessing the quality of the response corresponding to the first initial response information. In practice, the first evaluation information can be in numerical form or in label form. When it is in numerical form, a higher value indicates a higher quality response corresponding to the first initial response information. The reflection information can be reflections on logical problems in the response of the first initial response information. Based on the reflection information, the accuracy of the response content can be achieved.

[0080] As an example, the aforementioned implementing entity can generate evaluation prompts for the initial response and the target question. These prompts are then input into a large language model to obtain the first evaluation information. In response to the determination that the first evaluation information indicates a failure to pass the response quality assessment, reflection prompts are generated based on the first evaluation information, the target question, and the initial response. Finally, these reflection prompts are input into the large language model to obtain reflection information.

[0081] As another example, the aforementioned implementing entity can utilize techniques such as SAM-RAG self-evaluation verification, CRAG retrieval quality assessment, IterKey keyword optimization iteration, SIM-RAG knowledge boundary adjustment, and InstructRAG self-synthesis denoising to generate reflective information.

[0082] Step 2042: Based on the dynamic knowledge graph described above, generate a second initial response to the aforementioned reflection information.

[0083] In some embodiments, the executing entity may generate a second initial response to the reflected information based on the dynamic knowledge graph. The second initial response may be a response adjusted to address the reflected information.

[0084] As an example, firstly, the aforementioned implementing entity can reconstruct the target problem based on the reflected information and using a large language model, thus obtaining the reconstructed problem. Then, it generates reconstruction knowledge query information corresponding to the reconstructed problem. Next, it queries the dynamic knowledge graph for related knowledge information related to the aforementioned reconstruction knowledge query information, using this as reconstruction-related knowledge. Finally, based on the reconstruction-related knowledge, it uses the large language model to generate second initial response information.

[0085] As another example, the aforementioned implementing entity can leverage a large language model to extract reflective knowledge from the dynamic knowledge graph based on reflective information and the dynamic knowledge graph itself. Then, based on this reflective knowledge, it can use the large language model to generate a second initial response.

[0086] In some optional implementations of certain embodiments, the aforementioned executing entity may generate a second initial response to the aforementioned reflection information, including the following steps: The first step is to generate adjustment strategy information corresponding to the aforementioned reflection information. This adjustment strategy information includes at least one of the following: strategy information for adjusting knowledge query information, and strategy information for adjusting the query paths corresponding to related knowledge. The strategy information for adjusting knowledge query information can characterize the strategies for reconstructing the target problem and regenerating knowledge query information. In practice, the execution of the strategy for adjusting knowledge query information can be based on a large language model. Adjusting the query paths corresponding to related knowledge can involve adjusting the knowledge query paths in the knowledge graph.

[0087] The second step is to regenerate the second initial response information based on the above adjustment strategy information.

[0088] As an example, the aforementioned implementing entity can utilize a large language model to execute the strategy corresponding to the adjustment strategy information in order to regenerate the second initial response information.

[0089] Step 2043: In response to the second evaluation information characterization corresponding to the second initial response information, the second initial response information is determined as the response information corresponding to the target question through response quality evaluation.

[0090] In some embodiments, in response to the second initial response information corresponding to the second evaluation information representing a response quality assessment, the implementing entity may determine the second initial response information as the response information corresponding to the target question. The interpretation and generation of the second evaluation information can be found in the interpretation and generation of the first evaluation information. Further details will not be repeated here.

[0091] In some optional implementations of certain embodiments, after step 204, the steps further include: In response to the second assessment information indicating that the response quality assessment has not been passed, the implementing entity may use the second initial response information as the first initial response information and the second assessment information as the first assessment information, and continue to perform the above generation steps.

[0092] In some optional implementations of certain embodiments, after step 204, the steps further include: The first step is to obtain user feedback information regarding the above-mentioned responses. This user feedback information can be the results of users' responses to the replies.

[0093] The second step involves optimizing at least one of the following based on the user feedback: the generation method of the initial response information, the information query method of the dynamic knowledge graph, and the adjustment method of the adjustment strategy for the reflection information. Specifically, the generation method of the initial response information can be a method of first generating knowledge query information, then performing related knowledge queries, and finally using a large language model to generate the initial response information. User feedback allows for adaptive adjustments to the extraction of knowledge query information, the query method of the knowledge graph, and the training of the large language model. The information query method of the dynamic knowledge graph can be a method of querying related information. Based on user feedback, the path for querying related information is adjusted. The adjustment method for the adjustment strategy for the reflection information can be an adjustment to how the adjustment strategy is generated under the reflection information, so that the execution of the adjusted strategy can effectively respond to user feedback.

[0094] The third step is to implement at least one of the above optimization methods in order to provide a subsequent response. Implementing at least one optimization method in this context can mean applying it during the subsequent response process.

[0095] The above embodiments of this disclosure have the following beneficial effects: By using the knowledge retrieval and error correction mechanism of dynamic knowledge graphs through the response information generation method of some embodiments of this disclosure, the quality of knowledge retrieval can be improved and the response content corresponding to the target question can be accurately generated. Specifically, the reason why the relevant response content is not accurate enough is that traditional RAGs rely on a fixed number of retrieval documents, resulting in problems such as incomplete retrieval information, noise interference, and lack of verification mechanisms. Especially in the automotive aftermarket, its "static" characteristics make it difficult to capture deep semantic relationships, limiting retrieval quality and failing to meet high-precision requirements. Based on this, the response information generation method of some embodiments of this disclosure first determines the knowledge query information corresponding to the target question to facilitate high-quality knowledge retrieval later. Then, it queries the associated knowledge related to the above knowledge query information from the dynamic knowledge graph. Here, based on the real-time updating characteristic of the knowledge corresponding to the dynamic knowledge graph, the drawback of relying on a fixed number of retrieval documents can be avoided, and the problem of incomplete retrieval information can be solved. In addition, based on the knowledge query information, accurate querying from the dynamic knowledge graph can avoid the problem of noise interference. Then, based on the above associated knowledge, the first initial response information can be generated accurately. Next, in response to the first evaluation information representation corresponding to the first initial response information failing the response quality assessment, the following generation steps are executed: First, based on the first evaluation information, reflective information corresponding to the first initial response information is generated. Here, when it is determined that the evaluation result corresponding to the first initial response information does not meet the response quality requirements, reflective information is generated to achieve self-reflection and error correction of the corresponding defects in the response content, so as to facilitate continuous learning and evolution of the response content in the future. This allows for the generation of more accurate response information in the future. Second, based on the dynamic knowledge graph, a second initial response information is generated for the reflective information. Here, the dynamic knowledge graph is used again for further learning of related knowledge to effectively process the reflective information, resulting in a second initial response information with higher accuracy. Third, in response to the second evaluation information representation corresponding to the second initial response information passing the response quality assessment, the second initial response information is determined as the response information corresponding to the target question. In summary, based on the real-time updating characteristics of knowledge query information and dynamic knowledge graph, accurate querying and positioning of related knowledge can be achieved, providing sufficient material for the subsequent generation of response information. In addition, the reflection mechanism enables the relearning of knowledge and the re-analysis of responses. Through multiple rounds of iterative and dynamic strategy adjustments, high-quality responses are generated. This embedded self-reflection and error-correction capability significantly improves the robustness and accuracy of the question-and-answer system, allowing it to autonomously learn from mistakes and optimize its problem-solving process.

[0096] Further reference Figure 3The diagram illustrates a flow 300 of another embodiment of the response information generation method according to the present disclosure. This response information generation method includes the following steps: Step 301: Determine the knowledge query information corresponding to the target question.

[0097] Step 302: Query related knowledge from the dynamic knowledge graph that is related to the knowledge query information mentioned above.

[0098] Step 303: Generate the first initial response information based on the aforementioned related knowledge.

[0099] Step 304: In response to the first evaluation information corresponding to the first initial response information indicating that the response quality assessment was not passed, the generation step is executed: Step 3041: Based on the first assessment information mentioned above, generate reflection information corresponding to the first initial response information mentioned above.

[0100] Step 3042: Based on the dynamic knowledge graph described above, generate a second initial response to the aforementioned reflection information.

[0101] Step 3043: In response to the second evaluation information representation corresponding to the second initial response information, the second initial response information is determined as the response information corresponding to the target question through response quality evaluation.

[0102] In some embodiments, the specific implementation of steps 301-304 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-204 in the corresponding embodiments will not be repeated here.

[0103] Step 305: Package the second context information, the corresponding case event, the corresponding evaluation information, and the corresponding cited article information in the dynamic knowledge graph corresponding to the above response information to obtain a packaged file.

[0104] In some embodiments, the execution entity (e.g. Figure 1 The electronic device 101 shown can package the second context information, the corresponding case event, the corresponding evaluation information, and the corresponding cited article information in the dynamic knowledge graph corresponding to the above response information to obtain a packaged file.

[0105] Step 306: Display the above response information and the above packaged file on the target page to support the display of the various contents of the above packaged file on the target page.

[0106] In some embodiments, the executing entity may display the response information and the packaged file on a target page to support the display of various contents within the packaged file on the target page. The target page may be a response page.

[0107] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 In some corresponding embodiments, the process 300 of the response information generation method enhances the credibility and traceability of the response content by presenting the final answer or solution and providing detailed traceability information, indicating which nodes, document fragments or historical cases in the knowledge graph the answer comes from.

[0108] like Figure 4 The diagram shows a module schematic of a question-and-answer system in the automotive aftermarket sector.

[0109] like Figure 4As shown, the question-and-answer system can include: a user interaction layer, a feedback optimization layer, a core processing layer, and a knowledge layer. The user interaction layer is responsible for receiving the technician's query input (i.e., the target question, which can be in voice or text form) and displaying the system's response (i.e., the answer content). The core processing layer includes a deep research agent, a dynamic knowledge graph module, and an adaptive RAG module, which work in complex coordination to understand the query, retrieve information, generate answers, and perform self-evaluation and error correction. The knowledge layer includes a raw document library (such as repair manuals and technical bulletins), a dynamic case library (stores historical repair cases), and external tool interfaces (used to call diagnostic tools, query component information, etc.). The feedback optimization layer collects user feedback and internal evaluation results from the question-and-answer system to continuously optimize the entire system. The deep research agent is responsible for driving the dynamism and adaptability of the entire question-and-answer process. The deep research agent can dynamically adjust the reasoning path and information acquisition strategy based on the complexity of the user query (i.e., the target question) (i.e., the complexity of the question's information) and the retrieval results (i.e., related knowledge). For simple questions, the question-and-answer system directly retrieves and generates the answer. For complex problems, in-depth analysis can be achieved through sub-problem decomposition, multi-hop reasoning, and phased retrieval. Leveraging the agent's dynamic reasoning and adaptive long-range planning characteristics, combined with deep iterative thought chain (CoT) reasoning to gradually activate relevant knowledge and breadth reasoning to generate diverse initial paths, the system ensures flexible handling of queries of varying complexity. Furthermore, when a single knowledge retrieval is insufficient, the deep research agent can perform multi-hop retrieval. Based on preliminary results, new queries (i.e., new target questions) are generated to further explore deeper information. Simultaneously, the deep research agent supports calling external tools (e.g., diagnostic interfaces, maintenance manual APIs, case library retrieval tools, etc.) to obtain structured data or perform specific operations, overcoming the limitations of simple information matching to achieve complex knowledge exploration and problem solving. Next, the deep research agent can use a built-in evaluator (e.g., LLM-as-a-judge) to evaluate the fidelity, relevance, and completeness of the response content in real time. If the standards are not met, the deep research agent identifies error patterns and triggers a reflection mechanism, dynamically adjusting the retrieval strategy, reconstructing the query, or regenerating the answer until the confidence level or convergence condition is met. This process integrates SAM-RAG's self-evaluation and verification, CRAG's retrieval quality assessment, IterKey's keyword optimization iteration, SIM-RAG's knowledge boundary adjustment, and InstructRAG's self-synthesis and denoising techniques to form "metacognitive" capabilities, overcoming the static nature and lack of reflection inherent in traditional RAGs. The feedback optimization layer can include: triggering the optimization engine to optimize the response process based on user feedback and optimizing the response content based on the question-and-answer system's self-evaluation (i.e., generating evaluation information). The knowledge layer can be generated based on knowledge acquired from the original document library, dynamic case library, and external tool interfaces.

[0110] Further reference Figure 5As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a response information generation apparatus, which are similar to... Figure 2 Corresponding to the method embodiments shown, this response information generation device can be specifically applied to various electronic devices.

[0111] like Figure 5 As shown, a response information generation device 500 includes: a determining unit 501, a querying unit 502, a generating unit 503, and an execution unit 504. The determining unit 501 is configured to determine knowledge query information corresponding to a target question; the querying unit 502 is configured to query related knowledge from a dynamic knowledge graph; the generating unit 503 is configured to generate first initial response information based on the related knowledge; and the execution unit 504 is configured to, in response to a first evaluation information indicating that the response quality assessment has not been passed, perform the following generation steps: generating reflection information corresponding to the first initial response information based on the first evaluation information; generating second initial response information for the reflection information; and, in response to a second evaluation information indicating that the response quality assessment has been passed, determining the second initial response information as the response information corresponding to the target question.

[0112] In some optional implementations of certain embodiments, the apparatus 500 further includes a step execution unit (not shown in the figure). This step execution unit can be configured to: in response to the second evaluation information indicating that the response quality assessment has failed, use the second initial response information as the first initial response information, use the second evaluation information as the first evaluation information, and continue executing the generation steps.

[0113] In some optional implementations of some embodiments, the knowledge query information includes entity information and entity relationship information; and the query unit 502 can be further configured to: use a multi-hop retrieval method to query first context information related to the entity information and entity relationship information from the dynamic knowledge graph; and generate the associated knowledge based on the first context information.

[0114] In some optional implementations of some embodiments, the knowledge query information further includes: intent information; the dynamic knowledge graph further includes: case knowledge; and the query unit 502 can be further configured to: recall case knowledge from the dynamic knowledge graph that satisfies the target association condition with the target problem based on the intent information and the first context information, as target case knowledge; and generate associated knowledge based on the first context information and the target case knowledge.

[0115] In some optional implementations of some embodiments, the dynamic knowledge graph further includes: fault information in a multimodal manner; and the query unit 502 can be further configured to: use a multi-hop retrieval method to query fault information related to the entity information and the entity relationship information from the dynamic knowledge graph as target fault information; and generate the associated knowledge based on the first context information, the target case knowledge and the target fault information.

[0116] In some optional implementations of certain embodiments, the apparatus 500 further includes a packaging unit and a display unit (not shown in the figure). The packaging unit can be configured to package the second context information, corresponding case events, corresponding evaluation information, and corresponding cited article information referenced in the dynamic knowledge graph corresponding to the response information, to obtain a packaged file. The display unit can be configured to display the response information and the packaged file on a target page, supporting the display of various contents within the packaged file on the target page.

[0117] In some optional implementations of some embodiments, the generation unit 503 may be further configured to: generate adjustment strategy information corresponding to the above-mentioned reflection information, wherein the adjustment strategy information includes at least one of the following: strategy information for adjusting knowledge query information, strategy information for adjusting the query path corresponding to associated knowledge; and regenerate the second initial response information according to the above-mentioned adjustment strategy information.

[0118] In some optional implementations of certain embodiments, the apparatus 500 further includes: an information determination unit, a response execution unit, and an information generation unit (not shown in the figure). The information determination unit can be configured to: determine the problem complexity information corresponding to the target problem. The response execution unit can be configured to: in response to the problem complexity information satisfying the target complexity conditions, execute a single-step reasoning response to the target problem to obtain third initial response information. The information generation unit can be configured to: generate response information based on the third initial response information.

[0119] In some optional implementations of some embodiments, the information generation unit may be further configured to: in response to determining that the third initial response information corresponds to the third evaluation information indicating that the response quality assessment has not been passed, use the third initial response information as the first initial response information and the third evaluation information as the first evaluation information, and continue to perform the above generation steps.

[0120] In some optional implementations of certain embodiments, the apparatus 500 further includes an acquisition unit, an optimization unit, and an online unit (not shown in the figure). The acquisition unit can be configured to acquire user feedback information regarding the response information. The optimization unit can be configured to optimize at least one of the following based on the user feedback information: the generation method corresponding to the first initial response information, the information query method corresponding to the dynamic knowledge graph, and the adjustment method of the adjustment strategy for the reflection information, to obtain at least one optimized method. The online unit can be configured to apply the at least one optimized method online for subsequent responses.

[0121] It is understandable that the units described in the response information generation device 500 are related to the reference. Figure 2 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the response information generation device 500 and the units contained therein, and will not be repeated here.

[0122] The following is for reference. Figure 6 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 A schematic diagram of the structure of electronic device 101)600. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0123] like Figure 6 As shown, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage device 608 into a random access memory 603. The random access memory 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, the read-only memory 602, and the random access memory 603 are interconnected via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0124] Typically, the following devices can be connected to the input / output interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 6 Each box shown can represent a device or multiple devices as needed.

[0125] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a read-only memory 602. When the computer program is executed by the processing device 601, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0126] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0127] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0128] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: determine knowledge query information corresponding to the target question; query related knowledge from a dynamic knowledge graph; generate first initial response information based on the related knowledge; in response to a first evaluation information characterizing that the response quality assessment has not been passed, perform the generation step: generate reflection information corresponding to the first initial response information based on the first evaluation information; generate second initial response information for the reflection information based on the dynamic knowledge graph; and in response to a second evaluation information characterizing that the response quality assessment has been passed, determine the second initial response information as the response information corresponding to the target question.

[0129] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0131] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a determining unit, a querying unit, a generating unit, and an executing unit. The names of these units do not necessarily limit the specific unit; for example, a determining unit may also be described as "a unit that determines knowledge query information corresponding to a target problem."

[0132] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0133] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the above-described response information generation methods.

[0134] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for generating a reply information, comprising: determining knowledge query information corresponding to a target question; querying associated knowledge related to the knowledge query information from a dynamic knowledge graph; generating first initial reply information according to the associated knowledge; in response to first evaluation information corresponding to the first initial reply information indicating that the reply quality evaluation fails, performing the generating step: generating reflection information corresponding to the first initial reply information according to the first evaluation information; generating second initial reply information for the reflection information according to the dynamic knowledge graph; in response to second evaluation information corresponding to the second initial reply information indicating that the reply quality evaluation passes, determining the second initial reply information as the reply information corresponding to the target question.

2. The method of claim 1, wherein, The method further comprises: in response to the second evaluation information indicating that the reply quality evaluation fails, continuing to perform the generating step by taking the second initial reply information as the first initial reply information and taking the second evaluation information as the first evaluation information.

3. The method of claim 1, wherein, The knowledge query information comprises entity information and entity relationship information; and The querying of the associated knowledge related to the knowledge query information from the dynamic knowledge graph comprises: querying first context information related to the entity information and the entity relationship information from the dynamic knowledge graph by using a multi-hop retrieval method; and generating the associated knowledge according to the first context information.

4. The method of claim 3, wherein, The knowledge query information further comprises intention information, and the dynamic knowledge graph further comprises case knowledge; and The generating of the associated knowledge according to the first context information comprises: recalling case knowledge associated with the target question to a degree satisfying a target association condition as target case knowledge from the dynamic knowledge graph according to the intention information and the first context information; and generating the associated knowledge according to the first context information and the target case knowledge.

5. The method of claim 4, wherein, The dynamic knowledge graph further comprises fault information in a multi-modal manner; and The generating of the associated knowledge according to the first context information and the target case knowledge comprises: querying fault information related to the entity information and the entity relationship information from the dynamic knowledge graph as target fault information; and generating the associated knowledge according to the first context information, the target case knowledge and the target fault information.

6. The method of claim 1, wherein, The method further comprises: packing second context information referenced in the dynamic knowledge graph, corresponding case events, corresponding evaluation information and corresponding referenced article information corresponding to the reply information to obtain a packed file; and displaying the reply information and the packed file on a target page to support the display of each content in the packed file on the target page.

7. The method of claim 1, wherein, The generating of the second initial reply information for the reflection information comprises: generating adjustment strategy information corresponding to the reflection information, wherein the adjustment strategy information comprises at least one of the following: strategy information for adjusting the knowledge query information, and strategy information for adjusting a query path corresponding to the associated knowledge; and regenerating the second initial reply information according to the adjustment strategy information.

8. The method of claim 1, wherein, Before the determining the knowledge query information corresponding to the target question, the method further includes: determining question complexity information corresponding to the target question; in response to the question complexity information satisfying a target complexity condition, performing single-reasoning reply for the target question to obtain third initial reply information; generating reply information according to the third initial reply information.

9. The method of claim 8, wherein, The generating reply information according to the third initial reply information includes: in response to determining that the third initial reply information corresponds to third evaluation information representing failure in reply quality evaluation, taking the third initial reply information as first initial reply information and the third evaluation information as first evaluation information, and continuing to perform the generating step.

10. The method of claim 1, wherein, The dynamic knowledge graph is updated by the following steps: in response to determining to update the dynamic knowledge graph, acquiring external material information; extracting entity information, entity relationship information and event information corresponding to the external material information by using a pre-trained large language model; updating the dynamic knowledge graph according to the entity information, the entity relationship information and the event information to obtain an updated dynamic knowledge graph; replacing the stored dynamic knowledge graph with the updated dynamic knowledge graph.

11. The method of claim 1, wherein, The method further includes: acquiring user feedback information for the reply information; optimizing at least one of a generation mode corresponding to the first initial reply information, an information query mode corresponding to the dynamic knowledge graph, and an adjustment mode of an adjustment strategy for the reflection information according to the user feedback information to obtain at least one optimization mode; applying the at least one optimization mode online to perform subsequent reply.

12. A reply information generation apparatus, comprising: a determination unit configured to determine knowledge query information corresponding to a target question; a query unit configured to query associated knowledge related to the knowledge query information from a dynamic knowledge graph; a generation unit configured to generate first initial reply information according to the associated knowledge; an execution unit configured to, in response to first evaluation information corresponding to the first initial reply information representing failure in reply quality evaluation, perform a generation step of generating reflection information corresponding to the first initial reply information according to the first evaluation information; generating second initial reply information for the reflection information; and in response to second evaluation information corresponding to the second initial reply information representing passing in reply quality evaluation, determining the second initial reply information as reply information corresponding to the target question.

13. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-11.

14. A computer readable medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the method of any one of claims 1-11.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-11.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-11.

Images