Replay generation method and device, equipment, medium and product

Abstract

The invention provides a reply generation method and device, equipment, a medium and a product. The method comprises the steps of obtaining a first question text; extracting at least one first entity from the first question text; querying in a pre-constructed knowledge graph to obtain a plurality of target entities related to the at least one first entity; generating a first prompt word; wherein the first prompt word comprises a first question text, entity information related to a plurality of target entities and information used for indicating to reply to the first question text; and sending the first prompt word to the first language model, and receiving a reply text of the first question text returned by the first language model. According to the method, the knowledge information is stored in the form of the knowledge graph, the knowledge information which is more atomic and better matched with the question text can be obtained in the mode of performing knowledge retrieval from the knowledge graph, the accuracy of knowledge retrieval is improved, and then the accuracy of language model reply generation is enhanced.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and in particular, to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating responses. Background Art

[0002] With the rapid development of computer technologies, language models have emerged. Language models usually have natural language processing capabilities and can handle different types of natural language tasks. To improve the reasoning ability of language models in specific fields, the industry usually adopts the retrieval-augmented generation (RAG) technology to assist language models in task processing.

[0003] Specifically, the RAG technology is divided into two stages: a knowledge retrieval stage and a response generation stage. Among them, the knowledge retrieval stage is used to retrieve knowledge information related to the question text from a knowledge base, and the response generation stage is used to generate a response text by combining the knowledge information related to the question text.

[0004] In related technologies, the knowledge base is usually a vector database, that is, the knowledge information is split and then vector calculations are performed, and it is stored in the form of vectors. In the knowledge retrieval stage, retrieval is performed by calculating vector similarity. However, the above-mentioned method of knowledge retrieval from a vector database often fails to obtain retrieval results with a high degree of accuracy, thereby affecting the response generation results of language models. Summary of the Invention

[0005] The present application provides a method for generating responses. This method can improve the accuracy of knowledge retrieval, thereby enhancing the accuracy of response generation of language models. The present application also provides an apparatus, an electronic device, a computer-readable storage medium, and a computer program product corresponding to the above method.

[0006] In a first aspect, the present application provides a method for generating responses, the method comprising:

[0007] Obtain a first question text;

[0008] Extract at least one first entity from the first question text;

[0009] Query from a pre-constructed knowledge graph to obtain a plurality of target entities related to the at least one first entity;

[0010] Generate a first prompt; wherein, the first prompt includes the first question text, entity information related to the plurality of target entities, and information for indicating a response to the first question text;

[0011] Send the first prompt word to the first language model and receive the response text of the first question text returned by the first language model.

[0012] In a second aspect, the present application provides a response generation device, which includes:

[0013] An acquisition module, configured to acquire a first question text;

[0014] An extraction module, configured to extract at least one first entity from the first question text;

[0015] A query module, configured to query from a pre-constructed knowledge graph to obtain a plurality of target entities related to the at least one first entity;

[0016] A generation module, configured to generate a first prompt word; wherein, the first prompt word includes the first question text, entity information related to the plurality of target entities, and information for instructing to reply to the first question text;

[0017] A response module, configured to send the first prompt word to the first language model and receive the response text of the first question text returned by the first language model.

[0018] In a third aspect, the present application provides an electronic device, which includes a processor and a memory. The processor and the memory communicate with each other. The processor is configured to execute instructions stored in the memory so that the electronic device executes the response generation method as described in the first aspect or any implementation manner of the first aspect.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and the instructions instruct an electronic device to execute the response generation method as described in the first aspect or any implementation manner of the first aspect.

[0020] In a fifth aspect, the present application provides a computer program product including instructions, which, when running on an electronic device, enables the electronic device to execute the response generation method as described in the first aspect or any implementation manner of the first aspect.

[0021] Based on the implementation manners provided in the above aspects, the present application can be further combined to provide more implementation manners.

[0022] From the above technical solutions, it can be seen that the present application has the following advantages:

[0023] The present application provides a reply generation method. First, the method obtains a first question text, extracts at least one first entity from the first question text, and then queries a pre-constructed knowledge graph to obtain multiple target entities related to the at least one first entity, and generates a first prompt word. The first prompt word includes the first question text, entity information related to the multiple target entities, and information for indicating a reply to the first question text. The first prompt word is sent to a first language model, and a reply text for the first question text returned by the first language model is received.

[0024] In this method, different from the traditional way of storing knowledge information using a vector database, the knowledge information is stored in the form of a knowledge graph. In the knowledge retrieval stage, in combination with the entities in the question text, the target entities associated with them are queried from the knowledge graph to assist the language model in the process of generating the reply text. By means of knowledge retrieval from the knowledge graph, more atomic and more question-text-matching knowledge information can be obtained, improving the accuracy of knowledge retrieval, and thus enhancing the accuracy of reply generation by the language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the technical methods of the embodiments of the present application, the drawings required for the embodiments will be briefly introduced below.

[0026] Figure 1 It is a schematic structural diagram of a reply generation system provided by an embodiment of the present application;

[0027] Figure 2 It is a schematic flowchart of a reply generation method provided by an embodiment of the present application;

[0028] Figure 3 It is a schematic structural diagram of a reply generation device provided by an embodiment of the present application;

[0029] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The terms "first" and "second" in the embodiments of the present application are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0031] First, some technical terms and application scenarios involved in the embodiments of the present application are introduced.

[0032] With the rapid development of computer technology, language models have emerged. Language models have powerful expressive and learning abilities and can handle various types of natural language tasks, such as dialogue tasks, translation tasks, speech recognition tasks, text generation tasks, semantic analysis tasks, etc.

[0033] Under normal circumstances, a language model can perform natural language processing on the question text and generate a corresponding response text. To improve the reasoning performance of the language model, the industry usually adopts the retrieval-augmented generation (RAG) technology to assist the language model in task processing. Specifically, the RAG technology is divided into two stages: the knowledge retrieval stage and the response generation stage. In the knowledge retrieval stage, for the question text, relevant knowledge information is retrieved from the knowledge base. In the response generation stage, the knowledge information obtained in the knowledge retrieval stage and the question text are sent to the language model together, enabling the language model to analyze the question text in combination with the knowledge information and then generate a response text.

[0034] In related technologies, the knowledge base is usually a vector database. In this case, the knowledge information is split and then vector calculations are performed, and it is stored in the form of vectors to form a knowledge base. In the knowledge retrieval stage, vector calculations are performed on the question text, and then the vector similarity between the question text and the knowledge information in the knowledge base is calculated using a vector model. The knowledge information with a higher similarity (for example, the similarity is greater than the similarity threshold) is used as the knowledge information related to the question text to complete the knowledge retrieval.

[0035] However, the above method of knowledge retrieval from the vector database has the following problems: First, in the process of constructing the knowledge base, the knowledge information needs to be split, and then vector calculations are performed on the split knowledge information. However, when the splitting of the knowledge information is illogical, or the splitting logic of the knowledge information does not match the business scenario, it may lead to the inability to retrieve the knowledge information related to the question text in the knowledge retrieval stage. Second, commonly used vector models are difficult to fully fit the business scenario. In the case of similarity calculation for a specific business scenario, the vector model cannot understand proper nouns and semantic similarities but different topics. Without being strongly related to the business scenario training, the vector model is difficult to accurately perform similarity calculations, affecting the accuracy of knowledge retrieval. In addition, in the knowledge retrieval stage, in order to retrieve the knowledge information related to the question text, the similarity threshold may need to be lowered. However, after lowering the similarity threshold, the relevance between the retrieved knowledge information and the question text will be reduced, and in the response generation stage, it will cause the language model to generate responses by combining weakly relevant knowledge information, affecting the quality of the response text.

[0036] In view of this, the present application provides a reply generation method. The method first obtains a first question text, extracts at least one first entity from the first question text, and then queries from a pre-constructed knowledge graph to obtain multiple target entities related to the at least one first entity, and generates a first prompt word. The first prompt word includes the first question text, entity information related to the multiple target entities, and information for indicating a reply to the first question text. The first prompt word is sent to a first language model, and a reply text to the first question text returned by the first language model is received.

[0037] In this method, different from the traditional way of storing knowledge information using a vector database, the knowledge information is stored in the form of a knowledge graph. In the knowledge retrieval stage, combined with the entities in the question text, the target entities associated with them are queried from the knowledge graph to assist the language model in the process of generating the reply text. By retrieving knowledge from the knowledge graph, more atomic and more question-text-matching knowledge information can be obtained, improving the accuracy of knowledge retrieval, and thus enhancing the accuracy of the language model's reply generation.

[0038] To facilitate the understanding of the technical solution provided by the embodiments of the present application, the following will be described with reference to the accompanying drawings. Refer to Figure 1 the architecture diagram of the reply generation system shown. The reply generation system can be in the form of a hierarchical architecture. Specifically, the reply generation system can be divided into an access layer, an application layer, a domain layer, and an infrastructure layer from top to bottom, which will be introduced separately below.

[0039] The access layer, also called the presentation layer, is used for users to interact with the reply generation system. In the embodiments of the present application, the access layer can include a knowledge input unit, a knowledge debugging unit, and a knowledge retrieval unit. Among them, the knowledge input unit can be used for users to input knowledge, the knowledge debugging unit can be used for users to debug knowledge, and the knowledge retrieval unit can be used for users to retrieve the knowledge graph.

[0040] The application layer is used to coordinate the operations of the reply generation system. In the embodiments of the present application, the application layer can include a user pre-input knowledge unit, a business pre-set knowledge unit, and a knowledge graph operation unit. Among them, the user pre-input knowledge unit can be used to receive the knowledge documents and knowledge entries input by users, the business pre-set knowledge unit can be used to receive the knowledge entries pre-set by the business, and the knowledge graph operation unit can be used to receive the operations of users for the construction, debugging, modification, and query of the knowledge graph.

[0041] The domain layer, also called the business logic layer, is used to maintain the business state and business rules and execute business functions. In the embodiments of the present application, the domain layer can maintain the pre-processed knowledge text and the association relationships between the entities indicated in the knowledge graph.

[0042] The infrastructure layer, which can also be referred to as the infrastructure tier, is used to provide technical services to support the upper-layer business logic. In the embodiments of the present application, the infrastructure layer may include a relational database for storing knowledge texts, a vector database for storing vectors of entities in the knowledge graph, and a relational database for storing the association relationships between entities in the knowledge graph.

[0043] In this way, the reply generation system with a layered architecture helps to maintain the clarity and maintainability of the system. Each layer focuses on specific responsibilities, making the reply generation system easier to understand and expand.

[0044] Based on the above reply generation system, the present application also provides a reply generation method. Refer to Figure 1 the schematic flowchart of a reply generation method shown in

[0045] S201: Obtain a first question text.

[0046] Among them, the first question text can be understood as the text representing the question. In other words, the first question text can be used for asking questions.

[0047] In the embodiments of the present application, the first question text may be related to the business. That is to say, the first question text may be a question text for asking questions regarding issues related to a specific business scenario.

[0048] Specifically, the business party can access the digital assistant and use the digital assistant to conduct a human-machine conversation with the user to automatically reply to questions related to the business for the user. In this case, in response to the question input operation in the first conversation, the first text indicated by the question input operation and at least part of the second text in the first conversation are determined as the first question text.

[0049] That is to say, during the human-machine conversation between the user and the digital assistant, the user can input text to ask questions. Considering the accuracy of reply generation, the first text input by the user and the conversation history of the current human-machine conversation (i.e., at least part of the second text) are used as the first question text, and subsequent replies are made for the first question text, so that the reply text is not only related to the first text input by the user, but also related to the conversation context of the first conversation, improving the accuracy of reply generation.

[0050] S202: Extract at least one first entity from the first question text.

[0051] In the embodiments of the present application, an entity can be understood as a word with meaning. For example, when the first question text is related to the business, the entity can be understood as a noun with meaning in the business. That is to say, for the first question text, the first entity involved in the first question text is extracted.

[0052] The embodiments of the present application do not limit the method for extracting the first entity from the first question text. In some embodiments, an entity extraction model is used for entity extraction. By inputting the first question text into a pre-trained entity extraction model, at least one first entity in the first question text returned by the entity extraction model is received. In other embodiments, the first question text is segmented, and then the segmented results of the first question text are classified to obtain classification results. The segmented results representing entities in the classification results are determined as the first entity.

[0053] S203: Query from a pre-constructed knowledge graph to obtain multiple target entities related to at least one first entity.

[0054] Different from the knowledge retrieval stage of the traditional RAG technology, where vector calculations are performed on the question text and then retrieval is based on vector similarity from a knowledge base in the form of a vector database, in the embodiments of the present application, knowledge information is stored in the form of a knowledge graph. By extracting the first entity from the first question text and then querying the knowledge graph with the first entity as the index, target entities related to the first entity are obtained, completing the knowledge retrieval stage in the RAG technology.

[0055] Among them, the knowledge graph is a structured and semantic knowledge representation method. The knowledge spectrum graph usually represents knowledge information in the form of a graph. The nodes in the graph represent entities, and the edges in the graph represent the association relationships between entities.

[0056] In some possible implementation manners, multiple target entities related to at least one first entity are obtained through two rounds of recall. Specifically, according to at least one first entity, at least one first candidate entity is determined from multiple second entities indicated by the pre-constructed knowledge graph. According to the association relationships between the multiple second entities indicated by the knowledge graph, at least one second candidate entity associated with at least one first candidate entity is determined. The at least one first candidate entity and the at least one second candidate entity are determined as multiple target entities related to at least one first entity.

[0057] That is to say, in the first round of recall, from the multiple second entities included in the knowledge graph, first candidate entities with a relatively high similarity to the first entity (for example, the similarity is greater than the first similarity threshold) are screened out to complete the first round of recall. Then, according to the association relationships between the multiple second entities in the knowledge graph, second candidate entities associated with the first candidate entities are screened out to achieve the diffusion based on the first entity from the knowledge graph, obtaining multi-hop associated entities (i.e., second candidate entities).

[0058] In this way, target entities related to the first entity in the first question text are screened out from the knowledge graph. Since two rounds of recall are performed, while increasing the number of target entities, the relevance between the target entities and the first entity is ensured. At the same time, compared with storing knowledge information in a vector database, the knowledge graph can store indication information in a more atomic way, and it stores both entities and the association relationships between entities, enabling richer processing of knowledge information and a higher accuracy rate of knowledge recall.

[0059] S204: Generate a first prompt.

[0060] S205: Send the first prompt to the first language model and receive a response text for the first question text returned by the first language model.

[0061] After performing knowledge recall based on the first entity from the knowledge graph to obtain multiple target entities, the knowledge information retrieved in the knowledge retrieval stage of the RAG technology is used to assist in generating the response of the language model.

[0062] In the embodiment of this application, for generating a response to the first question text using the first language model, the first language model can be a language model with natural language processing capabilities, capable of understanding the meaning of natural language and handling different types of natural language tasks. For example, the first language model can be a deep learning model trained using text data.

[0063] The first language model can generate a response text for the first question text based on prompt learning. Among them, a prompt can be used to guide the first language model to perform a specific output in a generative task (such as a text generation task, a question-answering task, a dialogue task). By configuring the prompt, it helps the first language model understand the background and requirements of the task, enabling the first language model to handle different types of natural language processing tasks without retraining the first language model, and increasing the scalability and flexibility of the first language model.

[0064] Among them, the first prompt can include the first question text, entity information related to multiple target entities, and information for indicating a response to the first question text. Since the first prompt includes the above information, the first language model can analyze the first question text based on the prompting ability of the first prompt and in combination with the entity information related to multiple target entities, generate a response text for the first question text, and complete the response generation stage in the RAG technology.

[0065] In this method, different from the traditional way of storing knowledge information using a vector database, knowledge information is stored in the form of a knowledge graph. In the knowledge retrieval stage, in combination with the entities in the question text, the target entities associated with them are queried from the knowledge graph to assist the language model in the process of generating the response text. By retrieving knowledge from the knowledge graph, more atomic and more question-text-matching knowledge information can be obtained, improving the accuracy of knowledge retrieval and thus enhancing the accuracy of response generation by the language model.

[0066] The following is an explanation of the effect of the response generation method provided in this application. When the first question text is "For the requirement set where the number of process nodes is greater than 5 or the complexity is less than or equal to 2.35, calculate the average value of the requirement complexity and the maximum value of the number of process nodes under the requirements in terms of priority", the original knowledge text is as follows:

[0067] "Process node: A node created under a work item, representing the process topology structure of the work item

[0068] Complexity: Usually refers to the requirement complexity, which is a comprehensive rating of the number of project process nodes and the total estimated score. The formula for calculating the requirement complexity is: (total estimated score of sub-tasks / number of process nodes) / 100 + 1

[0069] Work item: Can abstractly refer to the overall information of the work item; can refer to a work item instance; without affecting understanding, it can also refer to the work item type

[0070] Instance: Also known as a work item instance, a piece of data created by the user, such as a requirement, a defect, etc.

[0071] Sub-task: A task created under a node, usually having two states: unfinished and finished

[0072] Status: Commonly used to describe the process flow status of a work item, or the completion status of a node or sub-task

[0073] Delivery: Usually refers to the action of entering the finished state

[0074] Lag: The full expression is schedule lag, usually referring to the situation where the progress does not meet the expectation, generally equivalent to delay, that is, the actual completion time is greater than the expected completion time, or the current time is greater than the expected completion time and the actual completion time is empty

[0075] Delay: For any task, only when it is still in the unfinished state when the expected completion time of the schedule is reached, it is determined to be a delayed task"

[0076] In the traditional way of storing knowledge information using a vector database, the knowledge information retrieved in the knowledge retrieval stage is as follows, and the knowledge hit rate is 33.3%:

[0077] "Process Node: A node created under a work item, representing the process topology of the work item"

[0078] Complexity: Usually refers to the requirement complexity, which is a comprehensive rating of the number of project process nodes and the total estimated score. The formula for requirement complexity is: (Total estimated score of subtasks / Number of process nodes) / 100 + 1"

[0079] After performing knowledge retrieval using the knowledge graph provided in the embodiments of this application, the retrieved knowledge information is as follows, and the knowledge hit rate is 100%:

[0080] "Process Node: A node created under a work item, representing the process topology of the work item"

[0081] Complexity: Usually refers to the requirement complexity, which is a comprehensive rating of the number of project process nodes and the total estimated score. The formula for requirement complexity is: (Total estimated score of subtasks / Number of process nodes) / 100 + 1

[0082] (From the "Process Node" knowledge) Work Item: Can abstractly refer to the overall information of the work item; can refer to a work item instance; without affecting understanding, it can also refer to the work item type

[0083] (From the "Complexity" knowledge) Subtask: A task created under a node, usually having two states: incomplete and completed

[0084] (From the "Work Item" knowledge) Instance: Also known as a work item instance, a piece of data established by the user, such as a requirement, a defect, etc.

[0085] (From the "Subtask" knowledge) Status: Commonly used to describe the process flow status of a work item, or the completion status of a node or subtask"

[0086] In the embodiments of this application, knowledge information is stored in the form of a knowledge graph. In the knowledge retrieval stage of the RAG technology, the target entity is queried from the knowledge graph to complete the knowledge retrieval. The construction process of the knowledge graph is described below.

[0087] Specifically, the knowledge graph can be constructed in the following way: Obtain multiple knowledge texts, perform entity extraction on the multiple knowledge texts to obtain the entity extraction result, and construct the knowledge graph according to the entity extraction result.

[0088] The embodiments of this application do not limit the way of obtaining knowledge texts. In some embodiments, the user can input knowledge texts in the form of key-value pairs. For example, the key in the key-value pair can be the entity name, and the value in the key-value pair can be the entity description information. In other embodiments, the user can also upload a knowledge document and use the knowledge document as multiple knowledge texts.

[0089] It should be noted that in the embodiments of the present application, the full knowledge text is used to construct the full knowledge graph. That is to say, after generating new knowledge text, the original knowledge text and the newly added knowledge text are used to reconstruct the knowledge graph to ensure that the knowledge graph can represent knowledge completely and comprehensively.

[0090] By performing entity extraction on multiple knowledge texts, an entity extraction result is obtained. The entity extraction result may include first information related to multiple second entities, and second information related to the association relationship between multiple second entities. In other words, through the entity extraction process, the second entities involved in multiple knowledge texts and the association relationship between multiple second entities are extracted. The second entities are used as nodes in the graph, and the association relationship between multiple second entities is used as edges in the graph to construct the knowledge graph.

[0091] In some possible implementation manners, the natural language processing ability of the language model is used for entity extraction. Specifically, a second prompt word is generated, the second prompt word is sent to the second language model, and the entity extraction result returned by the second language model is received.

[0092] Among them, the second language model can be understood as a language model used for entity extraction. Similarly, the second language model can perform entity extraction based on the prompt learning method. The second prompt word may include multiple knowledge texts and information for indicating entity extraction of multiple knowledge texts. Since the second prompt word includes the above information, the second language model can perform entity extraction on multiple knowledge texts based on the prompting ability of the second prompt word, and extract first information related to multiple second entities and second information related to the association relationship between multiple second entities to complete the entity extraction.

[0093] The first information related to multiple second entities can describe the second entities from multiple dimensions, and the second information related to the association relationship between multiple second entities can describe the association relationship between the second entities from multiple dimensions. In some embodiments, the first information related to multiple second entities includes the entity names of multiple second entities and the entity description information of multiple second entities. The entity information related to multiple second entities further includes at least one of the following: the entity identifiers of multiple second entities and the entity types of multiple second entities. For example, the first information related to multiple second entities can be presented in the following format:

[0094] "Entity identifier ## Entity name ## Entity type ## Entity description information"

[0095] Among them, the entity type can be a technical term, a personnel role, a data indicator, a process, a miscellaneous item, etc.

[0096] The second information related to the association relationships between multiple second entities includes: information indicating two second entities having an association relationship. The second information related to the association relationships between multiple second entities further includes at least one of the following: information indicating the relationship type of two second entities having an association relationship, information describing the association relationship between two second entities having an association relationship, and information indicating the relationship strength of two second entities having an association relationship. For example, the second information related to the association relationships between multiple second entities can be presented in the following format:

[0097] "Identifier of the second entity##Identifier of the second entity associated with this second entity##Relationship type##Relationship description##Relationship strength"

[0098] Among them, the relationship type can be related, subordinate, the same, etc. The relationship description can describe the reason why two second entities have an association relationship. The relationship strength can quantify the association relationship between two second entities.

[0099] In this way, the first information related to multiple second entities and the second information related to the association relationships between multiple second entities include rich content, making the knowledge graph more abundant.

[0100] In some embodiments, considering that the lengths of multiple knowledge texts may be relatively long, for accurate entity extraction, multiple knowledge texts can also be split into multiple text units, entity extraction is performed on the multiple text units, entity extraction sub-results of each text unit are obtained, and entity extraction results are determined based on the entity extraction sub-results of each text unit.

[0101] That is to say, by performing text segmentation on the knowledge text, for example, segmenting by a fixed text length or by a fixed token length, entity extraction is performed with text units as the unit, so that entity extraction is only performed on one text unit at a time, improving the accuracy of entity extraction.

[0102] In some possible implementation manners, a second language model is used for entity extraction. In this case, the second language model can be called in parallel to perform unified entity extraction on multiple text units after splitting multiple knowledge texts, improving the efficiency of entity extraction.

[0103] The following is an illustration with an example. Continuing with the formats of the first information and the second information shown in the previous text, when the text unit is "Example: Also known as a work item instance, a piece of data established by a user, related to requirements and defects", the first information related to multiple second entities can be:

[0104] "1##Work item instance##3##A piece of data created by the user, related to requirements and defects

[0105] 2 ## User ## 2 ## The personnel role of creating the work item instance

[0106] 3 ## Requirement ## 5 ## Miscellaneous related to the work item instance

[0107] 4 ## Defect ## 5 ## Miscellaneous related to the work item instance

[0108] The second information related to the association relationship between multiple second entities can be:

[0109] “1 ## 2 ## Related ## The instance is created by the user ## 0.9

[0110] 1 ## 3 ## Related ## The instance is related to the requirement ## 0.9

[0111] 1 ## 4 ## Related ## The instance is related to the defect ## 0.9

[0112] When the text unit is "Defect rate: Calculate all defect quantities under the associated requirement, and the calculation formula is: (Defect quantity / Number of process nodes * Number of subtasks) / 200 + 2", the first information related to multiple second entities can be:

[0113] “1 ## Defect rate ## 3 ## Calculate all defect quantities under the associated requirement, and the calculation formula is: (Defect quantity / Number of process nodes * Number of subtasks) / 200 + 2;

[0114] 2 ## Associated requirement ## 5 ## Miscellaneous related to the work item instance

[0115] 3 ## Defect quantity ## 3 ## Data index related to the defect rate

[0116] 4 ## Number of process nodes ## 3 ## Data index related to the defect rate

[0117] 5 ## Number of subtasks ## 3 ## Data index related to the defect rate

[0118] The second information related to the association relationship between multiple second entities can be:

[0119] “1 ## 2 ## Related ## The defect rate calculation involves the associated requirement ## 0.9

[0120] 1 ## 3 ## Related ## The defect rate calculation involves the defect quantity ## 0.9

[0121] 1 ## 4 ## Related ## The defect rate calculation involves the number of process nodes ## 0.9

[0122] 1 ## 5 ## Related ## The defect rate calculation involves the number of subtasks ## 0.9

[0123] When the text unit is "Example: Also known as a work item instance, a piece of data created by a user, such as a requirement, a defect, etc.; Work item: Can abstractly refer to the overall information of the work item, can refer to the work item instance, and, without affecting understanding, can also refer to the work item type; Process node: A node created under the work item, representing the process topology of the work item; Sub - task: A task created under the node, usually having two states: unfinished and finished; Estimated score interpolation calculation steps: 1. Calculate the number of days T1 within the estimated time period, that is, the number of days between the estimated start time EstimateStartDate and the estimated end time EstimateEndDate; 2. Calculate the number of days T2 in the intersection of the target time interval and the estimated time period; 3. Estimated score interpolation = Scheduled estimated score (Points) * T2 / T1; Complexity: Usually refers to the requirement complexity, which is a comprehensive rating of the number of project process nodes and the total estimated score. The formula for requirement complexity is: (Total estimated score of sub - tasks / Number of process nodes) / 100 + 1", the first information related to multiple second entities can be:

[0124] "1##Example##1##Also known as "work item instance", it is a piece of data created by the user, such as a requirement or a defect

[0125] 2##Work item##1,5##Can abstractly refer to the overall information of the work item, can refer to the work item instance, and, without affecting understanding, can also refer to the work item type

[0126] 3##Process node##1##A node created under the work item, representing the process topology of the work item

[0127] 4##Sub - task##1##A task created under the node, usually having two states: "unfinished" and "finished"

[0128] 5##Estimated score interpolation calculation steps##4##The estimated score interpolation calculation steps include: 1. Calculate the number of days T1 within the estimated time period, that is, the number of days between the estimated start time EstimateStartDate and the estimated end time EstimateEndDate; 2. Calculate the number of days T2 in the intersection of the target time interval and the estimated time period; 3. Estimated score interpolation = Scheduled estimated score (Points) * T2 / T1

[0129] 6##Complexity##1,5##Usually refers to the requirement complexity, which is a comprehensive rating of the number of project process nodes and the total estimated score. The formula for requirement complexity is: (Total estimated score of sub - tasks / Number of process nodes) / 100 + 1"

[0130] The second information related to the association relationship between multiple second entities can be:

[0131] “1##2##Subordinate##Instances are part of a work item.##0.9

[0132] 3##2##Subordinate##Process nodes are part of a work item.##0.9

[0133] 4##3##Subordinate##Sub - tasks are tasks created under a process node.##0.9

[0134] 6##3##Related##Complexity calculation involves the number of process nodes.##0.7

[0135] 6##4##Related##Complexity calculation involves the total estimated score of sub - tasks.##0.7”

[0136] Furthermore, since all knowledge texts are used for full - scale construction during the process of constructing a knowledge graph, considering construction efficiency, it is also possible to determine whether entity extraction has been performed on each of multiple text units. For the first text unit among the multiple text units, in response to the first text unit having undergone entity extraction, obtain the entity extraction sub - result of the first text unit from the cache; in response to the first text unit not having undergone entity extraction, perform entity extraction on the first text unit using a second language model to obtain the entity extraction sub - result of the first text unit. Here, the first text unit is any text unit among the multiple text units.

[0137] For example, calculate the Message Digest Algorithm (MD5) value of each text unit. If it is the same as the MD5 value of a historical text unit, it indicates that entity extraction has been performed on this text unit. For text units that have undergone entity extraction, there is no need to call the second language model for repeated entity extraction. Instead, directly obtain the corresponding entity extraction sub - result from the cache, reducing the time consumption of entity extraction and the cost of model invocation, and saving computing resources.

[0138] Considering that among the entity extraction sub - results of multiple text units, there may be the same entities. To ensure that the constructed knowledge graph does not include the same entities and improve the quality of the knowledge graph, it is possible to determine the similarity between multiple second entities indicated by the entity extraction sub - result of each text unit, and merge the entity extraction sub - results corresponding to the second entities whose similarity meets the set conditions to determine the entity extraction result.

[0139] For example, merge the entity extraction sub - results corresponding to two second entities whose similarity is greater than a second similarity threshold, so that the entity extraction result does not include entities with high similarity. At the same time, merge the association relationships indicated in the entity extraction sub - results corresponding to the second entities whose similarity meets the set conditions to ensure that the association relationships across shards (i.e., across text units) are not lost.

[0140] The following is an illustration with examples. Text unit 1 is as follows: "Example: Also known as a work item instance, it is a piece of data created by a user, such as a requirement or a defect; Work item: It can abstractly represent the overall information of the work item, can represent a work item instance, and, without affecting understanding, can also represent the work item type." The entity extraction sub-results are as follows:

[0141] "1##Example##1##Also known as a "work item instance", it is a piece of data created by a user, such as a requirement or a defect

[0142] 2##Work item##1,5##It can abstractly represent the overall information of the work item, can represent a work item instance, and, without affecting understanding, can also represent the work item type

[0143] 1##2##Subordinate##An example is a part of a work item##0.9"

[0144] Text unit 2 is as follows: "Process node: A node created under a work item, representing the process topology of the work item; Sub-task: A task created under a node, usually having two states: unfinished and finished." The entity extraction sub-results are as follows:

[0145] "1##Process node##1##A node created under a work item, representing the process topology of the work item

[0146] 2##Work item##5##Miscellaneous items related to the process node

[0147] 3##Sub-task##1##A task created under a node, usually having two states: "unfinished" and "finished"

[0148] 1##2##Subordinate##A process node is a part of a work item##0.9

[0149] 3##1##Subordinate##A sub-task is a task created under a process node##0.9"

[0150] Vector calculations are performed on the entity names of the second entities indicated by the entity extraction sub-results of text unit 1 and the entity names of the second entities indicated by the entity extraction sub-results of text unit 2, and the following results are obtained:

[0151] "Text unit 1:

[0152] Example: [0.5,0,0.2]

[0153] Work item: [1,0.3,0.4]

[0154] Text unit 2:

[0155] Process node: [0.3,0.1,0.9]

[0156] Work item: [1, 0.3, 0.4]

[0157] Sub - task: [0.6, 0, 0.5]”

[0158] After determining the similarity, it is found that the similarity between the second entity "Work item" in text unit 1 and the second entity "Work item" in text unit 2 meets the set conditions, and it is judged as an entity that needs to be merged. The entity extraction sub - results of text unit 1 and text unit 2 are merged and de - duplicated to obtain the entity extraction result. The first information related to multiple second entities in the entity extraction result can be:

[0159] "1##Instance##1##Also known as "work item instance", is a piece of data created by the user, such as a requirement or a defect

[0160] 2##Work item##1, 5##Can abstractly refer to the overall information of the work item, can refer to the work item instance; without affecting understanding, it can also refer to the work item type

[0161] 3##Process node##1##A node created under the work item, representing the process topology structure of the work item

[0162] 4##Sub - task##1##A task created under the node, usually having two states: "uncompleted" and "completed"

[0163] The second information related to the association relationship between multiple second entities can be:

[0164] "1##2##Subordinate##The instance is part of the work item##0.9

[0165] 3##2##Subordinate##The process node is part of the work item##0.9

[0166] 4##3##Subordinate##The sub - task is a task created under the process node##0.9"

[0167] In some embodiments, the vector representation of the first information related to multiple second entities in the entity extraction result can be determined, the vector representation of the first information related to multiple second entities is stored in a vector database, and, the second information related to the association relationship between multiple second entities in the entity extraction result is stored in a relational database.

[0168] That is, the information related to the entity itself is stored in the vector database, and the information related to the association relationships between entities is stored in the relational database. The two-step storage process can ensure consistency through transactions. In this way, using different types of databases to store the knowledge graph facilitates knowledge recall in the knowledge retrieval stage of subsequent RAG technology.

[0169] As described above in conjunction with Figure 1 and Figure 2 a detailed introduction to the response generation method provided by the embodiments of the present application has been given. Next, the devices and equipment provided by the embodiments of the present application will be introduced with reference to the accompanying drawings.

[0170] See Figure 3 the structural schematic diagram of the response generation device shown. The device 30 includes:

[0171] An acquisition module 301, configured to acquire a first question text;

[0172] An extraction module 302, configured to extract at least one first entity from the first question text;

[0173] A query module 303, configured to query from a pre-constructed knowledge graph to obtain a plurality of target entities related to the at least one first entity;

[0174] A generation module 304, configured to generate a first prompt word; wherein, the first prompt word includes the first question text, entity information related to the plurality of target entities, and information for instructing to reply to the first question text;

[0175] A response module 305, configured to send the first prompt word to a first language model and receive a response text of the first question text returned by the first language model.

[0176] In some possible implementation manners, the query module 303 is specifically configured to:

[0177] Determine at least one first candidate entity from a plurality of second entities indicated by the pre-constructed knowledge graph according to the at least one first entity;

[0178] Determine at least one second candidate entity associated with the at least one first candidate entity according to the association relationships between the plurality of second entities indicated by the knowledge graph;

[0179] Determine the at least one first candidate entity and the at least one second candidate entity as a plurality of target entities related to the at least one first entity.

[0180] In some possible implementation manners, the acquisition module 301 is specifically configured to:

[0181] In response to a question input operation in a first conversation, determine the first text indicated by the question input operation and at least part of the second text in the first conversation as the first question text.

[0182] In some possible implementation manners, the apparatus 30 further includes a construction module, and the construction module is configured to:

[0183] Obtain multiple pieces of knowledge text;

[0184] Perform entity extraction on the multiple pieces of knowledge text to obtain an entity extraction result; wherein, the entity extraction result includes first information related to multiple second entities, and second information related to the association relationship between the multiple second entities;

[0185] Construct a knowledge graph according to the entity extraction result.

[0186] In some possible implementation manners, the construction module is specifically configured to:

[0187] Generate a second prompt; wherein, the second prompt includes the multiple pieces of knowledge text and information for indicating entity extraction on the multiple pieces of knowledge text;

[0188] Send the second prompt to a second language model and receive the entity extraction result returned by the second language model.

[0189] In some possible implementation manners, the construction module is specifically configured to:

[0190] Split the multiple pieces of knowledge text into multiple text units;

[0191] Perform entity extraction on the multiple text units to obtain an entity extraction sub-result for each text unit;

[0192] Determine the entity extraction result according to the entity extraction sub-result of each text unit.

[0193] In some possible implementation manners, the construction module is specifically configured to:

[0194] Determine whether entity extraction has been performed on each of the multiple text units;

[0195] For a first text unit among the multiple text units, in response to the first text unit having undergone entity extraction, obtain the entity extraction sub-result of the first text unit from the cache;

[0196] In response to the fact that entity extraction has not been performed on the first text unit, use a second language model to perform entity extraction on the first text unit to obtain a sub-result of entity extraction for the first text unit; wherein, the first text unit is any text unit among the multiple text units.

[0197] In some possible implementation manners, the construction module is specifically configured to:

[0198] Determine the similarity between multiple second entities indicated by the sub-results of entity extraction of each text unit;

[0199] Merge the sub-results of entity extraction corresponding to the second entities whose similarity meets the set conditions to determine the entity extraction result.

[0200] In some possible implementation manners, the construction module is specifically configured to:

[0201] Determine the vector representation of the first information related to multiple second entities in the entity extraction result, and store the vector representation of the first information related to multiple second entities in a vector database; and,

[0202] Store the second information related to the association relationship between the multiple second entities in a relational database.

[0203] In some possible implementation manners, the first information related to multiple second entities includes the entity names of the multiple second entities and the entity description information of the multiple second entities, and the entity information related to the multiple second entities further includes at least one of the following: the entity identifiers of the multiple second entities and the entity types of the multiple second entities;

[0204] The second information related to the association relationship between the multiple second entities includes: information indicating two second entities having an association relationship, and the second information related to the association relationship between the multiple second entities further includes at least one of the following: information indicating the relationship type of two second entities having an association relationship, information describing the association relationship between two second entities having an association relationship, and information indicating the relationship strength of two second entities having an association relationship.

[0205] The reply generation device 30 according to the embodiment of the present application may correspondingly execute the method described in the embodiment of the present application, and the above and other operations and / or functions of each module / unit of the reply generation device 30 are respectively for implementing Figure 1 The corresponding processes of the respective methods in the illustrated embodiments, and for the sake of brevity, will not be described in detail here.

[0206] The embodiments of the present application also provide an electronic device. The electronic device is specifically used to implement the functions of the reply generation device 30 in the embodiments as Figure 3 shown.

[0207] Figure 4 A schematic structural diagram of an electronic device 400 is provided, as Figure 4 shown. The electronic device 400 includes a bus 401, a processor 402, a communication interface 403, and a memory 404. The processor 402, the memory 404, and the communication interface 403 communicate with each other through the bus 401.

[0208] The bus 401 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 4 only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0209] The processor 402 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0210] The communication interface 403 is used for external communication. For example, the communication interface 403 can be used for communication with a terminal.

[0211] The memory 404 can include a volatile memory, such as a random access memory (RAM). The memory 404 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0212] The memory 404 stores executable code, and the processor 402 executes the executable code to execute the foregoing reply generation method.

[0213] Specifically, when implementing Figure 3In the case of the illustrated embodiment, and Figure 3 When each module or unit of the response generation device 30 described in the embodiment is implemented by software, execute Figure 3 The software or program code required for the functions of each module / unit in can be partially or entirely stored in the memory 404. The processor 402 executes the program code corresponding to each unit stored in the memory 404 to execute the foregoing response generation method.

[0214] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc. The computer-readable storage medium includes instructions that direct the computing device to execute the foregoing response generation method applied to the response generation device 30.

[0215] The embodiment of the present application also provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the processes or functions according to the embodiments of the present application are entirely or partially generated.

[0216] The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, or data center to another website, computer, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.).

[0217] When the computer program product is executed by a computer, the computer executes any of the foregoing response generation methods. The computer program product can be a software installation package. In the case where any of the foregoing response generation methods is required, the computer program product can be downloaded and executed on the computer.

[0218] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0219] The units involved in the embodiments described in the present application can be implemented in software or in hardware. Among them, the name of the unit / module does not, in some cases, constitute a limitation on the unit itself.

[0220] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), Systems on Chip (SOC), Complex Programmable Logic Devices (CPLD), and so on.

[0221] In the context of the embodiments of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM or Flash Memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0222] It should be noted that the embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple. For related parts, reference can be made to the descriptions in the method section.

[0223] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or its similar expressions refer to any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0224] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0225] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0226] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A reply generation method, characterized in that: The method comprises: Get the first question text; Extracting at least one first entity from the first question text; Querying a pre-built knowledge graph to obtain a plurality of target entities related to the at least one first entity; Generate a first prompt word; wherein the first prompt word includes the first question text, entity information related to the multiple target entities, and information for indicating a reply to the first question text; The first prompt word is sent to a first language model, and a reply text of the first question text returned by the first language model is received.

2. The method according to claim 1, characterized in that: The querying from the pre-built knowledge graph to obtain multiple target entities related to the at least one first entity includes: Determine, according to the at least one first entity, at least one first candidate entity from a plurality of second entities indicated by a pre-constructed knowledge graph; Determine at least one second candidate entity associated with the at least one first candidate entity according to the association relationships between the multiple second entities indicated by the knowledge graph; The at least one first candidate entity and the at least one second candidate entity are determined as a plurality of target entities related to the at least one first entity.

3. The method according to claim 1, characterized in that The obtaining of the first question text comprises: In response to a question input operation in a first dialogue, a first text indicated by the question input operation and at least a portion of a second text in the first dialogue are determined as a first question text.

4. The method according to any one of claims 1 to 3, characterized in that: The knowledge graph is constructed in the following way: Get multiple knowledge texts; Performing entity extraction on the plurality of knowledge texts to obtain entity extraction results; wherein the entity extraction results include first information related to the plurality of second entities and second information related to the association relationship between the plurality of second entities; A knowledge graph is constructed based on the entity extraction results.

5. The method according to claim 4, characterized in that The performing entity extraction on the plurality of knowledge texts to obtain entity extraction results includes: Generate a second prompt word; wherein the second prompt word includes the plurality of knowledge texts and information for indicating entity extraction from the plurality of knowledge texts; The second prompt word is sent to a second language model, and an entity extraction result returned by the second language model is received.

6. The method according to claim 4, characterized in that The performing entity extraction on the plurality of knowledge texts to obtain entity extraction results includes: Splitting the plurality of knowledge texts into a plurality of text units; Performing entity extraction on the multiple text units to obtain entity extraction sub-results for each of the text units; An entity extraction result is determined according to the entity extraction sub-result of each of the text units.

7. The method according to claim 6, characterized in that After splitting the plurality of knowledge texts into a plurality of text units, the method further comprises: Determining whether each of the plurality of text units has been subjected to entity extraction; The performing entity extraction on the multiple text units to obtain an entity extraction sub-result of each text unit includes: For a first text unit among the multiple text units, in response to entity extraction being performed on the first text unit, obtaining an entity extraction sub-result of the first text unit from a cache; In response to the first text unit not having undergone entity extraction, entity extraction is performed on the first text unit using a second language model to obtain an entity extraction sub-result of the first text unit; wherein the first text unit is any text unit among the multiple text units.

8. The method according to claim 6, characterized in that The determining of the entity extraction result according to the entity extraction sub-result of each text unit includes: Determining the similarity between the plurality of second entities indicated by the entity extraction sub-result of each of the text units; The entity extraction sub-results corresponding to the second entity whose similarity meets the set condition are merged to determine the entity extraction result.

9. The method according to claim 4, characterized in that The method further comprises: Determine a vector representation of first information related to a plurality of second entities in the entity extraction result, and store the vector representation of the first information related to the plurality of second entities in a vector database; and, The second information related to the association relationship between the plurality of second entities in the entity extraction result is stored in a relational database.

10. The method according to claim 4, characterized in that The first information related to the plurality of second entities includes entity names of the plurality of second entities and entity description information of the plurality of second entities, and the entity information related to the plurality of second entities also includes at least one of the following: entity identifiers of the plurality of second entities and entity types of the plurality of second entities; The second information related to the association relationship between the multiple second entities includes: information used to indicate two of the second entities that have an association relationship, and the second information related to the association relationship between the multiple second entities also includes at least one of the following: information used to indicate the relationship type of the two second entities that have an association relationship, information used to describe the association relationship between the two second entities that have an association relationship, and information used to indicate the relationship strength of the two second entities that have an association relationship.

11. A reply generating device, characterized in that: The device comprises: An acquisition module, used for acquiring the first question text; An extraction module, used for extracting at least one first entity from the first question text; A query module, configured to query from a pre-built knowledge graph to obtain a plurality of target entities related to the at least one first entity; A generating module, configured to generate a first prompt word; wherein the first prompt word includes the first question text, entity information related to the plurality of target entities, and information for indicating a reply to the first question text; The reply module is used to send the first prompt word to the first language model and receive a reply text of the first question text returned by the first language model.

12. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct an electronic device to execute the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product comprises computer readable instructions for implementing the method according to any one of claims 1 to 10.

Images