Text reply method and device, electronic equipment and storage medium

Abstract

The invention discloses a text reply method, belongs to the technical field of computers, and is used for accurately answering a query text input by a user. The method comprises the following steps: acquiring a query text input by a target user, and determining a query keyword based on the query text; a corresponding relation between preset first entities is determined, a knowledge graph is generated based on the first entities and the corresponding relation, the first entities comprise one or more of a sample article, a sample text segment and a sample keyword, the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample article. The sample keyword is a keyword in the sample text segment; performing similarity query on the query keyword and each first entity in the knowledge graph to obtain a query result; and inputting the query text and the query result into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a text reply method, device, electronic device and storage medium. Background Art

[0002] Large Language Model (LLM) is an artificial intelligence system that uses deep learning technology and a large amount of text data to train, and is specially designed to understand, generate and respond to natural language. These models learn the structure and usage of language by analyzing a large amount of text data, so that they can perform various language-related tasks, such as machine translation, text summarization, question-answering systems, writing assistants, code generation, conversational robots, etc. However, the breadth of the model's knowledge is limited by the training data set. When it does not have a certain knowledge scenario, it will cause hallucination problems and cannot provide a good response to the query text entered by the user.

[0003] Therefore, a text reply method is needed that can accurately reply to the query text input by the user. Summary of the invention

[0004] The embodiment of the present application provides a text reply method, which can accurately reply to the query text input by the user.

[0005] In a first aspect, an embodiment of the present application provides a text reply method, the method comprising: obtaining a query text input by a target user, and determining a query keyword based on the query text; determining a correspondence between preset first entities, and generating a knowledge graph based on the first entities and the correspondence, the first entity comprising one or more of a sample article, a sample text segment, and a sample keyword, the sample text segment being a text segment in the sample article, and the sample keyword being a keyword in the sample text segment; performing a similarity query on the query keyword and each of the first entities in the knowledge graph to obtain a query result; inputting the query text and the query result into a pre-trained large language model so that the pre-trained large language model outputs a reply text.

[0006] In a second aspect, an embodiment of the present application provides a text reply device, which includes: a first acquisition module, used to obtain a query text input by a target user, and determine a query keyword based on the query text; a first generation module, used to determine the correspondence between preset first entities, and generate a knowledge graph based on the first entities and the correspondence, the first entity includes one or more of a sample article, a sample text segment and a sample keyword, the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment; a first query module, used to perform a similarity query on the query keyword and each of the first entities in the knowledge graph to obtain a query result; a first input module, used to input the query text and the query result into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0010] In an embodiment of the present application, a query text input by a target user is obtained, and a query keyword is determined based on the query text; a correspondence between preset first entities is determined, and a knowledge graph is generated based on the first entities and the correspondence, wherein the first entity includes one or more of a sample article, a sample text segment, and a sample keyword, wherein the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment; a similarity query is performed between the query keyword and each of the first entities in the knowledge graph to obtain a query result; the query text and the query result are input into a pre-trained large language model so that the pre-trained large language model outputs a reply text that can accurately answer the inquiry text input by the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a flowchart of a text reply method provided in an embodiment of the present application; Figure 2 It is a flowchart of a second text reply method provided in an embodiment of the present application; Figure 3 It is a flowchart of a third text reply method provided in an embodiment of the present application; Figure 4 It is a flowchart of the fourth text reply method provided in the embodiment of the present application; Figure 5 It is a structural schematic diagram of a text reply device provided in an embodiment of the present application; Figure 6 It is a structural schematic diagram of a text reply device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0013] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0014] The text reply method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0015] Figure 1 A text reply method provided by an embodiment of the present invention is shown. The method can be executed by an electronic device, and the electronic device may include: a server and / or a terminal device, wherein the terminal device may be, for example, a vehicle-mounted terminal or a mobile phone terminal. In other words, the method can be executed by software or hardware installed in the text reply device, and the method includes the following steps: Step 102: Acquire the query text input by the target user, and determine the query keyword based on the query text.

[0016] The text reply system obtains the query text input by the target user, and determines the query keywords included therein based on the query text, wherein the query text is the text input by the target user and includes the question to be queried, and the number of query keywords can be one or more. In other words, the text reply system performs text recognition on the query text to determine the keywords or key words in the question to be queried by the user.

[0017] Specifically, the text reply system can directly receive the query text entered by the target user, and can also receive the query voice sent by the target user, and then obtain the query text by performing text recognition operations on the query voice. After obtaining the query text, the text reply system can recognize the query text through a pre-trained large language model to determine the keywords in the query text. For example, when the text entered by the user is "Use detailed data to explain which operators have higher revenue than Company A in 2023", the text reply system can determine the keywords "Company A", "Revenue" and "Operator" through the large language model.

[0018] Step 104: Determine the corresponding relationships between the preset first entities, and generate a knowledge graph based on the first entities and the corresponding relationships.

[0019] The first entity includes one or more of a sample article, a sample text segment and a sample keyword. The sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment.

[0020] The text reply system obtains a plurality of pre-set first entities, where the first entity is one or more of a sample article, a sample text segment, and a sample keyword, wherein the sample text segment is a text segment obtained from a sample article, and the sample keyword is a keyword obtained from a sample text segment. After the text reply system obtains a plurality of pre-set first entities, it determines the corresponding relationship between the first entities, and generates a knowledge graph based on the corresponding relationship between the plurality of first entities and the first entities.

[0021] Specifically, in the process of constructing the knowledge graph, the text reply system extracts knowledge from sample articles, sample text segments, and sample keywords, performs knowledge fusion processing on them, and determines the relationship between sample articles and sample keywords, the relationship between sample articles and sample text segments, and the relationship between sample text segments and sample keywords (i.e., the sample keywords are only used as the "object" in the knowledge graph's "subject", "predicate", and "object"), and finally generates the knowledge graph through ontology construction, graph construction (including but not limited to RDF), and pattern engineering. Furthermore, the corresponding relationship between the first entities determined by the text reply system can be a pre-set corresponding relationship, or it can be obtained by identifying sample articles and sample text segments using a pre-trained attribute relationship recognition model. In this way, the knowledge graph constructed not only retains text information and semantic information, but also includes text relationship information.

[0022] More specifically, when the text reply system generates a knowledge graph based on the correspondence between first entities and first entities, it can first generalize each first entity, then determine whether the same first entity exists in the multiple first entities after generalization, merge the same first entities, and finally generate a knowledge graph based on the multiple first entities after merging the same first entities and the corresponding relationship. The generalization process can discover existing entity nodes so that they can be merged. Map triples to the knowledge graph. Using mature entity linking and relationship mapping techniques, map triples to existing fixed schema graphs. For partially mapped or completely mismatched triples, an open schema knowledge graph is constructed, and finally the two types of knowledge graphs are merged to obtain a flexible open knowledge graph and save the knowledge graph in the graph database Neo4j.

[0023] Step 106: Perform a similarity query on the query keyword and each of the first entities in the knowledge graph to obtain a query result.

[0024] After determining the query keyword and the knowledge graph, the text reply system can perform a similarity query between the query keyword and each first entity in the knowledge graph and obtain the query result. When there are multiple query keywords, the text reply system can perform a similarity query based on each query keyword and obtain the query result corresponding to each query keyword.

[0025] Specifically, when the text reply system performs a query based on a query keyword, it can query whether each first entity in the knowledge graph has the same text content as the query keyword, thereby determining the query result. It can also determine the query result through the text vector corresponding to the query keyword and the text vector corresponding to each first entity. For example, the text reply system can determine the first text vector of the query keyword and the second text vector corresponding to each first entity in the knowledge graph, and then perform similarity matching on the first text vector and each second text vector, and determine the second text with a similarity greater than a preset threshold as a similar text vector, and finally determine the first entity corresponding to the similar text vector to determine the query result. The vectorized identification model includes but is not limited to language models such as the bge1.5 model and the gte-qwen7B large model, and the similarity matching algorithm includes but is not limited to cosine similarity, ANN and other algorithms.

[0026] More specifically, after the text reply system determines in the knowledge graph the first entity whose similarity with the query keyword meets preset conditions, it can obtain the attribute relationship of the first entity and the first entity connected to the determined attribute relationship (i.e., the context information of the first entity in the knowledge graph), and finally, based on the first entity determined by the similarity query, the attribute relationship corresponding to the first entity, and the first entity corresponding to the attribute relationship, determine it as the query result, that is, determine the context information of the first entity determined by the similarity query as the query result.

[0027] Step 108: Input the query text and the query result into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

[0028] After obtaining the query results, the text reply system inputs the query text and the query results into the pre-trained large language model so that the large language model outputs a reply text based on the query text and the query results. The pre-trained large language model can generate a reply text based on the query results determined by the text reply system of the query text input by the target user, that is, the large language model learns relevant knowledge about the keywords in the query text according to the query results determined by the text reply system, and then generates a reply text to reply to the target user based on the relevant knowledge and the query text.

[0029] In the embodiment of the specification, a query text input by a target user is obtained, and a query keyword is determined based on the query text; a correspondence between preset first entities is determined, and a knowledge graph is generated based on the first entities and the correspondence, wherein the first entity includes one or more of a sample article, a sample text segment, and a sample keyword, wherein the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment; a similarity query is performed between the query keyword and each of the first entities in the knowledge graph to obtain a query result; the query text and the query result are input into a pre-trained large language model so that the pre-trained large language model outputs a reply text that can accurately answer the query text input by the user.

[0030] In one implementation, the determining of the corresponding relationships between the preset first entities and generating a knowledge graph based on the first entities and the corresponding relationships (step 104) may include executing steps A1-A4: Step A1: Obtain the preset sample article, and perform segmentation processing on the sample article to obtain the corresponding sample text segment.

[0031] The text reply system obtains a plurality of preset sample articles, and then performs segmentation processing on each sample article to obtain a sample text segment corresponding to each sample article.

[0032] Specifically, when the text reply system performs segment slicing processing on the sample article, the sample article can be sliced ​​according to a preset length, and the sample article can also be sliced ​​according to punctuation mark information in the sample article.

[0033] Step A2: Perform keyword recognition on the sample text segment to obtain the corresponding sample keyword.

[0034] After determining the multiple sample text segments corresponding to the multiple sample articles, the text reply system performs keyword recognition on each sample text segment to obtain text keywords corresponding to each sample text segment.

[0035] Specifically, when the text reply system performs keyword recognition on a sample text segment, the text reply system can recognize the sample text segment according to a preset keyword set, thereby determining the keywords in the preset keyword set included in the sample text segment, and determining the keywords included in the sample text segment as text keywords.

[0036] Step A3: Determine a first attribute relationship between the sample article and the sample text segment, a second attribute relationship between the sample text segment and the sample keyword, and a third attribute relationship between the sample article and the sample keyword based on the pre-trained large language model.

[0037] After determining the sample articles, sample text segments and sample keywords, the text reply system determines the first attribute relationship between the sample articles and the sample text segments, the second attribute relationship between the sample text segments and the sample keywords, and the third attribute relationship between the sample articles and the sample keywords through a pre-trained large language model.

[0038] Specifically, before inputting entities into LLM, the entities need to be aligned and standardized first to ensure that the entity representation is consistent, and then select the entity pairs that need to establish relationships according to the needs in the knowledge graph. An entity pair can be a combination of any two entities, and then a template or prompt is designed to format the input. This prompt can guide LLM to understand the task and generate the required output. For example, "What is the relationship between [Entity1] and [Entity2]?", "What is the role of [Entity1] in [Entity2]?", etc. are used as prompts. Finally, the formatted entity pairs are input into LLM. The model will infer the relationship between entities based on its training data and language understanding ability, that is, the first attribute relationship, the second attribute relationship, or the third attribute relationship. After determining the corresponding attribute relationship between the first entities, the text reply system can also verify the relationship generated by the large language model (LLM) to ensure its accuracy and rationality, and monitor its impact on downstream tasks.

[0039] More specifically, when the text reply system determines the correspondence between the first entities, it can not only determine the attribute relationship between the sample article and the multiple sample text segments it includes, and the attribute relationship between the sample text segment and the multiple sample keywords it includes, but also can determine the correspondence between different sample articles, between different sample segments, between any sample article and any sample text segment, between any sample article and any sample keyword, and between any sample text segment and any sample keyword.

[0040] Step A4: Generate a knowledge graph based on the first attribute relationship, the second attribute relationship and the third attribute relationship.

[0041] After determining the first attribute relationship, the second attribute relationship and the third attribute relationship, the text reply system is based on the knowledge graph of the multiple first entities and the determined first attribute relationship, the second attribute relationship and the third attribute relationship.

[0042] Specifically, based on the three parts of sample articles, sample text segments and sample keywords as knowledge entities, and as nodes of the knowledge graph, they are also the head and tail parts of the triple, where the sample keyword only appears as the tail. Through such construction, the relationship between sample articles and sample articles, the relationship between sample articles and sample text segments, the relationship between sample text segments and sample text segments, the relationship between sample articles and sample keywords, and the relationship between sample text segments and sample keywords are constructed, and then the knowledge graph is constructed based on the determined relationships.

[0043] Figure 2 is a flow chart of a second text reply method provided in an embodiment of this specification, such as Figure 2 As shown, the schematic diagram includes: Step 202: Acquire the query text input by the target user, and determine the query keyword based on the query text.

[0044] Step 204: Obtain the preset sample article, and perform segmentation processing on the sample article to obtain the corresponding sample text segment.

[0045] Step 206: Perform keyword recognition on the sample text segment to obtain the corresponding sample keyword.

[0046] Step 208: Determine a first attribute relationship between the sample article and the sample text segment, a second attribute relationship between the sample text segment and the sample keyword, and a third attribute relationship between the sample article and the sample keyword based on the pre-trained large language model.

[0047] Step 210: Generate a knowledge graph based on the first attribute relationship, the second attribute relationship and the third attribute relationship.

[0048] Step 212: Perform a similarity query on the query keyword and each first entity in the knowledge graph to obtain a query result.

[0049] Among them, the first entity is a knowledge entity in the knowledge graph.

[0050] Step 214: Input the query text and the query result into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

[0051] In the embodiments of the specification, by obtaining sample text segments included in sample articles, sample keywords included in sample text segments, and obtaining the attribute relationships between sample articles, sample text segments, and sample keywords, a knowledge graph is generated based on the sample articles, sample text segments, sample keywords, and attribute relationships, so that the constructed knowledge graph not only retains text information and semantic information, but also includes text relationship information, increases the knowledge included in the knowledge graph, and enables the large language model to learn more knowledge related to the query text, thereby improving the accuracy of responding to the user's query text.

[0052] In one implementation, before determining the corresponding relationships between the preset first entities and generating a knowledge graph based on the first entities and the corresponding relationships (step 104), steps B1-B4 may also be performed: Step B1: Acquire multimodal data.

[0053] The multimodal data includes one or more of video data, audio data and picture data.

[0054] The text reply system obtains multimodal data, which includes one or more of video data, audio data and picture data, that is, multimodal data refers to data in multiple forms including text, image and voice.

[0055] Step B2: performing speech recognition on the video data and the audio data in the multimodal data to obtain a first text.

[0056] Since the multimodal data obtained by the text reply system is not uniform in form and content, it is necessary to align the multimodal data when introducing it into the knowledge graph, that is, to describe all the data in a unified format in a certain dimension, so as to normalize the multimodal data.

[0057] Therefore, after the text reply system acquires the multimodal data, the text reply system can normalize the video data, image data, and voice data in the multimodal data. Furthermore, the text reply system performs voice recognition on the video data and voice data that also include voice content to obtain a first text, that is, the first text is a text determined based on the voice content in the video data and voice data.

[0058] Step B3: extracting features from the video data and the picture data in the multimodal data to obtain a second text.

[0059] After the text reply system acquires the multimodal data, the text reply system can normalize the video data and the image data in the multimodal data, that is, extract features from the video data and the image data that also include image content to obtain the second text. In other words, the second text is a text determined based on the image content in the video data and the image data.

[0060] Specifically, the text reply system can perform image description on the video data and picture data through a pre-trained image description model, and generate a second text based on the image description content. When performing image description on the video data, the text reply system can determine each video frame in the video data, regard each video frame as a picture data and perform image description on it, and then determine the second text corresponding to the video data. Furthermore, the video data may include multiple video frames, and thus a large amount of text will be generated when performing image description and generating text for each video frame data, and since the content similarity between two adjacent video frames is high, the similarity of the description text generated is also high, so the text reply system can deduplicate the generated description and determine the second text based on the deduplicated video frame description text.

[0061] Step B4: determining the sample article based on the first text and the second text.

[0062] After determining the first text and the second text, the text reply system determines the sample article based on the first text and the second text obtained by identifying the multimodal data. Specifically, when the multimodal data is video data, the text reply system performs speech recognition and feature extraction operations on the video data respectively, and determines the sample article based on the first text and the second text obtained. When the multimodal data is voice data, the text reply system performs voice recognition on the voice data and determines the sample article based on the obtained first text. When the multimodal data is image data, the text reply system performs feature extraction on the image data and determines the sample article based on the obtained second text.

[0063] Traditional text reply systems only include text data, which will lose a large part of the diverse data. Therefore, in the embodiment, multimodal data is introduced to expand the boundaries of the knowledge graph, so that the generated knowledge graph is not limited to text input, but can also include voice, picture and video input, expanding the scope of knowledge included in the knowledge graph.

[0064] In one implementation, the feature extraction is performed on the video data and the picture data in the multimodal data to obtain the second text (step B3), and steps C1-C3 may be performed: Step C1: acquiring data of each video frame in the video data, and performing feature recognition based on the data of each video frame to obtain a first feature.

[0065] After acquiring the video data and / or picture data included in the multimodal data, the text reply system can acquire the data of each video frame and / or picture data in the video data, and perform feature recognition based on the acquired video frame data and / or picture data to obtain the first feature included in the video frame data, that is, the first feature is used to characterize the features of each video frame and / or picture data in the video data and / or picture data.

[0066] Specifically, the text reply system includes a dual video encoding module that captures visual information through spatial modeling and local temporal modeling, allowing image and video inputs to share a standard Transformer module to model spatial information, and use an additional local temporal modeling module to handle video-related tasks. Images and video frames are segmented into a series of visual tokens, and a standard Transformer module is used for spatial information, including a self-attention layer and a feedforward network (FFN).

[0067] Step C2: Obtaining time feature information between the video frames in the video data, and determining it as the second feature.

[0068] After determining the first feature of each video frame of the video data, the text reply system can also determine the time feature information between each video frame in the video data, and determine the second feature based on the determined time feature information, that is, the second feature is used to characterize the time feature between each video frame in the video data.

[0069] In order to capture the temporal dynamics in video data, the text reply system can introduce a local temporal modeling module (Local Temporal Modeling Module, LTM). This module is specifically used to process the temporal relationship between video frames. Multi-Group Fusion is used to learn rich temporal information from different temporal positions.

[0070] The local time modeling module is as follows:

[0071]

[0072]

[0073] Among them, Vn represents the visual features of the nth layer, LTM is the local temporal modeling module, SA is the self-attention layer, FFN is the feedforward network, and LN is the layer normalization.

[0074] The multiple groups of fusion modules are as follows:

[0075]

[0076] in and is a linear transformation function, is a learnable temporal relation parameter, G represents the number of groups, and Concat represents the connection function.

[0077] In addition, the text reply system can also include a text encoding module, a general layer module and a multi-module fusion module, wherein the text encoder module uses BERT as a text encoder to convert the input text into text embeddings, the general layer module acts as a bridge between different modalities, and projects the visual and language modalities into a shared semantic space while retaining the original representations of different modalities, and the multimodal fusion module uses a cross-attention layer to fuse the general visual representation and the original fine-grained visual representation to effectively capture the cross-modal interaction between the visual and language modalities.

[0078] Step C3: Generate text based on the first feature and the second feature to obtain the second text.

[0079] After determining the first feature and the second feature, the text reply system generates text based on the determined first feature and the second feature, thereby obtaining a description text of the multimodal data, thereby obtaining a text describing the picture in the video data and / or picture data, that is, the second text.

[0080] Figure 3 is a flowchart of a third text reply method provided in an embodiment of this specification, such as Figure 3 As shown, the schematic diagram includes: Step 302: Acquire the query text input by the target user, and determine the query keyword based on the query text.

[0081] Step 304: Acquire multimodal data.

[0082] The multimodal data includes one or more of video data, audio data and picture data.

[0083] Step 306: Perform speech recognition on the video data and the audio data in the multimodal data to obtain a first text.

[0084] Step 308: Acquire data of each video frame in the video data, and perform feature recognition based on the data of each video frame to obtain a first feature.

[0085] Step 310: Obtain time feature information between the video frames in the video data, and determine it as the second feature.

[0086] Step 312: Generate text based on the first feature and the second feature to obtain the second text.

[0087] Step 314: Determine a sample article based on the first text and the second text.

[0088] Step 316: Determine the corresponding relationships between the preset first entities, and generate a knowledge graph based on the first entities and the corresponding relationships.

[0089] The first entity includes one or more of a sample article, a sample text segment and a sample keyword. The sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment.

[0090] Step 318: Perform a similarity query on the query keyword and each of the first entities in the knowledge graph to obtain a query result.

[0091] Step 320: Input the query text and the query result into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

[0092] In the embodiments of the specification, multimodal data is obtained, and speech recognition and feature extraction operations are performed on the video data, image data, and speech data in the multimodal data, thereby obtaining a descriptive text (i.e., a sample article) corresponding to the multimodal data, and then a knowledge graph is generated based on the determined sample article. In this way, by introducing multimodal data to expand the boundaries of the knowledge graph, the generated knowledge graph is not limited to text input, but can also include speech, image, and video input, thereby expanding the scope of knowledge included in the knowledge graph.

[0093] In one implementation, the query keyword is subjected to a similarity query with each of the first entities in the knowledge graph to obtain a query result (step 106), and steps D1-D3 may be performed: Step D1: Determine a second entity corresponding to the query keyword in the knowledge graph.

[0094] When the text reply system performs a similarity query based on the query keyword and each first entity in the knowledge graph, the text reply system can first determine the second entity corresponding to the query keyword in the knowledge graph.

[0095] Specifically, the first entity in the knowledge graph includes sample articles, sample text segments and sample keywords. Therefore, when performing similarity identification, it is possible to determine whether the sample articles, sample text segments and sample keywords contain query keywords or keywords whose similarity with the query keywords is greater than a preset threshold, and determine the knowledge entity corresponding to the determined keyword as the second entity.

[0096] Step D2: Determine the fourth attribute relationship between each of the first entities and the second entities in the knowledge graph through the preset large language model.

[0097] After determining the second entity, the text reply system determines the attribute relationship between each first entity and the second entity in the knowledge graph through a preset large language model, i.e., the fourth attribute relationship. In other words, the text reply system determines whether each knowledge entity in the knowledge graph has an attribute relationship with the second entity through a preset large language model, and if so, determines the attribute relationship between the two.

[0098] Specifically, the knowledge graph pre-generated by the text reply system can continuously expand knowledge according to actual needs (such as adding new sample articles), which may cause the correspondence between the first entities in the knowledge graph to be incomplete. Therefore, after determining the second entity, the text reply system can not only directly determine that the attribute relationship corresponding to the second entity in the knowledge graph is the fourth attribute relationship, but also determine whether there is an attribute relationship between the first entities and the second entity in the knowledge graph and determine the existing attribute relationship based on the preset large language model, and then determine the attribute relationship determined by the large language model as the fourth attribute relationship.

[0099] Step D3: Determine the query result based on the fourth attribute relationship.

[0100] After determining the fourth attribute relationship of the second entity, the text reply system determines the query result based on the second entity, the fourth attribute relationship corresponding to the second entity, and the knowledge entities corresponding to each fourth attribute relationship.

[0101] In this embodiment, there is no need to construct all the knowledge in the knowledge graph when the text reply system is initialized. Instead, the details are constructed on demand based on the user's query text, thereby saving the initialization time of the text reply system and enabling the text reply system to be put online more quickly.

[0102] In one implementation, steps E1-E3 may also be performed: Step E1: Obtain the feedback result returned by the target user.

[0103] The feedback result is used to describe the accuracy of the reply text.

[0104] After the text reply system sends the reply text to the target user, the text reply system can also receive a feedback result returned by the target user, where the feedback result is information generated by the target user based on the reply text.

[0105] Specifically, after the target user receives the reply text returned by the text reply system, it can determine the relevance and / or accuracy of the reply text to the query text, and generate a feedback result for characterizing the relevance and / or accuracy of the reply text to the query text, and then send the feedback result to the text reply system. In other words, the feedback result is the return information fed back by the target user, which is used to characterize the accuracy or inaccuracy of the reply text.

[0106] Step E2: When the feedback result indicates that the reply text is inaccurate, determine a third entity included in the reply text and a fifth attribute relationship corresponding to the third entity.

[0107] When the feedback result fed back by the target user indicates that the reply text is accurate, the text reply system can determine that the text replied to the target user is the correct text, that is, the knowledge included in the knowledge graph is the correct knowledge. When the feedback result fed back by the target user indicates that the reply text is inaccurate, the text reply system can determine that the text replied to the target user is the incorrect text, that is, the knowledge included in the knowledge graph is the incorrect knowledge.

[0108] Specifically, when the feedback result fed back by the target user indicates that the reply text is incorrect, the text reply system can determine the third entity included therein and the fifth attribute relationship corresponding to the third entity based on the reply text, that is, determine the third entity included in the reply text and the fifth attribute relationship corresponding to the third entity in the knowledge graph.

[0109] Step E3: Delete the fifth attribute relationship corresponding to the third entity in the knowledge graph.

[0110] After determining the third entity included in the reply text and the fifth attribute relationship corresponding to the third entity, the text reply system can delete the fifth attribute relationship corresponding to the third entity in the knowledge graph. In other words, the text reply system can receive attribute relationships that are incorrectly recognized by the large language model and are fed back by the target user, and delete the incorrect attribute information in the knowledge graph, thereby continuously updating the content of the knowledge graph according to the feedback results of the target user, and dynamically constructing and updating the content of the knowledge graph in a reasonable and well-founded manner.

[0111] Figure 4 is a flowchart of a fourth text reply method provided in an embodiment of this specification, such as Figure 4As shown, the schematic diagram includes: Step 402: Acquire the query text input by the target user, and determine the query keyword based on the query text.

[0112] Step 404: Determine the corresponding relationships between the preset first entities, and generate a knowledge graph based on the first entities and the corresponding relationships.

[0113] The first entity includes one or more of a sample article, a sample text segment and a sample keyword. The sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment.

[0114] Step 406: Determine a second entity corresponding to the query keyword in the knowledge graph.

[0115] Step 408: Determine the fourth attribute relationship between each of the first entities and the second entities in the knowledge graph through the preset large language model.

[0116] Step 410: Determine a query result based on the fourth attribute relationship.

[0117] Step 412: Input the query text and the query result into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

[0118] Step 414: Obtain the feedback result returned by the target user, where the feedback result is used to describe the accuracy of the reply text.

[0119] Step 416: When the feedback result indicates that the reply text is inaccurate, determine a third entity included in the reply text and a fifth attribute relationship corresponding to the third entity.

[0120] Step 418: Delete the fifth attribute relationship corresponding to the third entity in the knowledge graph.

[0121] In the embodiments of the specification, by determining the query keywords in the query text, and determining the second entity in the knowledge graph whose similarity with the query keywords is greater than a preset threshold, and then determining the attribute relationship between each first entity and the second entity based on the large language model, and then determining the query result based on the attribute relationship and generating a reply text, the attribute relationship between the first entities in the knowledge graph can be continuously improved during the user query process, reducing the generation time of the knowledge graph, and updating the attribute relationship between the first entities in the knowledge graph through the feedback results of the target user, which can realize the continuous updating of the knowledge graph, improve the accuracy and completeness of the knowledge in the knowledge graph, and accurately reply to the query text input by the target user.

[0122] It should be noted that the text reply method provided in the embodiment of the present application can be executed by a text reply device or a control module in the text reply device for executing the text reply method. In the embodiment of the present application, the text reply device provided in the embodiment of the present application is described by taking the execution of the text reply method by the text reply device as an example.

[0123] Figure 5 Schematic diagram of the structure of a text reply device according to an embodiment of the present invention. Figure 5 As shown, the text reply device includes: a first acquisition module 502 , a first generation module 504 , a first query module 506 , and a first input module 508 .

[0124] A first acquisition module 502 is used to acquire a query text input by a target user and determine a query keyword based on the query text; A first generating module 504 is used to determine the corresponding relationship between the preset first entities, and generate a knowledge graph based on the first entities and the corresponding relationship, wherein the first entity includes one or more of a sample article, a sample text segment and a sample keyword, wherein the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment; A first query module 506, configured to perform a similarity query between the query keyword and each of the first entities in the knowledge graph to obtain a query result; The first input module 508 is used to input the query text and the query result into the pre-trained large language model, so that the pre-trained large language model outputs a reply text.

[0125] The text reply in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0126] The text reply device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0127] The text reply device provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented in the method embodiment are not described here.

[0128] Based on the same technical concept, an embodiment of the present application further provides an electronic device, which is used to execute the above-mentioned text reply method. Figure 6 A schematic diagram of the structure of an electronic device for implementing various embodiments of the present application. The electronic device may have relatively large differences due to different configurations or performances, and may include a processor (processor) 602, a communication interface (Communications Interface) 604, a memory (memory) 606, and a communication bus 608, wherein the processor 602, the communication interface 604, and the memory 606 communicate with each other through the communication bus 608. The processor 602 may call a computer program stored in the memory 606 and executable on the processor 602 to perform the following steps: Acquire a query text input by a target user, and determine a query keyword based on the query text; Determine the correspondence between preset first entities, and generate a knowledge graph based on the first entities and the correspondence, wherein the first entity includes one or more of a sample article, a sample text segment, and a sample keyword, wherein the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment; Perform a similarity query between the query keyword and each of the first entities in the knowledge graph to obtain a query result; The query text and the query result are input into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

[0129] In one implementation, determining the corresponding relationships between the preset first entities and generating a knowledge graph based on the first entities and the corresponding relationships includes: Acquire the preset sample article, and perform segmentation processing on the sample article to obtain the corresponding sample text segment; Perform keyword recognition on the sample text segment to obtain the sample keyword corresponding thereto; Determine, based on the pre-trained large language model, a first attribute relationship between the sample article and the sample text segment, a second attribute relationship between the sample text segment and the sample keyword, and a third attribute relationship between the sample article and the sample keyword; A knowledge graph is generated based on the first attribute relationship, the second attribute relationship and the third attribute relationship.

[0130] In one implementation, before determining the corresponding relationships between the preset first entities and generating a knowledge graph based on the first entities and the corresponding relationships, the method further includes: Acquire multimodal data, where the multimodal data includes one or more of video data, audio data, and picture data; Performing speech recognition on the video data and the audio data in the multimodal data to obtain a first text; Performing feature extraction on the video data and the picture data in the multimodal data to obtain a second text; The sample article is determined based on the first text and the second text.

[0131] In one implementation, the extracting features of the video data and the picture data in the multimodal data to obtain the second text includes: Acquire data of each video frame in the video data, and perform feature recognition based on the data of each video frame to obtain a first feature; Acquire time feature information between the video frames in the video data, and determine it as a second feature; Text is generated based on the first feature and the second feature to obtain the second text.

[0132] In one implementation, performing a similarity query on the query keyword and each of the first entities in the knowledge graph to obtain a query result includes: Determining a second entity corresponding to the query keyword in the knowledge graph; Determine, by using the preset large language model, a fourth attribute relationship between each of the first entity and the second entity in the knowledge graph; The query result is determined based on the fourth attribute relationship.

[0133] In one implementation, the method further includes: Obtaining a feedback result returned by the target user, wherein the feedback result is used to describe the accuracy of the reply text; When the feedback result indicates that the reply text is inaccurate, determining a third entity included in the reply text and a fifth attribute relationship corresponding to the third entity; The fifth attribute relationship corresponding to the third entity in the knowledge graph is deleted.

[0134] The specific execution steps can refer to the various steps of the above-mentioned text reply method embodiment, and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0135] It should be noted that the electronic devices in the embodiments of the present application include: servers, terminals, or other devices except terminals.

[0136] The above electronic device structure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the input unit may include a graphics processing unit (GPU) and a microphone, and the display unit may be configured with a display panel in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit includes a touch panel and at least one of other input devices. The touch panel is also called a touch screen. Other input devices may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0137] The memory can be used to store software programs and various data. The memory may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory may include a volatile memory or a non-volatile memory, or the memory may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM).

[0138] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor.

[0139] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned text reply method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0140] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0141] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned text reply method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0142] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0143] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0145] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A text reply method, comprising: Acquire a query text input by a target user, and determine a query keyword based on the query text; Determine the correspondence between preset first entities, and generate a knowledge graph based on the first entities and the correspondence, wherein the first entity includes one or more of a sample article, a sample text segment, and a sample keyword, wherein the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment; Perform a similarity query between the query keyword and each of the first entities in the knowledge graph to obtain a query result; The query text and the query result are input into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

2. According to the method of claim 1, determining the corresponding relationship between the preset first entities and generating a knowledge graph based on the first entities and the corresponding relationship comprises: Acquire the preset sample article, and perform segmentation processing on the sample article to obtain the corresponding sample text segment; Perform keyword recognition on the sample text segment to obtain the sample keyword corresponding thereto; Determine, based on the pre-trained large language model, a first attribute relationship between the sample article and the sample text segment, a second attribute relationship between the sample text segment and the sample keyword, and a third attribute relationship between the sample article and the sample keyword; A knowledge graph is generated based on the first attribute relationship, the second attribute relationship and the third attribute relationship.

3. According to the method of claim 1, before determining the corresponding relationship between the preset first entities and generating a knowledge graph based on the first entities and the corresponding relationship, the method further comprises: Acquire multimodal data, where the multimodal data includes one or more of video data, audio data, and picture data; Performing speech recognition on the video data and the audio data in the multimodal data to obtain a first text; Performing feature extraction on the video data and the picture data in the multimodal data to obtain a second text; The sample article is determined based on the first text and the second text.

4. The method according to claim 3, wherein the step of extracting features from the video data and the picture data in the multimodal data to obtain the second text comprises: Acquire data of each video frame in the video data, and perform feature recognition based on the data of each video frame to obtain a first feature; Acquire time feature information between the video frames in the video data, and determine it as a second feature; Text is generated based on the first feature and the second feature to obtain the second text.

5. According to the method of claim 1, performing a similarity query between the query keyword and each of the first entities in the knowledge graph to obtain a query result comprises: Determining a second entity corresponding to the query keyword in the knowledge graph; Determine, by using the preset large language model, a fourth attribute relationship between each of the first entity and the second entity in the knowledge graph; The query result is determined based on the fourth attribute relationship.

6. The method according to claim 1, further comprising: Obtaining a feedback result returned by the target user, wherein the feedback result is used to describe the accuracy of the reply text; When the feedback result indicates that the reply text is inaccurate, determining a third entity included in the reply text and a fifth attribute relationship corresponding to the third entity; The fifth attribute relationship corresponding to the third entity in the knowledge graph is deleted.

7. A text reply device, comprising: A first acquisition module, used to acquire a query text input by a target user, and determine a query keyword based on the query text; A first generating module, configured to determine a correspondence between preset first entities, and generate a knowledge graph based on the first entities and the correspondence, wherein the first entity includes one or more of a sample article, a sample text segment, and a sample keyword, wherein the sample text segment is a text segment in the sample article, and the sample keyword is a keyword in the sample text segment; A first query module, used to perform a similarity query between the query keyword and each of the first entities in the knowledge graph to obtain a query result; The first input module is used to input the query text and the query result into a pre-trained large language model, so that the pre-trained large language model outputs a reply text.

8. A computer device, characterized in that: The device comprises: Processor; and A memory arranged to store computer executable instructions, wherein the executable instructions are configured to be executed by the processor, and the executable instructions include instructions for executing the steps in the text reply method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is used to store computer-executable instructions, and the executable instructions enable a computer to execute the text reply method according to any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the text reply method according to any one of claims 1 to 6.

Images