AI large model multi-round conversation deduplication method, device and electronic equipment

Abstract

The present disclosure provides an AI large model multi-round conversation deduplication method, device and electronic equipment. The AI large model multi-round conversation deduplication method of the present disclosure comprises: when a first prompt submitted by a first user is monitored, performing a repetition detection on the first prompt according to historical question and answer data of the first user to determine whether the first prompt is repeated; when the first prompt is not repeated, transmitting the first prompt to an AI large model and returning a first reply provided by the AI large model for the first prompt to the first user; when the first prompt is repeated, obtaining repeated reason information for the first prompt fed back by the first user through a man-machine conversation interface, and obtaining a second reply according to the repeated reason information and the first prompt and returning the second reply to the first user. The present disclosure can effectively solve the problem of repeated questions of the user in the man-machine conversation under the premise of lower consumption of computing resources.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence (AI) technology, and more particularly to a method, apparatus, and electronic device for deduplication in multi-turn dialogue of a large AI model. Background Technology

[0002] The application of large AI models is becoming increasingly widespread. However, in the multi-turn dialogues between users and large AI models in these applications, there are often a large number of inefficient or even invalid repetitive question-and-answer sessions. This not only wastes a lot of dialogue token resources and interferes with the large AI module's understanding of the user's intent, but also increases the interaction cost and reduces the user experience. Summary of the Invention

[0003] In view of this, this disclosure provides a method, apparatus and electronic device for deduplication of multi-turn dialogue in large AI models.

[0004] According to a first aspect of this disclosure, a method for deduplication in multi-turn dialogue of a large AI model is provided, the method comprising: When the first prompt submitted by the first user is detected, a duplicate detection is performed on the first prompt based on the first user's historical question and answer data to determine whether the first prompt is duplicated; When the first prompt is not repeated, the first prompt is passed through to the AI ​​big model and the first response provided by the AI ​​big model for the first prompt is returned to the first user; When the first prompt is repeated, the system obtains the reason information for the repetition of the first prompt from the first user through the human-computer dialogue interface, obtains the second response based on the reason information and the first prompt, and returns it to the first user.

[0005] In some embodiments of the first aspect of this disclosure, obtaining a second response based on the repetition reason information and the first prompt and returning it to the first user includes: If the duplicate reason information indicates that the content of the previous reply from the AI ​​big model is irrelevant to the user's needs, then the reply information in the question-and-answer pair in the historical question-and-answer data that is duplicated with the first prompt will be used as the second reply and returned to the first user. If the duplicate reason information indicates that the content of the previous reply from the AI ​​big model does not meet the user's needs, then a second prompt is generated based on the first prompt and the duplicate reason information, the reply corresponding to the second prompt is obtained from the AI ​​big model, and the reply corresponding to the second prompt is returned to the first user as the second reply.

[0006] In some embodiments of the first aspect of this disclosure, before returning the reply corresponding to the second prompt as the second reply to the first user, the method further includes: performing a duplicate detection on the reply corresponding to the second prompt based on the historical question and answer data to determine whether the reply corresponding to the second prompt is duplicated; The step of returning the reply corresponding to the second prompt as the second reply to the first user includes: when the reply corresponding to the second prompt is not repeated, then returning the reply corresponding to the second prompt as the second reply to the first user.

[0007] In some embodiments of the first aspect of this disclosure, the method further includes: generating a third prompt when the reply corresponding to the second prompt is repeated; the third prompt is used to indicate that the reply corresponding to the second prompt is repeated; querying the historical question-and-answer data for question-and-answer pairs whose reply information is repeated with the reply corresponding to the second prompt; sending the third prompt and the queried question-and-answer pairs to the AI ​​big model to obtain the AI ​​big model's reply to the third prompt; and returning the reply to the third prompt as the second reply to the first user.

[0008] In some embodiments of the first aspect of this disclosure, the method further includes: when the first prompt is repeated, if the reason information for the repetition is invalid and / or the waiting time exceeds a predetermined duration and the reason information for the repetition is not received, then a fourth prompt containing the first prompt is generated according to a predetermined configuration, a response to the fourth prompt is obtained from the AI ​​big model, and the response to the fourth prompt is returned to the first user as the second response.

[0009] In some embodiments of the first aspect of this disclosure, obtaining duplicate reason information from a first user in response to the first prompt through the human-computer dialogue interface includes: providing a duplicate reason prompt box on the human-computer dialogue interface, the duplicate reason prompt box containing multiple duplicate reason options; and obtaining duplicate reason information generated by the first user's operation on the duplicate reason options in the duplicate reason prompt box.

[0010] In some embodiments of the first aspect of this disclosure, the step of performing a duplicate detection on the first prompt based on the first user's historical question-and-answer data to determine whether the first prompt is duplicated includes: determining whether the first prompt is duplicated with the prompt information of the question-and-answer pairs in the historical question-and-answer data by calculating the semantic similarity between the first prompt and the prompt information of each question-and-answer pair in the historical question-and-answer data.

[0011] In some embodiments of the first aspect of this disclosure, the method further includes: collecting historical dialogue information of a first user and performing structured processing to generate historical question-and-answer data of the first user, and setting an index for each question-and-answer pair in the historical question-and-answer data, wherein the index is used to query information of the question-and-answer pairs in the historical question-and-answer data.

[0012] According to a second aspect of this disclosure, an AI large-scale model multi-turn dialogue deduplication device is provided, the AI ​​large-scale model multi-turn dialogue deduplication device comprising: The prompt detection unit is used to detect whether the first prompt submitted by the first user is repeated by performing a duplicate detection on the first prompt based on the first user's historical question and answer data. The pass-through unit is used to pass through the first prompt to the AI ​​big model and return the first response provided by the AI ​​big model to the first user when the first prompt is not repeated. The repetition processing unit is used to obtain the repetition reason information of the first user in response to the first prompt through the human-computer dialogue interface when the first prompt is repeated, obtain the second reply based on the repetition reason information and the first prompt, and return it to the first user.

[0013] According to a third aspect of this disclosure, an electronic device is provided, comprising: one or more processors and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the aforementioned AI large-scale model multi-turn dialogue deduplication method.

[0014] As can be seen from the above technical solutions, the embodiments of this disclosure obtain duplicate reason information that accurately reflects the user's true intention in repeatedly asking questions through a user feedback mechanism. This duplicate reason information helps the AI ​​big data model accurately understand the user's true intention and return a response that meets the user's true needs. Therefore, users can efficiently obtain the responses they want in human-computer dialogue without submitting the same or similar questions again, effectively preventing the continued generation of duplicate questions and answers. Simultaneously, the embodiments of this disclosure achieve deduplication in multi-turn dialogues of the AI ​​big data model by performing duplicate detection on user prompts. Since user prompts are less detailed and have a simpler word order compared to the AI ​​big data model's responses, system efficiency can be further improved, system overhead reduced, computational resource consumption lowered, and costs saved. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram illustrating the implementation process of human-computer dialogue in related technologies; Figure 2 This is a schematic diagram of the system structure applicable to the embodiments of this disclosure; Figure 3 A flowchart illustrating the AI ​​large-model multi-turn dialogue deduplication method provided in this embodiment of the disclosure; Figure 4 A schematic diagram illustrating the specific implementation process of the AI ​​large-model multi-turn dialogue deduplication method provided in this embodiment of the disclosure; Figure 5 A schematic diagram illustrating the specific implementation process of the AI ​​large-model multi-turn dialogue deduplication method provided in this embodiment of the disclosure; Figure 6 This is an example diagram of a repeat reason prompt box according to an embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of the AI ​​large model multi-turn dialogue deduplication device provided in the embodiments of this disclosure; Figure 8 A schematic structural block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0016] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0017] The terminology used in the embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a,” “the,” and “the” as used in the embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0018] Depending on the context, words such as "if," "when," etc., used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrases "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0019] As mentioned earlier, although large AI models have made significant progress in the field of human-computer dialogue interaction, the inefficient interaction phenomenon of repetitive question-and-answer is becoming increasingly prominent in real-world multi-turn dialogue scenarios.

[0020] Figure 1 This diagram illustrates a multi-turn dialogue process in a large AI model. (See also...) Figure 1 Multi-turn dialogues provide contextual information to large AI models by using the dialogues that occur in multiple rounds as historical dialogue information, thereby helping the large AI models to understand the dialogues and generate responses.

[0021] Currently, in practical interactions between humans and large AI models, we have observed that in real-world multi-turn dialogue scenarios, users repeatedly ask the same or semantically similar questions due to ambiguous intent, lack of contextual information, or model misunderstandings. This forces the large AI model to generate repetitive or similar responses multiple times. In other words, there is a large amount of inefficient or even invalid repetitive question-and-answer processing. This phenomenon has at least three negative impacts: 1) Fragmented dialogue leads to distorted intent parsing: Repetitive questions disrupt the coherence of the dialogue, forcing the model to reconstruct user intent in a fragmented context, significantly increasing the risk of semantic ambiguity and interfering with the AI ​​model's understanding of user intent. 2) Ineffective consumption of token resources and computing power: Repeated question-and-answer sessions consume a large amount of context token quota, especially in long dialogue window scenarios such as 128K context. Redundant text crowds out effective information space, reduces the model's capacity to process key information, increases redundant calculations, and causes meaningless consumption of cloud computing resources. According to tests, repeated question-and-answer sessions can increase computing power overhead by ≥15%.

[0022] 3) Increased interaction costs and reduced user experience: Although large models have the ability to infer user intent through historical dialogues, they rely on complex context backtracking mechanisms. Users need to actively clarify or repeatedly supplement information, which increases operational costs, raises model response latency, prolongs the average number of dialogue rounds, and ultimately leads to a reduced user experience.

[0023] To address the aforementioned issues, relevant technologies primarily solve them by applying detection techniques to determine whether semantically similar question-and-answer pairs exist in multi-turn dialogues. For such pairs, a unique hash value, such as SimHash, is generated for the user input, and identical questions are simply and directly intercepted. However, this simple and direct interception of identical user input filters out many potentially effective, progressive questions, stifles potentially fruitful, deeper interactive dialogues, and ignores the emotional factors of the user in the dialogue. Furthermore, this method fails to fundamentally meet the user's genuine need to repeatedly ask questions. Because this genuine need is not met, users are likely to repeatedly submit the same or similar questions in subsequent rounds of dialogue, further increasing the interaction cost of human-computer dialogue, further degrading the user experience, and further wasting token resources and the computing power of large AI models.

[0024] In view of this, the embodiments of this disclosure provide the following AI large-scale model multi-turn dialogue deduplication method, apparatus and electronic device, which address repetitive question-and-answer in human-computer dialogue by processing based on the user's real needs to help the AI ​​large-scale model understand the user's true intentions and enable the user to receive the response they need. This fundamentally satisfies the user's need to ask repeated questions and avoids the continued generation of similar repetitive question-and-answer in the future. As a result, it fundamentally reduces the ineffective consumption of token resources and AI large-scale model computing power, reduces interaction costs, improves interaction efficiency and enhances user experience.

[0025] Figure 2 A schematic diagram illustrating the architecture of an exemplary application scenario according to embodiments of this disclosure is shown. See also... Figure 1 The exemplary application scenarios of this disclosure may include: a user device, an AI large model multi-turn dialogue deduplication device, and a server. The server is used to run an AI large model to provide human-computer dialogue services based on the AI ​​large model. The AI ​​large model multi-turn dialogue deduplication device is used to execute the AI ​​large model multi-turn dialogue deduplication method described in this disclosure. The user device can interact with the server through the AI ​​large model multi-turn dialogue deduplication device.

[0026] In practical applications, users can log in to the human-computer dialogue interface using their user devices and input prompts in the interface. The user device then sends the prompts to the AI ​​large model multi-turn dialogue deduplication device. The AI ​​large model multi-turn dialogue deduplication device performs the following AI large model multi-turn dialogue deduplication on the prompts, obtains the AI ​​large model's response from the server, and returns it to the user device. The user device then displays the dialogue in the human-computer dialogue interface, thus completing the human-computer dialogue.

[0027] User devices can be of various types, including but not limited to computers, servers, mobile devices, portable electronic devices, and clients. Servers can be, but are not limited to, a single server or a server cluster. The AI ​​large-scale model multi-turn dialogue deduplication device can be implemented as an agent program running on the server side or as a standalone electronic device.

[0028] It should be noted that the AI ​​large-scale model involved in the embodiments of this disclosure can be, but is not limited to, any large-scale model capable of realizing human-computer dialogue, and can also be referred to as a human-computer dialogue large-scale model. In practical applications, the AI ​​large-scale model can be implemented as, but is not limited to, a deep learning model. For example, the AI ​​large-scale model can be implemented as, but is not limited to, a neural network based on the Transformer architecture.

[0029] It should be noted that the first user can refer to any user, and the first prompt can be, but is not limited to, the prompt currently entered by the first user. There are no restrictions on the content or type of the first prompt.

[0030] Figure 3A flowchart illustrating the multi-turn dialogue deduplication method for large AI models provided in this disclosure is shown. See also... Figure 3 The deduplication method for multi-turn dialogue in this large AI model can include the following steps: Step 301: When the first prompt submitted by the first user is detected, the first prompt is checked for duplication based on the first user's historical question and answer data to determine whether the first prompt is duplicated; Step 302: When the first prompt is not repeated, the first prompt is passed through to the AI ​​big model and the first response provided by the AI ​​big model for the first prompt is returned to the first user; Step 303: When the first prompt is repeated, obtain the reason information for the repetition of the first prompt from the first user through the human-computer dialogue interface, obtain the second reply based on the reason information for the repetition and the first prompt, and return it to the first user.

[0031] Considering the numerous instances of repeated prompts and their relevance to users' personalized needs, the accurate reason for repeated questions is best provided or confirmed by the user. Therefore, this embodiment does not employ computationally intensive methods such as semantic analysis to understand user intent. Instead, it obtains repetition reason information that accurately reflects the user's true intent in repeating questions through a user feedback mechanism. This repetition reason information helps the AI ​​model accurately understand the user's true intent and return a response that meets the user's actual needs. Thus, users can efficiently obtain the desired response in human-computer dialogue without submitting the same or similar questions again, effectively preventing the continuation of duplicate question-and-answer sessions.

[0032] This embodiment of the disclosure achieves deduplication in multi-turn dialogue of the AI ​​large model by performing repetition detection on user prompts. Since the user prompts are less content and have a simpler word order compared to the AI ​​large model's responses, it can improve system efficiency, reduce system overhead, reduce computing resource consumption, and save costs.

[0033] Figure 4 This diagram illustrates the interactive processing of the AI ​​large-model multi-turn dialogue deduplication method according to an embodiment of the present disclosure. Figure 5 A schematic diagram illustrating the specific implementation process of the AI ​​large-model multi-turn dialogue deduplication method according to an embodiment of this disclosure is shown.

[0034] See Figure 4 and Figure 5Before step 301, the method of this embodiment may further include: step 300, collecting historical dialogue information of the first user and performing structured processing to generate historical question-and-answer data of the first user, and setting an index for each question-and-answer pair in the historical question-and-answer data, the index being used to query information of question-and-answer pairs in the historical question-and-answer data. Thus, structured processing can be performed on saved multi-turn dialogues to establish question-and-answer pairs and their indexes, assigning sequence numbers and identifiers to each round of questioning and answering, and establishing an index for subsequent processing, facilitating rapid location and efficient querying of question-and-answer pairs in historical question-and-answer data.

[0035] Each question-and-answer pair in the historical question-and-answer data can include a prompt message previously submitted by the first user and a response message returned by the AI ​​model in response to that prompt message. In addition to prompt messages and response messages, question-and-answer pairs may also include, but are not limited to, question-and-answer related conversation information such as conversation round, conversation time, conversation duration, and conversation ID.

[0036] In practical applications, the first user's historical question-and-answer data can be updated in real time based on the user's human-computer interaction. Specifically, to reduce information redundancy in historical question-and-answer data, question-and-answer pairs without repetition can be directly added to the first user's historical question-and-answer data and indexed. For repetitive question-and-answer pairs, some pairs are selectively added to the first user's historical question-and-answer data. For example, for repetitive question-and-answer pairs where the AI ​​model has new responses, they can be added to the first user's historical question-and-answer data and indexed; for cases where there are no new responses from the AI ​​model but the user is responded to using historical question-and-answer data, it is not necessary to add them to the first user's historical question-and-answer data.

[0037] Furthermore, a duplicate flag can be set for question-and-answer pairs in historical question-and-answer data. This flag indicates that the question-and-answer pair is a duplicate. In addition, the duplicate flag can also indicate the number of times the question-and-answer pair is repeated, the reason for the repetition, and other repetition-related characteristics. Specifically, when a question-and-answer pair is found to be duplicated with the user's current prompt (i.e., the prompt information in this question-and-answer pair is the same as the prompt information in other question-and-answer pairs), a duplicate flag can be added or updated for that pair. Setting duplicate flags for question-and-answer pairs facilitates a more accurate understanding of the user's intent.

[0038] In practical applications, the structured processing of the first user's historical dialogue information can be achieved through methods such as vector semantic indexing. This disclosure does not impose any limitations on this aspect.

[0039] In step 301, user prompts can be monitored in real time. Once a prompt is detected, a duplicate detection is performed on the prompt. Thus, the prompt monitoring event can be used as the trigger condition for prompt duplicate detection, which can trigger deduplication in multi-turn dialogues in a timely and efficient manner.

[0040] In step 301, when a user submits a first prompt, the first prompt can be compared one by one with the prompts in each question-and-answer pair in the historical question-and-answer information to determine whether the first prompt is duplicated. By performing duplicate detection on the first prompt, it is possible to efficiently and quickly determine whether there are question-and-answer pairs in the historical conversation that are duplicated with the user's current prompt.

[0041] Specifically, "duplication of the first prompt" means that one or more question-answer pairs in the first user's historical question-answer data have prompts that are similar to or identical to the content of the first prompt. If the prompts in all question-answer pairs in the first user's historical question-answer data are different from and dissimilar to the content of the first prompt, then the first prompt is not considered duplicated.

[0042] In step 301, the repeatability test of the first prompt can be performed in any applicable manner.

[0043] In some examples, the duplication detection in step 301 may include: determining whether the prompt information of the first prompt is duplicated by calculating the semantic similarity between the prompt information of each question-answer pair in the first prompt and the prompt information of each question-answer pair in the historical question-answer data. Specifically, the semantic similarity between the prompt information of the first prompt and the prompt information of each question-answer pair in the historical question-answer data is calculated. If the semantic similarity between the prompt information of a question-answer pair in the historical question-answer data and the first prompt exceeds a preset similarity threshold, it is determined that the prompt information contained in the first prompt and the historical question-answer data are duplicated; if the semantic similarity between the prompt information of all question-answer pairs in the historical question-answer data and the first prompt does not exceed the aforementioned preset similarity threshold, it is determined that the prompt information contained in the first prompt and the historical question-answer data are not duplicated. Using semantic analysis to achieve duplicate detection of user prompts can strike a balance between the accuracy of duplicate detection and the consumption of computational resources.

[0044] In other examples, the first prompt's repetition detection can also be achieved through methods such as lexical feature similarity. This disclosure does not limit the specific implementation of repetition detection.

[0045] In step 301, the similarity of the user prompts is used to determine whether the current prompt will repeat the historical questions and answers. Since the semantics of the prompts themselves are much simpler than the replies, the similarity calculation is greatly simplified, which can further reduce the consumption of computing resources and improve efficiency.

[0046] See Figure 4 and Figure 5 In step 302, the first prompt is not a repetitive question, and the subsequent processing can be carried out in accordance with normal human-computer dialogue.

[0047] Statistical analysis of a large number of human-computer dialogues revealed the following main reasons for users repeatedly asking questions: 1) Users do not reject the AI ​​model's response to the same question from the previous dialogue, but because they cannot retain or remember all historical dialogue information, they find it difficult to locate the same question in previous dialogues and are unwilling to scroll through historical dialogue information; 2) The AI ​​model misunderstands the user's intent, which is particularly prominent when the user's question is somewhat ambiguous; 3) Users accept some of the AI ​​model's responses, but the AI ​​model's response is too generalized or does not accurately address the user's pain point; 4) Users accept some of the AI ​​model's responses, but the AI ​​model's response is too professional and exceeds the user's understanding; 5) Users accept some of the AI ​​model's responses, but users may repeatedly confirm details due to concerns about errors to alleviate uncertainty. Therefore, repeated questioning can be divided into two types: 1) The AI ​​model's previous response fully meets the user's needs, but the user needs to find the corresponding response again; 2) The AI ​​model's previous response cannot fully meet the user's actual needs. Therefore, in step 303, it can be determined how to return a reply to the user based on whether the reason for the first user's repeated question indicated by the repeated reason information is related to the content of the previous reply from the AI ​​big model.

[0048] The above analysis shows that the reasons users repeatedly ask for are crucial for large models to understand user intent. However, requiring users to input repetitive reasons every time is inefficient and tedious, resulting in a poor user experience. Therefore, see [link to relevant section]. Figure 4 and Figure 5 Step 303, obtaining the repetition reason information from the first user's feedback regarding the first prompt through the human-computer dialogue interface, may include: popping up a repetition reason prompt box on the human-computer dialogue interface, the repetition reason prompt box containing multiple repetition reason options; and obtaining the repetition reason information generated by the first user's operation on the repetition reason options in the repetition reason prompt box. Thus, a very lightweight feedback mechanism for the reasons for repeated user inquiries can be used to obtain the repetition reason information from the user's feedback, minimizing user intervention.

[0049] Figure 6 The example image shows a prompt box indicating the reason for repetition. Specifically, if a repetitive question is detected, the system will notify the user that a repetitive question has occurred; that is, a pop-up message will appear on the human-computer interaction interface immediately after confirming that the question is repeated. Figure 6 The message box indicating the reason for the duplicate is shown. See also... Figure 6The "Repeated Reasons" prompt box includes a prompt for the user to provide feedback on the reasons for repeating the question, such as "We noticed you mentioned the above question multiple times. The reasons you mentioned the above question are: {Multiple selection boxes: The assistant's previous response was irrelevant to my question; The assistant's previous response partially answered my concerns, but was too general and requires further investigation; The assistant's previous response partially answered my concerns, but was not clear enough; The assistant's previous response answered my concerns, but I would like to confirm further; I did not remember the assistant's previous response; Other [input box]}". The input box corresponding to the "Other" option allows the user to enter user-defined reasons for repeating the question.

[0050] Therefore, a convenient and clear communication and feedback mechanism for user intent can be achieved by using a repeat reason prompt box.

[0051] It should be noted that the options in the duplicate reason prompt box are not limited to the examples above and can be flexibly modified according to actual needs. In specific applications, the options in the duplicate reason prompt box can be modified, supplemented, or customized according to actual needs, thereby supplementing and adjusting more reasons for users repeatedly asking questions.

[0052] Furthermore, while providing the first user with a prompt box indicating the reason for the duplicate prompt in the human-computer dialogue interface, a list of duplicate questions can also be generated using the prompt information that is duplicated with the first prompt from historical question-and-answer data. This list of duplicate questions can then be provided to the first user through the human-computer dialogue interface, allowing the first user to easily understand the situation of their duplicate questions. Specifically, the list of duplicate questions may include information such as the question-and-answer pair index corresponding to the prompt information that is duplicated with the first prompt.

[0053] Furthermore, the first user can select the question-answer pair index that is closest to the first prompt from the list of repeated questions. The AI ​​big data model multi-turn dialogue deduplication device can respond to the first user's selection operation on the list of repeated questions, and retrieve the corresponding reply information from the first user's historical question-answer data based on the question-answer pair index and return it to the first user. In this way, users can easily query their own historical conversations, further improving interaction efficiency and user experience.

[0054] See Figure 4 and Figure 5 Step 303 may include: Step 3031, if the duplicate reason information indicates that the content previously replied by the AI ​​big model can meet the user's needs, for example, if the reason for the first user's repeated question is unrelated to the content previously replied by the AI ​​big model, it can be considered that the previous reply of the AI ​​big model can meet the user's needs, then the reply information in the question-and-answer pair that is repeated by the first prompt in the historical question-and-answer data will be returned to the first user as the second reply.

[0055] In some examples, the repeated reason information indicating that the content previously replied by the AI ​​model can meet the user's needs may include, but is not limited to, one or more of the following: 1) the user did not remember the reply content; 2) the user did not find the historical reply.

[0056] If there are multiple question-and-answer pairs in the historical question-and-answer data where the prompt information is the same as the first information, the response information from the most recent question-and-answer pair can be selected as the second response, or a response information from a question-and-answer pair can be randomly selected as the second response. Alternatively, the response information from multiple question-and-answer pairs can be compared, and a response information with relatively complete content can be selected as the second response. Of course, other methods can also be used, and this disclosure does not limit these methods.

[0057] Therefore, when a user repeatedly asks a question because the AI ​​model's previous response could have fully met the user's needs, such as "I didn't remember the assistant's previous reply," but the user needs to find the corresponding reply again, the current repetitive question and answer can be blocked. That is, the prompt is not sent to the AI ​​model, but instead, the "previous" response to the same prompt is extracted from the historical information of multiple rounds of dialogue (i.e., the user's historical question and answer data) and returned to the user. This not only meets the user's real needs, but also responds to the user quickly, improves interaction efficiency, enhances user experience, reduces the ineffective overhead of the AI ​​model, saves token resources, and avoids these repetitive questions and answers interfering with the AI ​​model's understanding of the user's intent.

[0058] See Figure 4 and Figure 5 Step 303 may further include: Step 3032, if the duplicate reason information indicates that the content previously replied by the AI ​​big model does not meet the user's needs, for example, if the reason for the first user's repeated question is related to the content previously replied by the AI ​​big model, it can be considered that the content previously replied by the AI ​​big model does not meet the user's needs, then a second prompt can be generated based on the first prompt and the duplicate reason information, the reply corresponding to the second prompt can be obtained from the AI ​​big model, and the reply corresponding to the second prompt can be returned to the first user as the second reply.

[0059] In some examples, the repeated reason information indicating that the AI ​​model's previous response did not meet the user's needs may include, but is not limited to, one of the following: 1) The response is not related to the first user's prompt; 2) The response is related to the first user's prompt but the user finds it insufficiently in-depth; 3) The response is related to the first user's prompt but the user finds it not easily understood; 4) The response is related to the first user's prompt but the user still needs further confirmation; 5) User-defined reason.

[0060] In some examples, a second prompt can be generated based on the reason for repetition and the first prompt, and sent to the AI ​​model with the following content: "Because of {the reason selected by the user}, I am asking you {the first prompt} again, and I need you to answer again. You need to supplement and modify your answer according to the reason for my repetition," to help the AI ​​model accurately understand the user's true intention.

[0061] Therefore, when the previous responses from the AI ​​model cannot fully meet the user's actual needs, new prompts can be modified or generated based on the user's feedback. These new prompts, along with the prompts submitted by the user, can be submitted to the AI ​​model to help it better understand why the user is asking the same question repeatedly. This will guide the AI ​​model to provide progressive or supplementary answers and generate responses that match the user's intent, thereby effectively avoiding repetitive question-and-answer sessions.

[0062] Further, see Figure 4 and Figure 5 The second prompt may further include: a duplicate question-and-answer pair, which can be the reply information from a question-and-answer pair in the first user's historical question-and-answer data that duplicates the prompt information of the first prompt, or all the information of the question-and-answer pair. Specifically, if there are multiple question-and-answer pairs in the first user's historical question-and-answer data that duplicate the prompt information of the first prompt, the reply information from the most recent question-and-answer pair or that question-and-answer pair can be selected as the duplicate question-and-answer pair and attached to the second prompt. Of course, a question-and-answer pair can also be randomly selected or selected by comparison as the duplicate question-and-answer pair and attached to the second prompt; this embodiment of the present disclosure does not limit this.

[0063] In step 3032, if it is necessary to attach duplicate question-and-answer pairs in the second prompt, the information of the corresponding question-and-answer pairs can be quickly located and read from the first user's historical question-and-answer data through the index.

[0064] The following provides an example of how to handle different reasons for repetition.

[0065] If the user's feedback regarding the reason for repetition is: "The assistant's previous reply was irrelevant to my question," the AI ​​big data model's multi-turn dialogue deduplication device can generate a second prompt based on this reason for repetition and the corresponding first prompt: "The user will ask the question again: {First Prompt}. The reason the user is asking this question again is that they believe the big data model assistant's previous reply was irrelevant to the answer they want. The big data model assistant may have misunderstood the user's intention. The big data model assistant needs to analyze and answer the user's question from a different perspective. Attached is the big data model assistant's previous reply {Reply to the same question from the previous time}." This prompt is then sent to the AI ​​big data model. The AI ​​big data model can accurately understand the user's true intention based on this second prompt and generate a reply from a new perspective to meet the user's real needs.

[0066] If the user's feedback regarding the reason for the duplicate message is: "The assistant's initial response partially answered my concerns, but it was too general and requires further exploration," the AI ​​large-scale model's multi-turn dialogue deduplication device will generate a second prompt as follows: "The user will ask the question again: {First Prompt}. The user is asking this question again because they believe the large-scale model assistant's previous response answered part of their question. However, the user wants a more in-depth or professional answer. The large-scale model assistant should supplement the content of the same question, paying attention to increasing the depth and providing a more professional answer. Also, attach the large-scale model assistant's previous response {Reply to the same question from the previous session}."

[0067] If the user's feedback regarding the reason for duplication is: "The assistant's previous response partially answered my concerns, but it wasn't clear enough," the AI ​​large-scale model's multi-turn dialogue deduplication device can generate a second prompt with the following content: "The user will ask the question again: {Question}. The reason the user is asking this question again is that they believe the large-scale model assistant's previous response answered part of their question. However, the large-scale model assistant's answer may have been too technical, and the user may not have easily understood it. The large-scale model assistant should try to answer from a more frontline perspective, making it easier for non-professionals to understand. The large-scale model assistant's previous response {Reply to the same question from the previous session} is also attached."

[0068] If the user's feedback regarding the reason for the duplicate message is: "The assistant's previous response addressed my concerns, but I would like to confirm further," the AI ​​big data model's multi-turn dialogue deduplication device can generate a second prompt as follows to help the AI ​​big data model accurately understand the user's true intention: "The user will ask the question again: {question}. The reason the user is asking this question again is that the user is very cautious about this question and suspects that the big data model assistant's answer is not rigorous or careful enough. The user may be inclined to avoid the big data model's answer to this question being misleading. The big data model assistant should provide an answer that you are certain of. If you are not very confident in answering this question, please inform the user that you are not entirely sure of your answer. Attach the big data model assistant's previous response {response to the same question last time}."

[0069] If the user's feedback regarding the reason for the repetition is "Other," the AI ​​big data model's multi-turn dialogue deduplication device generates a second prompt as follows to help the AI ​​big data model accurately understand the user's true intention: "The user will ask the question again: {First prompt}. The reason the user is asking this question again is {the content the user entered in the other options input box}. The big data model assistant needs to understand and answer based on the user's intention in asking this question again, avoiding duplicate answers. Also included is the big data model assistant's previous response {The response to the same question last time}."

[0070] Furthermore, to avoid the second prompt returned by the AI ​​model due to misunderstanding and thus duplicate its previous reply, failing to reflect the user's true intent, in step 3032, a duplicate check can be performed on the reply before returning the new reply from the AI ​​model to the user, so as to avoid returning a reply with duplicate content to the user.

[0071] Specifically, see Figure 4 and Figure 5 Step 3032 may also include the following steps 3032a to 3032c: Step 3032a: Before returning the reply corresponding to the second prompt as the second reply to the first user, a duplicate check is performed on the reply corresponding to the second prompt based on historical Q&A data to confirm whether the reply corresponding to the second prompt is duplicated; Step 3032b: If the reply corresponding to the second prompt is duplicated, the reply is not duplicated. Then, the reply corresponding to the second prompt is returned to the first user as the second reply.

[0072] Step 3032c: When the reply corresponding to the second prompt is repeated, a third prompt indicating that the reply corresponding to the second prompt is repeated is generated. The question-and-answer pair with the reply information that is repeated with the reply corresponding to the second prompt is queried from the historical question-and-answer data. The third prompt and the queried question-and-answer pair are sent to the AI ​​big model to obtain the AI ​​big model's reply to the third prompt. The reply to the third prompt is returned to the first user as the second reply.

[0073] In step 3032a, "repeated response to the second prompt" means that the response information of one or more question-answer pairs in the historical question-answer data is the same as or similar to the response corresponding to the second prompt, that is, the semantic similarity or part-of-speech similarity is higher than the corresponding preset similarity threshold. "Non-repeated response to the second prompt" means that the response information of all question-answer pairs in the historical question-answer data is different from or dissimilar to the response corresponding to the second prompt, that is, the semantic similarity or part-of-speech similarity does not exceed the preset similarity threshold. Here, the preset similarity threshold for the response can be set to a different value than the preset similarity threshold for the first prompt. In specific applications, the implementation method for response duplication detection is the same as that for the duplication detection of the first prompt, and will not be repeated here.

[0074] In step 3032c, the third prompt can be a reminder word indicating that the reply is repeated. The content of the reminder word can be pre-configured and can be set to fixed content to remind the AI ​​model that this reply is repeated from the previous reply.

[0075] Therefore, by performing duplicate detection on the new responses returned by the AI ​​model, and by determining whether the AI ​​model's responses are duplicates, the responses from the AI ​​model can be effectively selected and integrated, avoiding users from continuing to ask repeated questions because the content of the responses they receive still does not meet their actual needs.

[0076] As can be seen from the above, the AI ​​large model multi-turn dialogue deduplication device in this embodiment does not perform too much specific monitoring on the responses of the large model. Only when the AI ​​large model multi-turn dialogue deduplication device detects that the current prompt of the user is repeated will it start the repetition detection of the response generated by the AI ​​large model. This can avoid increasing system overhead and further reduce the corresponding system latency.

[0077] See Figure 4 and Figure 5 The method described in this embodiment may further include: step 304, when the first prompt is repeated, if the reason information for the repetition is invalid and / or the waiting time exceeds a predetermined duration and the reason information for the repetition is not received, then a fourth prompt containing the first prompt and the repeated question-and-answer pair is generated according to a predetermined configuration, a response to the fourth prompt is obtained from the AI ​​big model, and the response to the fourth prompt is returned to the first user as a second response.

[0078] See Figure 4 and Figure 5 The repeated question-and-answer pairs in the fourth prompt can be the reply information from the question-and-answer pairs that are repeated with the first prompt in the first user's historical question-and-answer data, or all the information of the question-and-answer pairs. Here, the information of the corresponding question-and-answer pairs can be quickly located and read in the first user's historical question-and-answer data through indexing.

[0079] Specifically, if there are multiple question-and-answer pairs in the first user's historical question-and-answer data that repeat the prompt information of the first prompt, the reply information from the most recent question-and-answer pair or that question-and-answer pair can be selected as the repeating question-and-answer pair and attached to the second prompt. Of course, a question-and-answer pair can also be randomly selected or selected by comparison as the repeating question-and-answer pair and attached to the second prompt. This disclosure does not limit this aspect.

[0080] In step 304, before returning the reply to the fourth prompt as the second reply to the first user, a duplicate check can be performed on the reply to the fourth prompt. The same processing as for the second prompt can be executed. If it is determined that the reply to the fourth prompt is not duplicated, it can be returned to the first user as the second reply. If it is determined that the reply to the fourth prompt is duplicated, the same processing as for the second prompt can be executed, i.e., a third prompt is generated and the reply to the third prompt is obtained, and the reply to the third prompt is returned to the first user as the second reply.

[0081] Invalid duplicate reason information includes, but is not limited to, empty duplicate reason information or non-compliant content. The waiting time can start from when the AI ​​big data model multi-turn dialogue deduplication device provides the user with a duplicate reason prompt box, and the preset duration can be flexibly set or changed according to the actual application scenario.

[0082] For example, when the user feedback is empty, the AI ​​big data model's multi-turn dialogue deduplication device can generate a fourth prompt as follows to help the AI ​​big data model accurately understand the user's true intention: "The user will ask the question again: {First prompt}. The user did not directly give a reason for asking the question again. The big data model assistant needs to carefully understand the user's intention based on the user's historical question-and-answer information and provide an answer. Attached is {multi-turn dialogue history, with particular attention to previous repeated questions and answers {responses to the same question from the previous time}}." In this embodiment of the disclosure, "the reply to the same question" refers to the reply information in the previous round of question and answer where the prompt information is the same as the first prompt.

[0083] Furthermore, to avoid making invalid or erroneous judgments on the response content, the method of this disclosure embodiment may further include: submitting two question-and-answer pairs that are duplicated to other models that are different from the large AI model for adjudication.

[0084] After performing duplicate detection on the prompts, and combining this with duplicate response detection under the premise of prompt repetition, we can not only identify the similarity of AI large-scale model responses under the premise of similar questions, ensuring that duplicate question-and-answer pairs cover both the question and the answer, but also significantly reduce the computational cost of the entire duplicate question-and-answer identification algorithm, improve task completion speed, and increase overall efficiency. Furthermore, this method not only deduplicates the current question-and-answer session, but more importantly, it prevents the generation of subsequent identical duplicate questions and answers, thus fundamentally solving the problems of ineffective token resource and computing power consumption, increased interaction costs, and poor user experience caused by duplicate question-and-answer pairs.

[0085] The method disclosed in this embodiment is a highly effective solution for AI large-scale models to understand the true intent of users' repetitive questions. It is implemented through a simple and fast large-scale model-user interface interaction mechanism and concise prompt word engineering, avoiding the consumption of expensive computing power of AI large-scale models.

[0086] Figure 7 A schematic diagram of the composition structure of the AI ​​large-model multi-turn dialogue deduplication device provided in this embodiment of the disclosure is shown. See also Figure 7 The AI ​​large model multi-turn dialogue deduplication device 700 in this embodiment may include: The prompt detection unit 701 is used to detect whether the first prompt submitted by the first user is duplicated by performing a duplicate detection on the first prompt based on the first user's historical question and answer data. The pass-through unit 702 is used to pass through the first prompt to the AI ​​big model and return the first response provided by the AI ​​big model to the first user when the first prompt is not repeated. The repetition processing unit 703 is used to obtain the repetition reason information of the first user in response to the first prompt through the human-computer dialogue interface when the first prompt is repeated, obtain the second reply based on the repetition reason information and the first prompt, and return it to the first user.

[0087] Furthermore, the duplicate processing unit 702 can be specifically used to: if the duplicate reason information indicates that the content previously replied by the AI ​​big model meets the user's needs, then the reply information in the question-and-answer pair where the prompt information in the historical question-and-answer data is duplicated with the first prompt is used as the second reply and returned to the first user; if the duplicate reason information indicates that the content previously replied by the AI ​​big model does not meet the user's needs, then a second prompt is generated based on the first prompt and the duplicate reason information, the reply corresponding to the second prompt is obtained from the AI ​​big model, and the reply corresponding to the second prompt is used as the second reply and returned to the first user.

[0088] Furthermore, the AI ​​large model multi-turn dialogue deduplication device 700 may also include: a reply detection unit 704, used to perform a duplication detection on the reply corresponding to the second prompt based on historical question and answer data to determine whether the reply corresponding to the second prompt is a duplication before the duplication processing unit returns the reply corresponding to the second prompt as the second reply to the first user; the duplication processing unit 702 may specifically be used to: return the reply corresponding to the second prompt as the second reply to the first user when the reply corresponding to the second prompt is not a duplication.

[0089] Furthermore, the duplicate processing unit 702 can also be used to: generate a third prompt when the reply corresponding to the second prompt is repeated, the third prompt being used to indicate that the reply corresponding to the second prompt is repeated, and query the question-and-answer pair from the historical question-and-answer data where the reply information is repeated with the reply corresponding to the second prompt, send the third prompt and the queried question-and-answer pair to the AI ​​big model to obtain the AI ​​big model's reply to the third prompt, and return the reply to the third prompt as the second reply to the first user.

[0090] Furthermore, the duplicate processing unit 702 can also be used to: when the first prompt is repeated, if the duplicate reason information is invalid and / or the waiting time exceeds the predetermined time and the duplicate reason information is not received, generate a fourth prompt containing the first prompt and the duplicate question and answer pair according to the predetermined configuration, obtain the response to the fourth prompt from the AI ​​big model, and return the response to the fourth prompt as the second response to the first user.

[0091] Furthermore, the duplicate processing unit 702 can be specifically used to obtain duplicate reason information in the following ways: providing a duplicate reason prompt box on the human-computer dialogue interface, the duplicate reason prompt box containing multiple duplicate reason options; and obtaining duplicate reason information generated by the first user's operation on the duplicate reason options in the duplicate reason prompt box.

[0092] Furthermore, the prompt detection unit 701 can be used to perform duplicate detection on the first prompt in the following way: by calculating the semantic similarity between the first prompt and the prompt information of each question-answer pair in the historical question-answer data, it can be determined whether the prompt information of the first prompt is duplicated with the prompt information of the question-answer pair in the historical question-answer data.

[0093] Furthermore, the AI ​​large model multi-turn dialogue deduplication device 700 may also include: a conversation information processing unit 705, used to collect the historical dialogue information of the first user and perform structured processing to generate the historical question-and-answer data of the first user and set an index for each question-and-answer pair in the historical question-and-answer data, the index being used to query the information of the question-and-answer pair in the historical question-and-answer data.

[0094] In practical applications, the AI ​​large-scale model multi-turn dialogue deduplication device 700 can be implemented through software, hardware, or a combination of both. For example, the AI ​​large-scale model multi-turn dialogue deduplication device 700 can be implemented as software running in the electronic device 800 described below.

[0095] In addition, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program thereon, the program including instructions that, when executed by one or more processors of a computing device, execute the steps of the aforementioned AI large-scale model multi-turn dialogue deduplication method.

[0096] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. See also... Figure 8 The electronic device 800 may include one or more processors 801, and a memory 802 storing one or more programs, which are executed by the one or more processors 801 to implement the method flow and / or program units corresponding to each unit in the apparatus shown in the above embodiments of this disclosure.

[0097] The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. Processor 801 can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a user interface on an external input / output device (such as a display device coupled to an interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired.

[0098] Processor 801 may include one or more single-core or multi-core processors. Processor 801 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.).

[0099] Memory 802 is the computer-readable storage medium provided in this disclosure, which can be used to store non-transitory software programs, non-transitory computer-executable programs, and units, such as those in the embodiments of this disclosure. Figure 3 The program instructions / units corresponding to the multi-turn dialogue deduplication method for the large AI model shown are as follows. The processor 801 executes non-transient software programs, instructions, and units stored in the memory 802, thereby performing operations such as those described in the above method embodiment. Figure 3 The program, instructions, and units corresponding to the multi-turn dialogue deduplication method of the AI ​​large model shown are illustrated.

[0100] The electronic device 800 may further include an input device 803 and an output device 804. The processor 801, memory 802, input device 803, and output device 804 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0101] Input device 803 can receive input digital or character information, and generate signal inputs related to user settings and function control, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device 804 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0102] The aforementioned programs (also known as software, software applications, or code) include the machine instructions of a programmable processor and can be implemented using object-oriented programming languages, assembly language, or machine language.

[0103] With the development of time and technology, the meaning of "medium" has become increasingly broad. The dissemination of computer programs is no longer limited to tangible media; they can also be downloaded directly from the network. Any combination of one or more computer-readable storage media can be used. Computer-readable storage media can be, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0104] The technical solutions provided in this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this disclosure. Furthermore, those skilled in the art will recognize that, based on the ideas of this disclosure, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

[0105] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications or equivalent substitutions made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for deduplication in multi-turn dialogue of a large AI model, characterized in that, The method includes: When the first prompt submitted by the first user is detected, a duplicate detection is performed on the first prompt based on the first user's historical question and answer data to determine whether the first prompt is duplicated; When the first prompt is not repeated, the first prompt is passed through to the AI ​​big model and the first response provided by the AI ​​big model for the first prompt is returned to the first user; When the first prompt is repeated, the system obtains the reason information for the repetition of the first prompt from the first user through the human-computer dialogue interface, obtains the second reply based on the reason information for the repetition and the first prompt, and returns it to the first user. The step of obtaining a second reply based on the duplicate reason information and the first prompt and returning it to the first user includes: if the duplicate reason information indicates that the content of the previous reply from the AI ​​big model meets the user's needs, then the reply information in the question-and-answer pair where the prompt information in the historical question-and-answer data is duplicated with the first prompt is used as the second reply and returned to the first user; if the duplicate reason information indicates that the content of the previous reply from the AI ​​big model does not meet the user's needs, then a second prompt is generated based on the first prompt and the duplicate reason information, the reply corresponding to the second prompt is obtained from the AI ​​big model, and the reply corresponding to the second prompt is used as the second reply and returned to the first user.

2. The method according to claim 1, characterized in that, Before returning the reply corresponding to the second prompt to the first user as the second reply, the method further includes: Based on the historical question-and-answer data, a duplicate detection is performed on the response corresponding to the second prompt to determine whether the response corresponding to the second prompt is duplicated; The step of returning the reply corresponding to the second prompt as the second reply to the first user includes: when the reply corresponding to the second prompt is not repeated, then returning the reply corresponding to the second prompt as the second reply to the first user.

3. The method according to claim 2, characterized in that, The method further includes: When the reply corresponding to the second prompt is repeated, a third prompt is generated; the third prompt is used to indicate that the reply corresponding to the second prompt is repeated. The system retrieves question-and-answer pairs from the historical question-and-answer data that contain duplicate replies to the second prompt, and sends the third prompt and the retrieved question-and-answer pairs to the AI ​​big model to obtain the AI ​​big model's response to the third prompt. The response to the third prompt will be returned to the first user as the second response.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: If the duplicate reason information is invalid and / or the waiting time exceeds a predetermined period without receiving the duplicate reason information, a fourth prompt containing the first prompt and the duplicate question-and-answer pair is generated according to a predetermined configuration. A response to the fourth prompt is obtained from the AI ​​big model, and the response to the fourth prompt is returned to the first user as the second response.

5. The method according to claim 1, characterized in that, The human-computer dialogue interface obtains the reason information for the recurrence of the first prompt from the first user, including: A duplicate reason prompt box is provided on the human-computer dialogue interface, and the duplicate reason prompt box contains multiple duplicate reason options; Obtain duplicate reason information generated by the first user's operation on the duplicate reason option in the duplicate reason prompt box.

6. The method according to claim 1, characterized in that, The step of performing a duplicate detection on the first prompt based on the first user's historical question-and-answer data to determine whether the first prompt is duplicated includes: determining whether the first prompt is duplicated by calculating the semantic similarity between the first prompt and the prompt information of each question-and-answer pair in the historical question-and-answer data.

7. The method according to claim 1, characterized in that, The method further includes: collecting historical dialogue information of the first user and performing structured processing to generate historical question-and-answer data of the first user, and setting an index for each question-and-answer pair in the historical question-and-answer data, wherein the index is used to query the information of the question-and-answer pair in the historical question-and-answer data.

8. A deduplication device for multi-turn dialogue in large AI models, characterized in that, The AI ​​large-scale model multi-turn dialogue deduplication device includes: The prompt detection unit is used to detect whether the first prompt submitted by the first user is repeated by performing a duplicate detection on the first prompt based on the first user's historical question and answer data. The pass-through unit is used to pass through the first prompt to the AI ​​big model and return the first response provided by the AI ​​big model to the first user when the first prompt is not repeated. The repetition processing unit is used to obtain the repetition reason information of the first user in response to the first prompt through the human-computer dialogue interface when the first prompt is repeated, obtain the second reply according to the repetition reason information and the first prompt, and return it to the first user. Specifically, the duplicate processing unit is used to: if the duplicate reason information indicates that the content previously replied by the AI ​​big model meets the user's needs, then the reply information in the question-and-answer pair where the prompt information in the historical question-and-answer data is duplicated with the first prompt is used as the second reply and returned to the first user; if the duplicate reason information indicates that the content previously replied by the AI ​​big model does not meet the user's needs, then a second prompt is generated based on the first prompt and the duplicate reason information, the reply corresponding to the second prompt is obtained from the AI ​​big model, and the reply corresponding to the second prompt is used as the second reply and returned to the first user.

9. An electronic device, characterized in that, include: A memory for storing one or more processors and programs, the programs comprising instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images