Conversation processing method and device, computer equipment and storage medium

Abstract

The invention relates to a dialogue processing method and device, computer equipment and a storage medium. The method comprises the steps that when it is determined that the total number of first tokens corresponding to a dialogue instruction and historical dialogue content exceeds a first token threshold value, the historical dialogue content is summarized to obtain target dialogue content, and the summarized total number of tokens corresponding to the target dialogue content and the dialogue instruction is smaller than the first token threshold value; therefore, the dialogue content input into the preset artificial intelligence language model is prevented from exceeding the first token threshold value, the dialogue instruction and the target dialogue content are input into the preset artificial intelligence language model, the preset artificial intelligence language model can successfully output the corresponding reply content, earlier dialogue content in the historical dialogue content is not discarded, and the reply content can be successfully output. The user does not need to manually refresh and restart the dialogue application, and the number of tokens input into the preset artificial intelligence language model can be controlled to be smaller than the first token threshold value.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a dialogue processing method, apparatus, computer device, and storage medium. Background Art

[0002] With the development of the Internet, artificial intelligence language models have been developed, which can generate answers based on the patterns and statistical laws seen during the pre-training phase, can also interact according to the context of the chat, truly chat and communicate like humans, and can even complete tasks such as writing emails, video scripts, copywriting, translation, code, and writing papers. Artificial intelligence language models will use historical conversation records to assist in generating natural conversations, and their inputs are all processed in units of tokens (TOKENS). A TOKEN is the basic unit of text or code used by a large model to process and generate language.

[0003] When an artificial intelligence language model processes an input, an input that exceeds the maximum TOKEN limit will be truncated or rejected, which will result in the failure of dialogue generation. The existing processing method is to discard earlier conversation content or prompt the user to refresh the dialogue application, but discarding earlier conversation content has a negative effect on generating the dialogue, and prompting the user to refresh the dialogue application to restart will also reduce the user experience. Summary of the Invention

[0004] This application provides a dialogue processing method, apparatus, computer device, and storage medium to solve the problem that the existing processing method has a negative effect on generating a conversation or affects the user experience when the conversation content exceeds the maximum TOKEN limit.

[0005] In a first aspect, this application provides a dialogue processing method, and the method includes:

[0006] When receiving a dialogue instruction sent by a target account, obtain the historical conversation content corresponding to the target account;

[0007] Determine the total number of the first tokens corresponding to the dialogue instruction and the historical conversation content;

[0008] When the total number of the first tokens is greater than a first token threshold, perform a summarization process on the historical conversation content to obtain target conversation content, where the total number of tokens corresponding to the target conversation content and the dialogue instruction is less than the first token threshold, and the first token threshold is less than or equal to the maximum token limit corresponding to a preset artificial intelligence language model;

[0009] Use the dialogue instruction and the target conversation content as the input content of the preset artificial intelligence language model and input them into the preset artificial intelligence language model;

[0010] Feed the reply content output by the preset artificial intelligence language model according to the conversation instruction and the target conversation content back to the target account.

[0011] In a second aspect, the present application provides a conversation processing device, the device includes:

[0012] An acquisition module, configured to acquire the historical conversation content corresponding to the target account when receiving a conversation instruction sent by the target account;

[0013] A determination module, configured to determine the total number of the first tokens corresponding to the conversation instruction and the historical conversation content;

[0014] A judgment module, configured to perform a summary process on the historical conversation content to obtain a target conversation content when the total number of the first tokens is greater than a first token threshold, wherein the total number of tokens corresponding to the target conversation content and the conversation instruction is less than the first token threshold, and the first token threshold is less than or equal to the maximum token limit corresponding to the preset artificial intelligence language model;

[0015] An input module, configured to input the conversation instruction and the target conversation content as input content of the preset artificial intelligence language model into the preset artificial intelligence language model;

[0016] A feedback module, configured to feed the reply content output by the preset artificial intelligence language model according to the conversation instruction and the target conversation content back to the target account.

[0017] In a third aspect, the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the above-mentioned conversation processing method is implemented.

[0018] In a fourth aspect, the present application further provides a computer storage medium, storing computer executable instructions, where the computer executable instructions are used to execute the above-mentioned conversation processing method.

[0019] The above technical solution provided by the embodiments of the present application has the following advantages compared with the prior art: In the method provided by the embodiments of the present application, when a dialogue instruction sent by a target account is received, the historical dialogue content corresponding to the target account is obtained; the total number of the first tokens corresponding to the dialogue instruction and the historical dialogue content is determined; when the total number of the first tokens is greater than a first token threshold, the historical dialogue content is summarized to obtain target dialogue content, wherein the total number of tokens corresponding to the target dialogue content and the dialogue instruction is less than the first token threshold, and the first token threshold is less than or equal to the maximum token limit corresponding to a preset artificial intelligence language model; the dialogue instruction and the target dialogue content are input into the preset artificial intelligence language model as the input content of the preset artificial intelligence language model; and the reply content output by the preset artificial intelligence language model according to the dialogue instruction and the target dialogue content is fed back to the target account.

[0020] Based on the above method, when it is determined that the total number of the first tokens corresponding to the dialogue instruction and the historical dialogue content exceeds the first token threshold, the historical dialogue content is summarized to obtain the target dialogue content. The total number of tokens corresponding to the summarized target dialogue content and the dialogue instruction is less than the first token threshold, thereby preventing the dialogue content input into the preset artificial intelligence language model from exceeding the first token threshold. By inputting the dialogue instruction and the target dialogue content into the preset artificial intelligence language model, the preset artificial intelligence language model can successfully output the corresponding reply content. Among them, the earlier dialogue content in the historical dialogue content is not discarded, and the user does not need to manually refresh and restart the dialogue application, so that the number of tokens input into the preset artificial intelligence language model can be controlled to be less than the first token threshold, thereby solving the problem that the existing processing method has a negative effect on generating a conversation or affects the user experience when the dialogue content exceeds the maximum TOKEN limit. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention and, together with the specification, are used to explain the principles of the present invention.

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] One or more embodiments are illustrated by way of example in the accompanying drawings, and these exemplary illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, unless otherwise stated, and the drawings in the figures do not constitute a scale limitation.

[0024] Figure 1 An application environment diagram of a dialogue processing method provided by an embodiment of this application;

[0025] Figure 2 A schematic flowchart of a dialogue processing method provided by an embodiment of this application;

[0026] Figure 3 A schematic flowchart of a dialogue processing method provided by an embodiment of this application;

[0027] Figure 4 A schematic flowchart of a dialogue processing method provided by an embodiment of this application;

[0028] Figure 5 A schematic flowchart of a dialogue processing method provided by an embodiment of this application;

[0029] Figure 6 A schematic flowchart of a dialogue processing method provided by an embodiment of this application;

[0030] Figure 7 A schematic flowchart of a dialogue processing method provided by an embodiment of this application;

[0031] Figure 8 A schematic flowchart of a dialogue processing method provided by an embodiment of this application;

[0032] Figure 9 A structural block diagram of a dialogue processing device provided by an embodiment of this application;

[0033] Figure 10 An internal structural diagram of a computer device provided by an embodiment of this application. Detailed implementation manners

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Apparently, the described embodiments are some but not all of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of this application without creative efforts shall fall within the scope of protection of this application.

[0035] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present invention. In addition, the present invention may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0036] Figure 1 It is an application environment diagram of a dialogue processing method in an embodiment. Refer to Figure 1 , this dialogue processing method is applied to a dialogue processing system. The dialogue processing system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network. The terminal 110 may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may specifically be at least one of a mobile phone, a tablet computer, a laptop computer, etc. A dialogue application providing an artificial intelligence dialogue service is installed on the terminal 110, and the user interacts with the artificial intelligence language model in the server 120 by logging in to the dialogue application. An artificial intelligence language model and / or a storage service is installed on the server 120, that is, the storage service may be installed on the same server or different servers as the artificial intelligence language model. In this embodiment, the storage service is installed on the same server as the artificial intelligence language model, and the server 120 may be implemented by an independent server or a server cluster composed of multiple servers.

[0037] The artificial intelligence language model may be a Replicate language model, a Sequence Monkey large model, a DeepSpeed language model, or an LLM (large language model) large model, etc. Since the LLM large model uses deep learning technology and large-scale data sets to understand, summarize, generate, and predict new content, the LLM large model is adopted as the preset artificial intelligence language model in this embodiment.

[0038] In one embodiment, Figure 2 It is a schematic flowchart of a dialogue processing method in an embodiment. Refer to Figure 2 , a dialogue processing method is provided. This embodiment mainly takes the application of this method to the storage service in the above Figure 1 as an example to illustrate. The dialogue processing method specifically includes the following steps:

[0039] Step S210, when receiving a dialogue instruction sent by a target account, obtain the historical dialogue content corresponding to the target account.

[0040] Specifically, the target account is the account used by the user to log in to the conversation application in the user terminal 110. The conversation instruction is used to indicate the question content input by the user into the conversation application, and is used to request to obtain the reply content of the conversation application. The storage service stores the historical conversation content corresponding to different accounts. The storage service not only records the question content input by the account to the preset artificial intelligence language model, but also records the reply content output by the preset artificial intelligence language model. In order to generate a reply content that combines the context, the preset artificial intelligence model needs to combine the historical conversation content in the storage service. Therefore, each time the preset artificial intelligence language model generates conversation content, it will obtain the historical conversation content corresponding to the corresponding account from the storage service.

[0041] Step S220, determine the total number of the first tokens corresponding to the conversation instruction and the historical conversation content.

[0042] Specifically, a token (abbreviated as T) is the unit used by the preset artificial intelligence language model to calculate the usage. Both the conversation instruction and the historical conversation content are composed of characters. The characters specifically include English characters, Chinese characters, numbers, symbols, etc. Different characters correspond to different token numbers. For example, an English word corresponds to one token, and a Chinese character corresponds to two tokens. For example, the conversation instruction "What's the weather like today?" corresponds to 15 tokens, and the conversation instruction "what is the weather today?" corresponds to 6 tokens. In this way, the total number of tokens of the conversation instruction and the historical conversation content is counted to obtain the total number of the first tokens.

[0043] Step S230, when the total number of the first tokens is greater than the first token threshold, summarize the historical conversation content to obtain the target conversation content, where the total number of tokens corresponding to the target conversation content and the conversation instruction is less than the first token threshold, and the first token threshold is less than or equal to the maximum token limit corresponding to the preset artificial intelligence language model.

[0044] Specifically, the first token threshold is less than or equal to the maximum token threshold corresponding to the preset artificial intelligence language model. When the first token threshold is less than the maximum token threshold corresponding to the preset artificial intelligence language model, the first token threshold is used to judge when to summarize the historical conversation content, so as to ensure that the number of tokens corresponding to the subsequent conversation content input to the preset artificial intelligence model must be less than the maximum token threshold corresponding to the preset artificial intelligence language model.

[0045] When the total number of the first tokens is greater than the first token threshold, summarize the historical conversation content corresponding to the target account to obtain the target conversation content. The target conversation content is the compressed result after the purification of the historical conversation content. Therefore, purifying the historical conversation content can not only prevent information loss caused by the abandonment of some content in the historical conversation content, but also reduce the number of tokens input to the preset artificial intelligence language model subsequently.

[0046] Step S240, input the conversation instruction and the target conversation content as the input content of the preset artificial intelligence language model into the preset artificial intelligence language model.

[0047] Specifically, provide the purified target conversation content and the conversation instruction to the preset artificial intelligence language model. The trained preset artificial intelligence language model can analyze and process the target conversation content and the conversation instruction and output a matching reply content.

[0048] Step S250, feedback the reply content output by the preset artificial intelligence language model according to the conversation instruction and the target conversation content to the target account.

[0049] Specifically, since the input content of the preset artificial intelligence language model does not exceed its corresponding token threshold, it will not be truncated or rejected by the preset artificial intelligence model, and the preset artificial intelligence model can successfully generate a conversation.

[0050] Based on the above method, when it is determined that the total number of the first tokens corresponding to the conversation instruction and the historical conversation content exceeds the first token threshold, summarize the historical conversation content to obtain the target conversation content. The total number of tokens corresponding to the summarized target conversation content and the conversation instruction is less than the first token threshold, so as to prevent the conversation content input to the preset artificial intelligence language model from exceeding the first token threshold. Input the conversation instruction and the target conversation content into the preset artificial intelligence language model, and the preset artificial intelligence language model can successfully output the corresponding reply content. Among them, the earlier conversation content in the historical conversation content is not abandoned, and there is no need for the user to manually refresh and restart the conversation application, so that the number of tokens input to the preset artificial intelligence language model can be controlled to be less than the first token threshold, thereby solving the problem that the existing processing method has a negative effect on generating a conversation or affects the user experience when the conversation content exceeds the maximum TOKEN limit.

[0051] In one embodiment, as Figure 3 shown, when the total number of the first tokens is greater than the first token threshold, summarize the historical conversation content to obtain the target conversation content, including:

[0052] Step S231, when the total number of tokens of the first type is greater than the first token threshold, call the target artificial intelligence language model to summarize the historical conversation content to obtain the target conversation content, where the target artificial intelligence language model is the preset artificial intelligence language model or a non-preset artificial intelligence language model.

[0053] Specifically, if the total number of tokens of the first type exceeds the first token threshold, the historical conversation content is summarized by calling the target artificial intelligence language model. The target artificial intelligence language model can be a preset artificial intelligence language model or an artificial intelligence language model other than the preset one. The artificial intelligence language model is used to summarize and refine the historical conversation content, thereby compressing the number of tokens in the historical conversation content.

[0054] In one embodiment, as Figure 4 shown, when the total number of tokens of the first type is greater than the first token threshold, calling the target artificial intelligence language model to summarize the historical conversation content to obtain the target conversation content includes:

[0055] Step S2311, when the total number of tokens of the first type is greater than the first token threshold, generate a conversation summary instruction corresponding to the historical conversation content;

[0056] Step S2312, input the historical conversation content and the conversation summary instruction as the input content of the target artificial intelligence language model into the target artificial intelligence language model;

[0057] Step S2313, use the reply content output by the target artificial intelligence language model according to the historical conversation content and the conversation summary instruction as the target conversation content.

[0058] Specifically, when the total number of tokens of the first type exceeds the first token threshold, a conversation summary instruction (Prompt) is triggered, that is, a conversation content similar to "Please help me summarize the historical conversation content" is imitated and input to the target artificial intelligence language model, and the historical conversation content is provided to the target artificial intelligence language model, so that the target artificial intelligence language model automatically refines the historical conversation content according to the conversation summary instruction. The reply content output by the target artificial intelligence language model for the historical conversation content and the conversation summary instruction is the target conversation content after the historical conversation content is refined. The refined target conversation content has a significantly reduced number of tokens compared to the historical conversation content and also reduces the information loss of the conversation content.

[0059] In one embodiment, as Figure 5As shown, inputting the historical conversation content and the conversation summary instruction as the input content of the target artificial intelligence language model into the target artificial intelligence language model includes:

[0060] Step S23121, when the total number of second tokens corresponding to the historical conversation content and the conversation summary instruction is less than or equal to the second token threshold corresponding to the target artificial intelligence language model, input the historical conversation content and the conversation summary instruction as the input content of the target artificial intelligence language model into the target artificial intelligence language model, where the first token threshold is less than the second token threshold.

[0061] Specifically, to ensure that the target artificial intelligence language model can purify the historical conversation content according to the conversation summary instruction, it is necessary to ensure that the total number of tokens of the historical conversation content and the conversation summary instruction is lower than the second token threshold corresponding to the target artificial intelligence language model. The second token threshold is the maximum token limit of the target artificial intelligence language model. When the target artificial intelligence language model is a preset artificial intelligence language model, the second token threshold is greater than or equal to the first token threshold, that is, the first token threshold is equal to the second token threshold, which means purifying and judging the input content according to the maximum token limit of the preset artificial intelligence language model.

[0062] However, the greater the number of tokens corresponding to the input content of the artificial intelligence language model, the higher the probability of the artificial intelligence language model having hallucinations. That is, when the matching degree between the reply content output by the artificial intelligence language model and the input content is low, it is regarded as having hallucinations. Therefore, to reduce the probability of having hallucinations, the first token threshold is made less than the second token threshold. Before the number of tokens corresponding to the input content of the artificial intelligence language model reaches the second token threshold, the input content can be purified in advance to compress the number of tokens, which can not only ensure that the input content does not exceed the second token threshold, but also minimize the number of tokens of the input content when the number of tokens corresponding to the input content does not exceed the second token threshold, thereby reducing the probability of the artificial intelligence language model having hallucinations.

[0063] Only when the total number of second tokens corresponding to the historical conversation content and the conversation summary instruction is less than or equal to the second token threshold, the historical conversation content and the conversation summary instruction are input as the input content of the target artificial intelligence language model into the target artificial intelligence language model, avoiding the input content being truncated or rejected due to the number of tokens exceeding the total number of second tokens corresponding to the target artificial intelligence language model.

[0064] In one embodiment, as Figure 6 shown, inputting the historical conversation content and the conversation summary instruction as the input content of the target artificial intelligence language model into the target artificial intelligence language model includes:

[0065] Step S231211: When the total number of second tokens corresponding to the historical conversation content and the conversation summary instruction is greater than the second token threshold corresponding to the target artificial intelligence language model, the difference between the second token threshold and the number of tokens corresponding to the conversation summary instruction is used as the remaining number of tokens.

[0066] Step S231212: The historical conversation content is divided into multiple split contents according to the remaining number of tokens.

[0067] Step S231213: The different split contents and the conversation summary instruction are respectively used as the input content of the target artificial intelligence language model and input into the target artificial intelligence language model, where the total number of tokens corresponding to the conversation summary instruction and the split content is less than or equal to the second token threshold.

[0068] Regarding using the reply content output by the target artificial intelligence language model based on the historical conversation content and the conversation summary instruction as the target conversation content, it includes:

[0069] Step S23131: Determine the target conversation content according to the reply content output by the target artificial intelligence language model for the conversation summary instruction and different split contents.

[0070] Specifically, it is also possible that the total number of tokens of the historical conversation content and the conversation summary instruction exceeds the target artificial intelligence language model, that is, the total number of second tokens exceeds the second token threshold, indicating that the number of tokens corresponding to the historical conversation content is relatively large, and the target artificial intelligence language model cannot directly purify the historical conversation content. Therefore, the historical conversation content needs to be split. To ensure that the total number of tokens of the split content and the conversation summary instruction after splitting can be less than or equal to the second token threshold of the target artificial intelligence language model, the remaining number of tokens is obtained by subtracting the number of tokens corresponding to the conversation summary instruction from the second token threshold. The remaining number of tokens is the number of tokens corresponding to the split content. Therefore, the historical conversation content is split according to the remaining number of tokens, and the number of tokens corresponding to each split content is less than or equal to the remaining number of tokens, so as to ensure that the total number of tokens of the split content and the conversation summary instruction is less than or equal to the second token threshold.

[0071] Each split content, together with the dialogue summary instruction, is used as input content and input into the target artificial intelligence language model. That is, each time the dialogue summary instruction and a split content are input into the target artificial intelligence language model. The target artificial intelligence language model respectively performs summary and purification processing on different split contents, so as to complete the summary and purification processing of the complete historical dialogue content. According to the reply content output by the target artificial intelligence language model for the dialogue summary instruction and different split contents, the target dialogue content is integrated. Specifically, the reply content output by the target artificial intelligence language model for the dialogue summary instruction and different split contents can be combined to form the target dialogue content. That is, the reply content corresponding to each split content is combined together to form the target dialogue content. Since the reply content corresponding to each split content is the result of purification processing, combining the purification results corresponding to each split content can be regarded as the purification result of the historical dialogue content, and it can also achieve the effect of compressing the number of tokens corresponding to the historical dialogue content.

[0072] In one embodiment, as Figure 7 shown, determining the target dialogue content according to the reply content output by the target artificial intelligence language model for the dialogue summary instruction and different split contents includes:

[0073] Step S231311: Use the reply content output by the target artificial intelligence language model for the dialogue summary instruction and different split contents as the purified content corresponding to each split content;

[0074] Step S231312: Use the dialogue summary instruction and all the purified content as the input content of the target artificial intelligence language model and input it into the target artificial intelligence language model, where the total number of tokens corresponding to the dialogue summary instruction and all the purified content is less than or equal to the second token threshold;

[0075] Step S231313: Use the reply content output by the target artificial intelligence language model for the dialogue summary instruction and all the purified content as the target dialogue content.

[0076] Specifically, the response content corresponding to each split content is used as the refined content corresponding to each split content. However, when the target artificial intelligence language model summarizes and refines different split contents, it does not consider the correlation between different split contents. Therefore, in order to combine the correlation between different split contents, the refined content corresponding to each split content and the dialogue summary instruction are used as input content and input into the target artificial intelligence language model. The response content output by the target artificial intelligence language model for the dialogue summary instruction and all the refined content is used as the target dialogue content. This refining method not only combines the correlation between different refined contents for further summary and refinement processing, but also can further compress the number of tokens of the target dialogue content finally input into the preset artificial intelligence language model.

[0077] In one embodiment, as Figure 8 shown, after summarizing the historical dialogue content to obtain the target dialogue content when the total number of the first tokens is greater than the first token threshold, the method further includes:

[0078] Step S260, updating the historical dialogue content corresponding to the target account to the target dialogue content.

[0079] Specifically, after refining the historical dialogue content, the historical dialogue content corresponding to the target account is updated to the target dialogue content. Specifically, the historical record corresponding to the target account is modified to the refined target dialogue content through a session key, that is, only the refined dialogue content is saved. This can not only save the storage space of the storage service, but also reduce the number of tokens of the input content when it is provided as input content to the preset artificial intelligence language model in the future. It can not only ensure that the number of tokens corresponding to the input content does not exceed the maximum token limit corresponding to the preset artificial intelligence language model, but also reduce the probability of the preset artificial intelligence language model having hallucinations.

[0080] Figures 2 - 8 It is a schematic flowchart of a dialogue processing method in an embodiment. It should be understood that although Figures 2 - 8 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figures 2 - 8 at least a part of the steps in

[0081] In one embodiment, as Figure 9 shown, a dialogue processing apparatus is provided, including:

[0082] An acquisition module 310, configured to acquire historical dialogue content corresponding to the target account when receiving a dialogue instruction issued by the target account;

[0083] A determination module 320, configured to determine a total number of first tokens corresponding to the dialogue instruction and the historical dialogue content;

[0084] A judgment module 330, configured to summarize the historical dialogue content to obtain target dialogue content when the total number of first tokens is greater than a first token threshold, where a total number of tokens corresponding to the target dialogue content and the dialogue instruction is less than the first token threshold, and the first token threshold is less than or equal to a maximum token limit corresponding to a preset artificial intelligence language model;

[0085] An input module 340, configured to input the dialogue instruction and the target dialogue content as input content of the preset artificial intelligence language model into the preset artificial intelligence language model;

[0086] A feedback module 350, configured to feedback a reply content output by the preset artificial intelligence language model according to the dialogue instruction and the target dialogue content to the target account.

[0087] In one embodiment, the judgment module 330 is further configured to:

[0088] When the total number of first tokens is greater than the first token threshold, call a target artificial intelligence language model to summarize the historical dialogue content to obtain the target dialogue content, where the target artificial intelligence language model is the preset artificial intelligence language model or a non-preset artificial intelligence language model.

[0089] In one embodiment, the judgment module 330 is further configured to:

[0090] Generate a dialogue summary instruction corresponding to the historical dialogue content when the total number of first tokens is greater than the first token threshold;

[0091] Input the historical dialogue content and the dialogue summary instruction as input content of the target artificial intelligence language model into the target artificial intelligence language model;

[0092] Use the reply content output by the target artificial intelligence language model according to the historical dialogue content and the dialogue summary instruction as the target dialogue content.

[0093] In one embodiment, the judgment module 330 is further configured to:

[0094] When the total number of second tokens corresponding to the historical conversation content and the conversation summary instruction is less than or equal to the second token threshold corresponding to the target artificial intelligence language model, the historical conversation content and the conversation summary instruction are input into the target artificial intelligence language model as the input content of the target artificial intelligence language model, wherein the first token threshold is less than the second token threshold.

[0095] In one embodiment, the determination module 330 is further configured to:

[0096] When the total number of second tokens corresponding to the historical conversation content and the conversation summary instruction is greater than the second token threshold corresponding to the target artificial intelligence language model, the difference between the second token threshold and the number of tokens corresponding to the conversation summary instruction is used as the remaining number of tokens;

[0097] The historical conversation content is divided into multiple split contents according to the remaining number of tokens;

[0098] The different split contents and the conversation summary instruction are respectively input into the target artificial intelligence language model as the input content of the target artificial intelligence language model, wherein the total number of tokens corresponding to the conversation summary instruction and the split content is less than or equal to the second token threshold;

[0099] The target conversation content is determined according to the reply content output by the target artificial intelligence language model for the conversation summary instruction and different split contents.

[0100] In one embodiment, the determination module 330 is further configured to:

[0101] The reply content output by the target artificial intelligence language model for the conversation summary instruction and different split contents is used as the purified content corresponding to each split content;

[0102] The conversation summary instruction and all the purified contents are input into the target artificial intelligence language model as the input content of the target artificial intelligence language model, wherein the total number of tokens corresponding to the conversation summary instruction and all the purified contents is less than or equal to the second token threshold;

[0103] The reply content output by the target artificial intelligence language model for the conversation summary instruction and all the purified contents is used as the target conversation content.

[0104] In one embodiment, the device further includes an update module, configured to:

[0105] Update the historical conversation content corresponding to the target account to the target conversation content.

[0106] As Figure 10 shown, an embodiment of the present application provides a computer device, including a processor 711, a communication interface 712, a memory 713, and a communication bus 714. Among them, the processor 711, the communication interface 712, and the memory 713 complete communication with each other through the communication bus 714;

[0107] The memory 713 is used to store a computer program;

[0108] When the processor 711 is used to execute the program stored on the memory 713, it implements the dialogue processing method provided by any one of the foregoing method embodiments.

[0109] Those skilled in the art can understand that Figure 10 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0110] In one embodiment, the dialogue processing device provided by the present application can be implemented in the form of a computer program, and the computer program can run on a computer device as Figure 10 shown. Each program module constituting the dialogue processing device can be stored in the memory of the computer device. For example, Figure 9 the acquisition module 310, the determination module 320, the judgment module 330, the input module 340, and the feedback module 350 shown. The computer program constituted by each program module enables the processor to execute the dialogue processing methods of various embodiments of the present application described in this specification.

[0111] Figure 10 The computer device shown can be connected through as Figure 9The acquisition module 310 in the illustrated dialogue processing device executes to acquire the historical dialogue content corresponding to the target account when receiving a dialogue instruction issued by the target account. The computer device can execute, through the determination module 320, to determine the total number of first tokens corresponding to the dialogue instruction and the historical dialogue content. The computer device can execute, through the judgment module 330, to summarize the historical dialogue content when the total number of the first tokens is greater than a first token threshold to obtain target dialogue content, where the total number of tokens corresponding to the target dialogue content and the dialogue instruction is less than the first token threshold, and the first token threshold is less than or equal to the maximum token limit corresponding to a preset artificial intelligence language model. The computer device can execute, through the input module 340, to input the dialogue instruction and the target dialogue content as input content of the preset artificial intelligence language model into the preset artificial intelligence language model. The computer device can execute, through the feedback module 350, to feedback the reply content output by the preset artificial intelligence language model according to the dialogue instruction and the target dialogue content to the target account.

[0112] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the dialogue processing method provided in any one of the foregoing method embodiments.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the related technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, server 120, or network device, etc.) to execute the dialogue processing method described in each embodiment or some parts of the embodiments.

[0115] It should be understood that the terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. Unless the context clearly dictates otherwise, the singular forms "a", "an", and "the" as used herein may also include the plural forms. The terms "comprising", "including", "containing", and "having" are inclusive and thus specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the particular order described or illustrated, unless an execution order is explicitly stated. It should also be understood that additional or alternative steps may be used.

[0116] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A dialogue processing method, characterized in that, The method includes: When receiving a conversation instruction issued by a target account, obtaining the historical conversation content corresponding to the target account; Determining the total number of first tokens corresponding to the conversation instruction and the historical conversation content; When the total number of first tokens is greater than a first token threshold, performing a summarization process on the historical conversation content to obtain target conversation content, where the total number of tokens corresponding to the target conversation content and the conversation instruction is less than the first token threshold, and the first token threshold is less than or equal to the maximum token limit corresponding to a preset artificial intelligence language model; Taking the conversation instruction and the target conversation content as input content of the preset artificial intelligence language model and inputting them into the preset artificial intelligence language model; Feeding back the reply content output by the preset artificial intelligence language model according to the conversation instruction and the target conversation content to the target account.

2. The dialogue processing method according to claim 1, wherein The performing a summarization process on the historical conversation content to obtain target conversation content when the total number of first tokens is greater than the first token threshold includes: When the total number of first tokens is greater than the first token threshold, invoking a target artificial intelligence language model to perform a summarization process on the historical conversation content to obtain the target conversation content, where the target artificial intelligence language model is the preset artificial intelligence language model or a non-preset artificial intelligence language model.

3. The dialogue processing method according to claim 2, wherein The invoking a target artificial intelligence language model to perform a summarization process on the historical conversation content to obtain the target conversation content when the total number of first tokens is greater than the first token threshold includes: When the total number of first tokens is greater than the first token threshold, generating a conversation summarization instruction corresponding to the historical conversation content; Taking the historical conversation content and the conversation summarization instruction as input content of the target artificial intelligence language model and inputting them into the target artificial intelligence language model; Taking the reply content output by the target artificial intelligence language model according to the historical conversation content and the conversation summarization instruction as the target conversation content.

4. The dialogue processing method according to claim 3, characterized in that The taking the historical conversation content and the conversation summarization instruction as input content of the target artificial intelligence language model and inputting them into the target artificial intelligence language model includes: When the total number of second tokens corresponding to the historical conversation content and the conversation summarization instruction is less than or equal to the second token threshold corresponding to the target artificial intelligence language model, taking the historical conversation content and the conversation summarization instruction as input content of the target artificial intelligence language model and inputting them into the target artificial intelligence language model, where the first token threshold is less than the second token threshold.

5. The dialogue processing method according to claim 3, wherein The taking the historical conversation content and the conversation summarization instruction as input content of the target artificial intelligence language model and inputting them into the target artificial intelligence language model includes: When the total number of second tokens corresponding to the historical conversation content and the conversation summarization instruction is greater than the second token threshold corresponding to the target artificial intelligence language model, taking the difference between the second token threshold and the number of tokens corresponding to the conversation summarization instruction as the remaining number of tokens; Dividing the historical conversation content into multiple split contents according to the remaining number of tokens; Separate the different split contents and the dialogue summary instruction and input them into the target artificial intelligence language model as the input content of the target artificial intelligence language model, wherein the total number of tokens corresponding to the dialogue summary instruction and the split content is less than or equal to the second token threshold; The step of using the reply content output by the target artificial intelligence language model according to the historical dialogue content and the dialogue summary instruction as the target dialogue content includes: Determine the target dialogue content according to the reply content output by the target artificial intelligence language model for the dialogue summary instruction and different split contents.

6. The dialogue processing method according to claim 5, wherein The step of determining the target dialogue content according to the reply content output by the target artificial intelligence language model for the dialogue summary instruction and different split contents includes: Use the reply content output by the target artificial intelligence language model for the dialogue summary instruction and different split contents as the refined content corresponding to each split content; Input the dialogue summary instruction and all the refined contents into the target artificial intelligence language model as the input content of the target artificial intelligence language model, wherein the total number of tokens corresponding to the dialogue summary instruction and all the refined contents is less than or equal to the second token threshold; Use the reply content output by the target artificial intelligence language model for the dialogue summary instruction and all the refined contents as the target dialogue content.

7. The dialogue processing method according to claim 1, wherein After summarizing the historical dialogue content to obtain the target dialogue content when the total number of the first tokens is greater than the first token threshold, the method further includes: Update the historical dialogue content corresponding to the target account to the target dialogue content.

8. A dialogue processing device, characterized in that, The device includes: An acquisition module, configured to acquire the historical dialogue content corresponding to the target account when receiving a dialogue instruction sent by the target account; A determination module, configured to determine the total number of the first tokens corresponding to the dialogue instruction and the historical dialogue content; A judgment module, configured to summarize the historical dialogue content to obtain the target dialogue content when the total number of the first tokens is greater than the first token threshold, wherein the total number of tokens corresponding to the target dialogue content and the dialogue instruction is less than the first token threshold, and the first token threshold is less than or equal to the maximum token limit corresponding to the preset artificial intelligence language model; An input module, configured to input the dialogue instruction and the target dialogue content into the preset artificial intelligence language model as the input content of the preset artificial intelligence language model; A feedback module, configured to feedback the reply content output by the preset artificial intelligence language model according to the dialogue instruction and the target dialogue content to the target account.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the dialogue processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dialogue processing method according to any one of claims 1 to 7.

Images