Interaction method, electronic device, readable storage medium, and program product

Abstract

The present application relates to the technical field of computers, and in particular to an interaction method, an electronic device, a readable storage medium, and a program product. The method comprises: first acquiring conversation scene data of two conversation parties; then determining whether the conversation scene data needs to be retrieved; if not, directly generating a reply on the basis of the conversation scene data; and if yes, performing accurate retrieval from a resource database to obtain a required retrieval result, thereby generating a reply on the basis of the obtained retrieval result. It can be understood that whether retrieval needs to be performed is determined, specific information is retrieved from the resource database when retrieval needs to be performed, and a reply is generated on the basis of the retrieval result, that is, the reply content comprises the content of the retrieval result, so that the generated reply is more accurate, thereby improving the experience of a user using the conversation system.

Description

An interaction method, electronic device, readable storage medium and program product

[0001] The present application claims priority to the Chinese patent application No. 202410211616.7, filed on February 26, 2024, entitled "An interaction method, electronic device, readable storage medium and program product", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of computer, and particularly relates to an interaction method, electronic device, readable storage medium and program product. BACKGROUND

[0003] At present, artificial intelligence technology can be applied to the scene of message transmission between people through electronic devices. For example, when people chat through the application program in the mobile phone, the application program (APP) used by each user can help the user to generate reply content based on the message sent by the opposite party, so as to assist the user to reply to the opposite party.

[0004] In the prior art, since the large model has semantic analysis and logical reasoning ability, the large model, such as artificial intelligence generated context (AIGC) and other large language models, can be used to generate reply content according to the content based on network communication between the two parties, so as to assist the user to answer the message sent by the opposite party. However, in some retrieval scenes at present, for example, when the dialogue scene involves watching movies, activity arrangement and other content, it is impossible to generate accurate replies.

[0005] SUMMARY

[0006] In order to solve the above problems, the embodiments of the present application provide an interaction method, electronic device, readable storage medium and program product, which are used to solve the problem that in the one-on-one dialogue scene, when retrieval is needed, accurate replies cannot be provided to the user in real time.

[0007] In a first aspect, the embodiments of the present application provide an interaction method, which comprises: displaying a first user dialogue; in the case that the first user dialogue is a user dialogue that needs to be retrieved, displaying a first reply related to a retrieval result corresponding to the first user dialogue, the retrieval result being obtained by retrieving according to a retrieval intention of the first user dialogue; in the case that the first user dialogue is a user dialogue that does not need to be retrieved, displaying a second reply to the first user dialogue.

[0008] It can be understood that during the chat of the two parties, the user conversation will be displayed on the electronic device. At this time, the electronic device can display the reply in real time during the user chat process to help the user send a message to the other party. And sometimes when the user chats, some topics need to be retrieved in the database, such as involving watching movies, activity arrangement and the like, at which time retrieval needs to be performed. Therefore, different replies can be displayed according to whether the user conversation is a user conversation that needs to be retrieved. If retrieval is needed, for example, the current conversation scene involves knowledge questions and answers, personal activity arrangement, user information, watching movies, shopping and the like, at which time the first reply related to the retrieval result can be displayed, that is, the first reply includes content related to the retrieval result. If retrieval is not needed, for example, the two parties in the conversation are just chatting, the second reply is displayed, and the second reply does not include content related to the retrieval result.

[0009] It can be understood that since it is judged whether retrieval is needed, and different ways are adopted for processing when retrieval is needed and when retrieval is not needed, the generated reply is more targeted. When retrieval is needed, since the generated reply includes content related to the retrieval result, the generated reply is more accurate, and the user experience of using the conversation system is improved.

[0010] It can be understood that since the retrieval result is obtained according to the retrieval intention of the first user conversation, in the scene where retrieval is not needed, that is, the retrieval intention does not need to be recognized, time and computing resources are saved.

[0011] In a possible implementation of the first aspect, the method further includes: displaying the first reply or the second reply in the form of a card or a list, and the first reply or the second reply can include at least one data form of text, image, voice, video and link.

[0012] In a possible implementation of the first aspect, the method further includes:

[0013] obtaining conversation scene data related to the first user conversation, the conversation scene data including at least one of conversation content of the first user conversation, behavior information of the user for the first user conversation, and physiological indicators of the user for the first user conversation; calling a large language model, and the large language model judging whether the first user conversation is a user conversation that needs to be retrieved according to the conversation scene data; in the case where the first user conversation is a user conversation that needs to be retrieved, the large language model identifies a retrieval intention of the first user conversation according to the conversation scene data.

[0014] It can be understood that the dialogue content of the first user dialogue can include the current and historical dialogue content of the two parties in the dialogue; the behavior information of the user for the first user dialogue can include the user behavior information such as editing, browsing, page sliding of the current user in the dialogue system; and the physiological indicators of the user in the first user dialogue can include heart rate, expression, etc. It can be understood that using multi-dimensional dialogue scene data can improve the accuracy of determining whether retrieval is needed, so that when retrieval is needed, it can be reflected in time that retrieval is needed, thereby improving the accuracy and timeliness of dialogue processing. Moreover, since the dialogue scene data includes multi-dimensional data, the elements for generating a reply are more abundant, and when a reply is generated according to multi-dimensional data, the generated reply is more accurate.

[0015] In a possible implementation of the first aspect, the first reply includes a fusion retrieval result corresponding to the retrieval result.

[0016] It can be understood that the fusion retrieval result not only includes the content corresponding to the retrieval result, but also includes a retrieval reply corresponding to the retrieval result that can be used to reply to the message sent by the other party, and the retrieval reply corresponds to the retrieval result, so that the user can obtain a more comprehensive, more relevant and more accurate reply.

[0017] In a possible implementation of the first aspect, the fusion retrieval result is obtained by: a large language model generating the fusion retrieval result according to the input retrieval result and the dialogue scene data of the first user dialogue.

[0018] It can be understood that taking the retrieval result and the dialogue scene data of the first user dialogue as inputs of the large language model enables the large model to fuse the retrieval result corresponding to the retrieval intent and the content analyzed from the dialogue scene data, thereby providing a high-quality reply to the user.

[0019] In a possible implementation of the first aspect, the retrieval result is obtained by: performing retrieval in a resource database according to retrieval information corresponding to the retrieval intent to obtain the retrieval result, wherein the retrieval information includes at least one of a keyword corresponding to the retrieval intent and a search sentence corresponding to the retrieval intent; the resource database includes a local resource database or a cloud resource database, and the local resource database includes local data or a local application program; and the cloud resource database includes an application program or a service deployed in the cloud.

[0020] It can be understood that since retrieval is performed in various resource databases, the obtained retrieval result is sufficient.

[0021] In a possible implementation of the first aspect, the method further includes: in a case where the first user conversation is a user conversation that needs to be searched, generating, by the large language model, the second reply according to the conversation scenario data.

[0022] It can be understood that the large language model has semantic understanding and reasoning capability, and the large language model can effectively generate the second reply according to the conversation scenario data.

[0023] In a possible implementation of the first aspect, the second reply includes a conversation reply and a reply reason corresponding to the conversation reply.

[0024] It can be understood that when the reply is generated according to the conversation scenario data in a case where searching is not needed, the generated reply can include the conversation reply and the reply reason. The reply reason is an analysis of the generated conversation reply, and the reply reason or the conversation reply takes into account the emotional reactions of the conversation parties, thereby improving the accuracy of the reply information and avoiding the problem of causing the parties to be repelled due to inappropriate reply content. It can be understood that the electronic device can use the pre-trained large language model to obtain the conversation reply and the reply reason by taking the conversation scenario data as input, and the output reply needs to take into account the emotional tendencies of the parties, so that the large model can generate the conversation reply and the reply reason that meet the current user demand according to the conversation scenario data, and the output content can take into account the intentions and emotional tendencies of the parties.

[0025] In a second aspect, an embodiment of the present application provides an interaction method applied to an electronic device, the method including: displaying a first user conversation; and displaying a first reply of the first user conversation based on conversation scenario data of the first user conversation, wherein the conversation scenario data includes at least one of behavior information of a user with respect to the first user conversation and a physiological index of the user in the first user conversation.

[0026] It can be understood that the conversation content of the first user conversation can include current and historical conversation content of the conversation parties; the behavior information of the user with respect to the first user conversation can include user behavior information such as editing, browsing, and page sliding of the current user in the conversation system; and the physiological index of the user in the first user conversation can include heart rate, expression, and the like. It can be understood that the use of multi-dimensional conversation scenario data makes the elements for generating the reply more rich, and thus the generated reply is more accurate.

[0027] In a possible implementation of the second aspect, the first reply includes at least one of the following: a search result corresponding to the first user conversation, a fusion search result related to the search result, a conversation reply, and a reply reason corresponding to the conversation reply.

[0028] In a possible implementation of the second aspect, the method further includes: calling the large language model, and the large language model determining, according to the dialogue scene data, whether the first user dialogue is a user dialogue that needs to be retrieved; in a case where the first user dialogue is a user dialogue that needs to be retrieved, displaying the retrieval result or the fused retrieval result; and in a case where the first user dialogue is a user dialogue that does not need to be retrieved, displaying the dialogue reply and the reply reason corresponding to the dialogue reply.

[0029] It can be understood that, according to the dialogue scene data, it is determined whether retrieval is needed, and different content is displayed in a case where retrieval is needed and a case where retrieval is not needed, so that the displayed reply is more targeted, and user experience is improved.

[0030] In a possible implementation of the second aspect, the fused retrieval result is obtained in the following manner: the large language model identifies, according to the dialogue scene data, a retrieval intention of the first user dialogue; the large language model performs retrieval in a resource database according to retrieval information corresponding to the retrieval intention to obtain the retrieval result, where the retrieval information includes at least one of a keyword corresponding to the retrieval intention and a search sentence corresponding to the retrieval intention; and the large language model generates the fused retrieval result according to the input retrieval result and the dialogue scene data of the first user dialogue.

[0031] In a possible implementation of the second aspect, the dialogue reply and the reply reason corresponding to the dialogue reply are obtained in the following manner: in a case where the first user dialogue is a user dialogue that needs to be retrieved, the large language model generates the dialogue reply and the reply reason corresponding to the dialogue reply according to the dialogue scene data.

[0032] In a possible implementation of the second aspect, the method further includes: displaying the first reply in the form of a card or a list, and the first reply can include at least one of a text, an image, a voice, a video, and a link.

[0033] In a third aspect, an embodiment of the present application provides an interaction method applied to an electronic device, and the method includes: displaying a first user dialogue; and displaying a first reply corresponding to the first user dialogue, where the first reply includes a dialogue reply and a reply reason corresponding to the dialogue reply.

[0034] It can be understood that, according to the dialogue scene data, the reply generated in the reply generation process can include the dialogue reply and the reply reason. The reply reason is an analysis of the dialogue reply generated, and the reply reason or the dialogue reply takes into account the emotional reactions of both parties in the dialogue, so that the accuracy of the reply information is improved, and the problem of causing both parties to feel repelled due to inappropriate reply content is avoided.

[0035] In one possible implementation of the third aspect mentioned above, the method further includes: acquiring dialogue scenario data related to the first user's dialogue, the dialogue scenario data including at least one of the dialogue content of the first user's dialogue, user behavior information in response to the first user's dialogue, and physiological indicators of the user in the first user's dialogue; invoking a large language model, and the large language model generating a first response based on the dialogue scenario data.

[0036] It is understandable that electronic devices can use pre-trained large language models to take dialogue scenario data as input, obtain dialogue responses and reasons for responses, and the output responses need to take into account the emotional inclinations of both parties, so that the large model can generate dialogue responses and reasons for responses that meet the current user's needs based on dialogue scenario data, and the output content can take into account the intentions and emotional inclinations of both parties.

[0037] In one possible implementation of the third aspect mentioned above, the method further includes: the large language model determining whether the first user dialogue is a user dialogue that needs to be retrieved based on the dialogue scenario data; and generating the first response if the first user dialogue is a user dialogue that does not need to be retrieved.

[0038] Fourthly, embodiments of this application provide an interaction method applied to an electronic device. The method includes: displaying a first user dialogue, wherein the first user dialogue is a user dialogue that needs to be searched; displaying a first response related to the search results corresponding to the first user dialogue, wherein the first response includes fused search results corresponding to the search results, wherein the search results are obtained by searching according to the search intent of the first user dialogue.

[0039] In one possible implementation of the fourth aspect above, it further includes: acquiring dialogue scenario data related to the dialogue with the first user, the dialogue scenario data including at least one of the dialogue content of the first user dialogue, the user's behavioral information in response to the dialogue with the first user, and the physiological indicators of the user in the dialogue with the first user; invoking a large language model, and the large language model generating the first response based on the dialogue scenario data and the retrieval results.

[0040] In one possible implementation of the fourth aspect mentioned above, it also includes: the large language model determining whether the first user dialogue is a user dialogue that needs to be retrieved based on the dialogue scenario data; and if the first user dialogue is a user dialogue that needs to be retrieved, the large language model generating a first response.

[0041] In a fifth aspect, an electronic device is provided, including a memory configured to store instructions, and one or more processors configured to execute the instructions to perform the interaction method of the first aspect above and any of the implementations of the first aspect, the interaction method of the second aspect above and any of the implementations of the second aspect, the interaction method of the third aspect above and any of the implementations of the third aspect, or the interaction method of the fourth aspect above and any of the implementations of the fourth aspect.

[0042] In a sixth aspect, a computer-readable storage medium is provided, having stored thereon instructions to cause an electronic device to perform the interaction method of the first aspect above and any of the implementations of the first aspect, the interaction method of the second aspect above and any of the implementations of the second aspect, the interaction method of the third aspect above and any of the implementations of the third aspect, or the interaction method of the fourth aspect above and any of the implementations of the fourth aspect.

[0043] In a seventh aspect, a computer program product is provided, including computer program code to, when run on a computer, cause the computer to perform the interaction method of the first aspect above and any of the implementations of the first aspect, the interaction method of the second aspect above and any of the implementations of the second aspect, the interaction method of the third aspect above and any of the implementations of the third aspect, or the interaction method of the fourth aspect above and any of the implementations of the fourth aspect.

[0044] The advantages of the fifth aspect to the seventh aspect can refer to the advantages of the first aspect, the second aspect, the third aspect, and the fourth aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0045] FIG. 1A illustrates a schematic diagram of a human-to-human chat conversation scenario, according to some embodiments of the present application;

[0046] FIG. 1B illustrates a schematic diagram of a chat interface 001, according to some embodiments of the present application;

[0047] FIG. 2A illustrates a schematic diagram of a chat interface 002, according to some embodiments of the present application;

[0048] FIG. 2B illustrates a schematic diagram of a chat interface 003, according to some embodiments of the present application;

[0049] FIG. 3 illustrates a schematic diagram of an interaction method, according to some embodiments of the present application;

[0050] FIG. 4A illustrates a system architecture 400, according to some embodiments of the present application;

[0051] FIG. 4B illustrates an interaction between a terminal 100B and a cloud 200, according to some embodiments of the present application;

[0052] FIG. 5A illustrates an interface 004, according to some embodiments of the present application;

[0053] FIG. 5B illustrates a retrieval procedure, according to some embodiments of the present application;

[0054] FIG. 6 illustrates modules of a terminal 100 and a cloud 200, according to some embodiments of the present application;

[0055] FIG. 7 illustrates modules of another terminal 100 and another cloud 200, according to some embodiments of the present application;

[0056] FIG. 8 illustrates an apparatus, according to some embodiments of the present application. DETAILED DESCRIPTION

[0057] The illustrative embodiments of the present application include, but are not limited to, an interaction method, an electronic device, a readable storage medium, and a program product.

[0058] As mentioned above, currently, when the conversation content involves some scenarios that need to be retrieved, the conversation parties cannot generate accurate replies in the process of chatting.

[0059] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0060] FIG. 1A illustrates a human-to-human chatting conversation scenario, according to some embodiments of the present application. User KA and user KB use the same chatting application (for example, instant messaging application 1 in the figure) on terminals to communicate. It can be understood that the chatting application is a conversation system, so that the conversation parties can have a chatting conversation. For the convenience of description, the conversation system on terminal 100A is referred to as “conversation system A0”, and the conversation system on terminal 100B is referred to as “conversation system B0”. At this time, user KA and user KB use conversation system A0 and conversation system B0 to have a conversation, respectively.

[0061] Currently, the conversation system can also recommend automatic reply content for the user based on the chatting record of the user.

[0062] FIG. IB shows a chat interface 001 of a dialogue system B0 based on a chat process between a user KA and a user KB according to some embodiments of the present application. The sending content of "User A-1" on the interface 001 is the chat content sent by the user KA through the terminal 100A, and the sending content of "User B-1" on the interface 001 is the chat content sent by the user KB through the terminal 100B. Specifically, referring to the chat content on the interface 001, the user KA sends the content corresponding to the user A-1 in the figure "What do you have for the rest of the day?" through the terminal 100A to the terminal 100B; the user KB replies to the content corresponding to the user B-1 in the figure "I have nothing for the rest of the day" through the terminal 100B to the user KA; the user KA continues to send the content corresponding to the user A-1 in the figure "That?" through the terminal 100A to the terminal 100B. At this time, the user KB replies to the content corresponding to the user B-1 in the figure "Let's go and have fun together!" and the user KA continues to reply to the content corresponding to the user A-1 in the figure "Let's go and watch the newly released movie 1". At this time, the dialogue system generates a simple reply "OK!" in the prompt card K11 as shown in FIG. IB.

[0063] It can be understood that the dialogue scenario shown in FIG. IB includes content related to the movie 1, but the generated reply does not involve the movie content, and the generated reply is not accurate enough. It can be understood that at this time, if the user needs to obtain accurate information about the movie 1, the user needs to search in an application program where movie tickets can be purchased to obtain accurate information about the movie 1.

[0064] Therefore, in order to solve the above problems, the present application proposes an interaction method, in which the dialogue scenario data of the dialogue parties can be obtained first, and then it is judged whether the dialogue scenario data needs to be searched. If it does not need to be searched, for example, the dialogue parties are just chatting, then a reply is generated directly according to the dialogue scenario data; if it needs to be searched, for example, the current dialogue scenario involves knowledge questions and answers, personal activity arrangements, user information, watching movies, shopping, etc., then a precise search is performed in a resource database, for example, a local resource database or a cloud resource database, to obtain the required search result, so as to generate a reply based on the obtained search result. It can be understood that since it is judged whether the search needs to be performed, and different ways are adopted for processing when the search needs to be performed and when the search does not need to be performed, the generated reply is more targeted. Moreover, when in the scenario where the search needs to be performed, specific information needs to be searched in the resource database, and a reply is generated based on the search result, i.e., the content of the reply includes the content of the search result, so that the generated reply is more accurate, and the user experience of using the dialogue system is improved. In addition, when in the scenario where the search does not need to be performed, a reply is generated directly according to the dialogue scenario data, so that time and computing resources are saved at this time.

[0065] For example, for the scenario shown in FIG. 1B described above, when it is determined that retrieval is needed, the large model can be used to understand and analyze the dialogue, so as to obtain a keyword or phrase related to watching a movie that can be used for retrieval, for example, a keyword "movie 1" is obtained. Then, according to the keyword, retrieval is performed in various resource databases to obtain accurate information, and then a reply related to the retrieval result is generated according to the retrieval result, so that the generated reply is more accurate.

[0066] In some embodiments, in order to improve the accuracy of the judgment result of whether retrieval is needed, multi-dimensional data can be obtained as dialogue scene data. For example, the dialogue scene data can include not only the current and historical dialogue content of the two parties in the dialogue, but also the user behavior information of the current user in the dialogue system, such as editing, browsing, page sliding, and the physiological index data of the current user, such as heart rate, expression, etc., which are used to judge whether retrieval is needed, so as to improve the accuracy of judging whether retrieval is needed, so that when retrieval is needed, the need for retrieval can be timely reflected, and the accuracy and timeliness of dialogue processing are improved. In addition, since the dialogue scene data includes multi-dimensional data, the elements for generating a reply are more abundant, and when a reply is generated according to the multi-dimensional data, the generated reply is more accurate.

[0067] In addition, in some embodiments, when retrieval is not needed, according to the dialogue scene data, the reply generated can include a dialogue reply and a reply reason. The reply reason is an analysis of the generated dialogue reply, and the dialogue reply or the reply reason takes into account the emotional reactions of the two parties in the dialogue, so as to improve the accuracy of the reply information and avoid the problem of causing both parties to be disgusted due to inappropriate reply content. For example, the terminal device can use a pre-trained large model to obtain a dialogue reply and a reply reason by taking the dialogue scene data as input, and the output reply needs to take into account the emotional tendencies of both parties, so that the large model can generate a dialogue reply and a reply reason that meet the current user's needs according to the dialogue scene data, and the output content can take into account the intentions and emotional tendencies of both parties.

[0068] FIG. 2A shows a chat interface 002 of a terminal 100B based on a dialogue system B1 during a chat between a user KA and a user KB, according to some embodiments of the present application. On the interface 002 of the terminal 100B, a content “What?” sent by the user A-2 to the user B-2 is displayed. The user KB can perform some reply operations in an input box K21 of the dialogue system B1 in response to the content sent by the user KA, such as clicking and inputting the text “I am ready to go to the internet bar to play” in the input box K21. When the dialogue system B1 detects that the duration of the user clicking and inputting the text reaches a preset reply duration, a large model can be called, and the text input by the current user and the previous dialogue content between the dialogue parties are taken as dialogue scene data, so that the large model generates a reply according to the dialogue scene data; the psychological index data of the current user KB and the behavior information of the user KB can also be collected, so that the dialogue scene data includes the psychological index data of the current user KB and the behavior information of the user KB, for the large model to generate a reply. It can be understood that the large model can analyze the intention of the user KA chatting with the current user KB according to the dialogue scene data, which is actually to expect to do something with the user KB, and then the large model automatically generates a reply including a dialogue reply and a reply reason. The dialogue system B1 displays the generated reply in the form of a suggestion card K22.

[0069] As shown in the suggestion card K22 in FIG. 2A, the reply includes a box K22A corresponding to the reply reason and a box K22B corresponding to the dialogue reply. Referring to FIG. 2A, the box K22A is a specific reply reason “The other party asks about your arrangement for the evening, and expects to do something with you. Your current reply may cause the other party to have an emotional problem”. And the box K22B is a dialogue reply generated by the large model and displayed by the dialogue system B1 “Let's go and play together!”. The user KB can click a sending control K22C corresponding to the dialogue reply “Let's go and play together!” to directly send the dialogue reply to the user KA. And the user KB can directly select and modify the dialogue reply “Let's go and play together!” in the box K22B, and then send the modified content by clicking the sending control K22C. In addition, the user KB can modify the original content in the input box according to the displayed dialogue reply “Let's go and play together!”, such as modifying it to the same content as the dialogue reply “Let's go and play together!”, and then the user clicks a sending control K23 corresponding to the content in the input box to send. It can be understood that the sending control can not be displayed in the suggestion card K22, and the user can directly select the dialogue reply and the reply reason in the suggestion card, and then the terminal 100B directly sends them to the user KA, and the specific sending method is not required here.

[0070] It can be understood that the specific reply reason displayed in the suggestion card K22 analyzes the intention of the user KA chatting with the current user KB, which is actually expecting to do something with the user KB, so the original reply of the user KB may cause the user KA to be in a bad mood. And according to the obtained psychological index data of the user KB, such as normal heart rate and calm mood, it also shows that the user KB does not have a repulsion to the user KA, so a suitable dialogue reply “Let's play together!” can be given. At this time, the generated reply reason and dialogue reply fully consider the emotional tendencies of both parties, providing the current user with more accurate, friendly and appropriate replies, so that the user will not cause the other party to be repelled or uncomfortable when communicating with the other party according to the accurate and reasonable reply.

[0071] In some other embodiments, when retrieval is needed, the retrieval results can be obtained by searching various resource databases, and then the retrieval results obtained from various resource data are fused with the content analyzed from the dialogue scene data to provide the user with high-quality replies. For example, the large model can fuse the retrieval results obtained from various resource data and the reply content generated by the large model from the dialogue scene data, so that the generated reply includes the fused retrieval results corresponding to the retrieval results. It can be understood that the fused retrieval results not only include the content corresponding to the retrieval results, but also include the retrieval replies corresponding to the retrieval results that can be used to reply to the messages sent by the other party, and the retrieval replies correspond to the retrieval results. It can be understood that the retrieval reply reflects the reply content generated by the large model from the dialogue scene data, and since the obtained retrieval results may include information from the local and cloud services, the fused retrieval results generated and displayed at this time enable the user to obtain more comprehensive, more relevant and more accurate replies.

[0072] For example, FIG. 2B shows a chat interface 003 of a terminal 100B based on a dialogue system B1 in a chat process between a user KA and a user KB according to some embodiments of the present application. The dialogue content of "User A-2" and "User B-2" on the interface 003 is the same as that of "User A-1" and "User B-1" sent in FIG. IB. And the user KB inputs "Let's see where to watch" in the input box K31. At this time, the dialogue system B1 generates a reply as shown in the suggestion card K32 in FIG. 2B. The reply displayed in the suggestion card K32 includes the fusion search result. Referring to FIG. 2B, the content corresponding to the search result "Helping you search for ticket information for Movie 1 to watch tonight: XXX Cinema (XXX Store): ￥49.9, tonight: 19:30 | 20:00 | 21:15" is displayed in the box K32A. And the dialogue system generates a search reply corresponding to the search result "Shall we go to this place to watch?" in the box K32B. It can be understood that the content related to watching a movie in the current dialogue is strongly related to the application program related to watching a movie. At this time, the terminal can search for effective information in the corresponding application program, and the generated reply is presented in a better way. This enables the user KB to quickly obtain the information he wants without having to search for other resource databases, and the fusion search result includes the content generated by the large model based on the dialogue scene data, so that the user does not need to input a search reply related to the searched movie ticket information, improving the user experience. Similarly, the user can modify the content in the box K32A or the box K32B, and then send it through the corresponding sending control K32C and the sending control K32D, or directly send the content in the box K32A or the box K32B through the sending control. The user can also modify the content in the input box K31 according to the reply in the suggestion card K32, and then send it through the sending control K33 corresponding to the input box K31. The user can also directly select the content in the suggestion card K32 for sending without sending through the sending control. The specific sending method is not required here.

[0073] It can be understood that the large language model includes but is not limited to artificial intelligence generated context (AIGC), ChatGPT, LaMDA, PaLM, OPT-IML, which are not described here.

[0074] It can be understood that the terminal can display the reply through a card, a list or other forms to enable the user to intuitively obtain relevant information.

[0075] It can be understood that the dialogue system can be any application program that can provide chat functions and can be deployed on any electronic device.

[0076] It can be understood that the terminal 100A and the terminal 100B can be any electronic device running a dialogue system, for example, can include but is not limited to a mobile phone, a tablet computer, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a server, a server cluster, etc., without limitation here.

[0077] FIG. 3 provides a schematic diagram of an interaction method according to an embodiment of the present application. In the method, the terminal device displays a user dialogue, and determines whether retrieval is needed. In different cases where retrieval is needed and retrieval is not needed, different replies are used. For convenience of description, the processor in the terminal device is taken as the execution subject for description, and the specific steps are as follows:

[0078] S301, dialogue scene data of a current user dialogue is acquired.

[0079] In some embodiments, the terminal device displays a user dialogue, and the dialogue process on the dialogue system can be monitored in real time. When it is determined that a reply needs to be generated, dialogue scene data related to the current user dialogue is collected. For example, when a first operation of the user is detected, the dialogue scene data is acquired. It can be understood that the user dialogue is any dialogue content displayed on the terminal device, for example, chat messages sent by both parties of the dialogue displayed on the terminal device, etc.

[0080] The first operation can be that the user clicks an input box of an interaction interface, the user inputs text, an emoticon, etc. in the input box, or the user clicks and inputs in the input box for a preset reply duration, etc. The first operation can also be that the user selects a microphone for voice input. The first operation can also be that a reply control (not shown in the figure) is configured on the interface, and the user selects the reply control. It can be understood that the first operation can be various operations, as long as the operation can indicate that the user needs to chat with the other party, which will not be described here.

[0081] The dialogue scene data can include, but is not limited to, current and historical dialogue content of both parties, user behavior information, and physiological index data of the current user, and the like. The dialogue content includes sent content and pre-sent content, which can be text, voice, video, picture, and the like. The user behavior information includes user behavior information such as editing, browsing, and page sliding of the current user in the dialogue system. For example, the current chat content relates to a topic that has been discussed in the past, at which time the user can return to the previous chat content for viewing and editing. At this time, the user behavior information includes user page sliding operations, editing operations on past chat content, and viewed chat content data. The physiological index data of the current user includes the user's expression, heart rate, and the like. For example, for the expression, the terminal device can obtain the user's expression through the camera under the permission of the user. The expression can correspond to emotion categories such as anger, calm, happiness, and sadness. For the heart rate, the user's heart rate data can be obtained through an internal device or other supporting devices that can monitor the heart rate. It can be understood that the dialogue scene data can only include dialogue content, which is not limited herein. It can be understood that when using multi-dimensional dialogue scene data, the accuracy of the judgment result can be improved due to the comprehensive information of the dialogue scene data, so that the retrieval can be performed in real time when retrieval is needed, and the accuracy and timeliness of dialogue processing are improved.

[0082] For example, for the scenario shown in FIG. 2A, when it is detected that the user KB inputs text in the input box K21, the last 10 pieces of sent content of the user A-2 and the user B-2 in the current chat dialogue, and the pre-sent content input by the user KB in the input box K21 can be obtained. The expression of the user collected by the camera, such as “calm”, and the heart rate data of “75bpm” collected by the watch can also be obtained.

[0083] For example, for the scenario shown in FIG. 2B, when it is detected that the user KB inputs text in the input box K31, the last 10 pieces of sent content of the user A-2 and the user B-2 in the current chat dialogue, and the content input by the user KB in the input box K31 can be obtained. The user slides to the topic of “movie 1” of the previous day, and the behavior information of viewing the previous content. The expression of the user collected by the camera, such as “calm”, and the heart rate data of “75bpm” collected by the watch can also be obtained.

[0084] In S302, whether retrieval is needed is determined according to the dialogue scene data.

[0085] If not, S303 is entered, and if yes, S304 is entered.

[0086] It can be understood that the current user dialogue can involve some content that needs to be retrieved, such as movies, food, restaurants, buildings, names, trademarks, place names, customs, travel services, transportation, gossip information, activity arrangements, and specific content of various disciplines such as law, music, painting, medicine, etc. not listed here. And the current user dialogue can have multiple contents that need to be retrieved, so it is necessary to determine whether the current user dialogue is a user dialogue that needs to be retrieved. In the case of a current user dialogue that needs to be retrieved, it needs to be retrieved, and in the case of a current user dialogue that does not need to be retrieved, it does not need to be retrieved. Therefore, when retrieval is needed, the retrieval intent can be accurately identified subsequently, and the retrieval result can be obtained. And when retrieval is not needed, retrieval intent identification and subsequent retrieval are not needed, and a reply is directly generated, thereby saving time and computing resources.

[0087] In some embodiments, the terminal device calls the trained large model to determine whether the current dialogue scene data needs to be retrieved. For example, the large model can determine whether the current user dialogue is a user dialogue that needs to be retrieved, i.e., whether retrieval is needed, by whether it extracts keywords or key sentences in the current dialogue scene data that involve content that needs to be retrieved. When it is determined that retrieval is needed using the large model, S304 is entered, and when it is determined that retrieval is not needed using the large model, S303 is entered.

[0088] For example, for the scenario corresponding to the idle chat shown in FIG. 2A, the large model can be used to determine that retrieval is not needed, and S303 is entered.

[0089] For example, for the scenario shown in FIG. 2B, the large model can be used to identify keywords related to movie content, thereby determining that retrieval is needed, and S304 is entered.

[0090] It can be understood that the large model can be deployed on the terminal device or in the cloud. When the large model is deployed in the cloud, the terminal device can send the dialogue scene data to the cloud, so that the cloud specifically performs the process of determining whether retrieval is needed according to the dialogue scene data, i.e., step S302 can also be performed in the cloud, and the cloud feeds back the determination result of the large model to the terminal device.

[0091] Specifically, the large model can include a discriminant model to determine whether retrieval is needed. It can be understood that the discriminant model can include a neural network model, a convolutional network model, and other algorithms, etc., which are not required here. In some embodiments, any electronic device, such as any terminal device or cloud device, can be used to train the discriminant model in the large model in advance.

[0092] Taking the cloud device training the large model as an example, the discriminant model can be trained in the following way: the dialog scene data related to various dialog scenes is prepared in advance as training sample data. For the convenience of description, the dialog scene data used for training is referred to as “dialog scene sample data”. And each dialog scene sample data is configured with a corresponding label. The label of the dialog scene sample data that needs to be retrieved is configured as “retrievable identifier”, and the label of the dialog scene sample data that does not need to be retrieved is configured as “non-retrievable identifier”. Then the training sample data and the corresponding label are input into the initial model to be trained to perform model training, thereby obtaining the discriminant model.

[0093] It can be understood that the dialog scene sample data not only includes dialog scenes that do not need to be retrieved, such as asking the other party how old they are this year, home address, clothing, praising the other party, etc. for casual conversation, but also includes dialog scenes that need to be retrieved. Among them, the dialog scene sample data that needs to be retrieved can be generated by using the pre-configured resource database, so that the training sample data includes various contents of multiple dimensions. It can be understood that the resource database configured in the training process is only used for the training process. When the large model is trained and actually used, the resource database for retrieval can include but not limited to the aforementioned resource database, or can not include the aforementioned resource database, which is not required here.

[0094] For example, the pre-configured resource database for training the model can be: (1) Knowledge base: containing data of common questions and related answers of users, such as the birth date of “Qin Shihuang” and other data with related answers. (2) External data source: including external data sources related to user information and other information, such as online activity calendar, social media platform or user personal calendar, etc. (3) Third-party application (APP) information: including registration information and specific function information of third-party application (APP). It can be understood that the third-party application can provide various related functions, and retrieval is needed when the function is involved, so it can be used as important data to determine whether retrieval is needed. Optionally, the dialog scene sample data used for training can not only include dialog content, but also include user behavior information and psychological indicator data.

[0095] It can be understood that the data content in the foregoing several data sources is more involved in the conversation in life. At this time, the conversation scene sample data can be generated according to the above-mentioned data sources, which is widely representative.

[0096] It can be understood that the discriminant model can also be obtained by other means, which will not be described here.

[0097] S303, generating a reply according to the conversation scene data.

[0098] It can be understood that step S303 is an optional step.

[0099] It can be understood that after the retrieval and judgment according to the conversation scene data, only steps S304-S306 can be executed.

[0100] In some embodiments, the terminal device generates a reply according to the conversation scene data, and the generated reply can only include a conversation reply.

[0101] In some other embodiments, the terminal device generates a reply according to the conversation scene data, and the generated reply can include a conversation reply and a reply reason corresponding to the conversation reply.

[0102] For example, referring to the interface 002 shown in FIG. 2A, the conversation reply and the reply reason displayed based on the current conversation scene are shown in the prompt card K22.

[0103] In some embodiments, the terminal device can generate a reply according to the conversation scene data, which can be executed by a large model. The large model can include a generation model for generating a reply according to the conversation scene data and other information. The generation model can be a generative model such as a recurrent neural network or a transformer model, which is not required here.

[0104] It can be understood that the large model can be deployed on the terminal device or the cloud device. When the large model is deployed on the cloud device, the terminal device can receive the reply content sent by the cloud to obtain the reply.

[0105] In some embodiments, any electronic device, such as any terminal device or cloud device, can be used to train the generation model in the large model in advance. For example, the cloud device trains the large model by using the Prompt fine-tuning training method to obtain the generation model.

[0106] Specifically, a large amount of dialogue scenario sample data obtained from the data source will be input into the initial model for training, enabling the model to learn language capabilities and thus complete the context, such as using pre-configured templates with fixed structures. Then, the parameters of a certain part of the model will be fine-tuned to ensure that the output includes reasonable dialogue responses and reasons for those responses, taking into account the intentions and emotional inclinations of both parties. For example, a pre-defined template structure will be defined and filled into the appropriate positions to generate human-readable responses and suggestions. For instance, the template could be: "The other party is asking about your plans for tonight (where 'tonight' represents the time), hoping to do something with you ('do something with you' represents the action content), and your current response might cause the other party to feel bad ('the other party's bad mood' represents the consequence)." Then, key information will be filled into the template to obtain the corresponding response.

[0107] Understandably, other dialogue understanding technologies and semantic understanding algorithms can be used to understand the user's question and recommend responses that match the user's intent and take into account factors such as the other party's emotions and tone, in order to provide a satisfactory answer.

[0108] S304, Identify the search intent and obtain search information corresponding to the search intent.

[0109] In some embodiments, since a retrieval is determined to be necessary (i.e., the dialogue scenario data contains content that can be retrieved), the terminal device can invoke a large model. This large model then identifies the retrieval intent based on the current dialogue scenario data. The terminal device then generates retrieval information, such as query parameters, that can be retrieved from various resource databases supported by the current terminal device, based on the retrieval intent. This retrieval is performed based on the retrieval information. It is understood that a retrieval intent is the intention to obtain precise information through accurate retrieval in various resource databases. The retrieval intent is a more precise representation of the intent used when retrieving content, and can cover various fields, including but not limited to music, law, writing, painting, learning, and daily life. The retrieval information can be search sentences, keywords, etc., corresponding to the retrieval intent.

[0110] Understandably, in some implementations, the intent recognition capability of the large model can be deployed on either a terminal device or a cloud device. When the intent recognition capability of the large model is deployed on a terminal device, the terminal device can identify the search intent and generate search information when a search is required. When the intent recognition capability of the large model is deployed on a cloud device, the cloud device can continue to identify the search intent and generate search information when a search is required; that is, step S304 can be executed in the cloud.

[0111] Specifically, the large model can further include an intent recognition model for identifying the retrieval intent. The intent recognition model can be a neural network model, a convolutional network model, and other algorithms, without limitation. The intent recognition model can understand and classify the user's intent in the dialogue scenario to obtain a feature vector related to the intent. The output feature vector is input into a classification algorithm or a probability model to determine the most likely retrieval intent. It can be understood that in the case of retrieval, the retrieval intent is further accurately identified by the large model, so that the retrieval information corresponding to the retrieval intent can accurately retrieve the required content in various resource databases.

[0112] For example, referring to the interface 003 shown in FIG. 2B, the large model can identify the retrieval intent related to the movie 1, according to the identified retrieval intent, etc. Then, the retrieval information is generated based on the retrieval intent, for example, "order 1 movie ticket for movie 1 tonight", or the keywords "order", "movie 1", "watch", "tonight", etc. The form of the specific retrieval information is not required here.

[0113] In some implementations, when training the intent recognition model, the cloud device can use the Prompt fine-tuning training method to obtain the intent recognition model: train the model through training sample data including various corpora, so that the model learns the general language representation, and then fine-tune the pre-trained model according to different downstream tasks. For example, the intent recognition model is used to output a preliminary intent, and then the preliminary intent is fine-tuned to obtain a target retrieval intent, and the model is fine-tuned according to the target retrieval intent. The training sample data used to train the intent recognition model can be the dialogue scenario sample data from the configured resource database used in the process of training the discrimination model, which is specifically described in step S302 above and will not be repeated here.

[0114] S305, obtaining a retrieval result according to the retrieval information.

[0115] In some embodiments, according to the retrieval information, the retrieval is performed in various resource databases to obtain a retrieval result. The various resource databases can include a local resource database of the current terminal device and a cloud resource database connected to the current terminal device. For example, the data in the local resource database can be the data stored in the local storage space, including device information, user preferences, contacts, short messages, call records, photos, local videos, application data, etc. The data in the cloud resource database can be third-party application or service data provided by the cloud, such as various applications using positioning, ticketing, ticketing, reservation, etc. third-party services deployed in the cloud.

[0116] For example, for the above search information related to movie 1, the search information "Book 1 movie ticket for movie 1 tonight" and other information can be used to find the required content in the local data corresponding to the ticketing application in the terminal device, and in the cloud service corresponding to the ticketing application, to obtain the local search result and the cloud search result. For example, the cloud search result is obtained: XXX Cinema (XXX store): ￥49.9 for tonight: 19:30 | 20:00 | 21:15.

[0117] S306, generating a reply based on the search result.

[0118] In some embodiments, the search result is used as the reply. For example, for the above local search result and cloud search result related to movie 1, after merging and sorting, the combined result is displayed as a reply on the interface.

[0119] In other embodiments, the large model fuses the search result into the reply content obtained based on the dialogue scene data, thereby generating the reply, and the reply contains the fused search result. Specifically, the large model includes a generation model, and the large model inputs the dialogue scene data and the search result into the trained generation model, and the generation model fuses the search result to obtain the fused search result. The terminal device displays the fused search result generated by the large model to the user to provide a high-quality reply. It can be understood that the fusion method is not required here.

[0120] In addition, in other embodiments, the large model fuses the generation content of the dialogue scene data and the search result to obtain the reply, and the generated reply not only contains the fused search result, but also contains the reply reason corresponding to the fused search result.

[0121] It can be understood that in some implementations, the large model has the ability to generate a reply, which can be deployed on a terminal device or a cloud device. When the intent recognition capability of the large model is deployed on the terminal device, the terminal device can directly display the reply generated by the large model. When the large model has the ability to generate a reply, which can be deployed on a cloud device. At this time, the cloud device can generate a reply, that is, the above step S306 can be executed in the cloud. Then, the cloud device sends the reply to the terminal device, and the terminal device displays the received reply.

[0122] For example, referring to FIG. 2B, the content corresponding to the search result in the box K32A is “Helping you search for ticket information for the movie 1 that can be watched tonight: XXX Cinema (XXX store): ￥49.9, tonight: 19:30 | 20:00 | 21:15”. And the search reply displayed by the terminal device in the box K32B is “Do we go to this place to watch a movie?”. It can be understood that the content related to watching a movie in the current dialogue is strongly related to the application program related to watching a movie. At this time, the large model can retrieve effective information from the corresponding application program, and the terminal device can present the reply generated by the large model in a better way, avoiding the user from not getting accurate and detailed answers or solutions.

[0123] It can be understood that in some implementations, for example, the generated model used for fusion is the same as the generated model used in step S303 above when retrieval is not required. Only when training the generated model, the training sample used for training includes dialogue scene sample data related to retrieval intent, and the obtained retrieval result is input into the training sample of the generated model together with the dialogue scene sample, and then training is performed. The training method can be the same as or different from the training method of the generated model in step S303 above, which is not limited here.

[0124] It can be understood that the execution order of steps S301 to S306 above is only an example. In other embodiments, other execution orders can also be used, and some steps can be split or combined, which is not limited here.

[0125] It can be understood that the terminal device displays the generated reply on the screen, such as displaying the generated reply in the form of a card or a list. The generated reply can include at least one of text, image, voice, video, and link. The user can edit or send the displayed reply. For example, the user can directly modify the displayed reply, click the sending control on the screen for sending the displayed reply after modification, and send the modified reply. For another example, the user can send the displayed reply directly through the sending control for sending the displayed reply without modification. In addition, the user can also refer to the displayed reply to modify the input content in the input box, send the modified content in the input box through the sending control for sending the content in the input box, or not modify the input content in the input box and directly send the content in the input box through the sending control for sending the content in the input box. For details, please refer to the description of the user sending the reply or the content in the input box in FIGS. 2A and 2B above, which is not repeated here.

[0126] It can be understood that in other embodiments, for the above step S302, the terminal device obtains dialogue scene data, wherein the dialogue scene data includes current and historical dialogue content of the dialogue parties, user behavior information, and physiological indicator data of the current user, etc. At this time, the terminal device can directly generate a reply using a large model, and does not need to perform operations such as whether to need to search, and search in the resource database to obtain a search result. It can be understood that since the dialogue scene data includes multi-dimensional data, based on the multi-dimensional dialogue scene data, the generated reply can be more accurate. It can be understood that the generated reply can be at least one of a search result, a fused search result, a dialogue reply, and a corresponding reply reason.

[0127] It can be understood that the execution subject of the interaction method shown in the above FIG. 3 can also be an application program, for example, the above dialogue system, which can call other application programs to generate a reply, or does not need to call other application programs, and directly generates a reply in real time.

[0128] It can be understood that in some implementations, the function of generating a reply in real time can be configured in the intelligent dialogue service (Agent) of the terminal device. The dialogue system on the terminal device can call the intelligent dialogue service on the terminal device to provide high-quality and accurate replies to the user in the case that the user allows when specifically interacting with the user. Specifically, the intelligent dialogue service can operate in combination with multi-round dialogue content. The intelligent dialogue service can call a large model, and display the reply output by the large model. The large model can determine whether the dialogue content needs to be searched, so as to determine whether to need to search, and in the case that the search is needed, the large model fuses the search result obtained from the local and cloud services with the reply content generated by the large model based on the dialogue scene data to generate a fused search result. At this time, the terminal side can display the reply to the user in an appropriate manner, and at the same time, save locally, so as to provide accurate, reasonable, and high-quality reply content for the user.

[0129] In other implementations, the dialogue system can also be directly configured with a function of generating a reply, at this time, the intelligent dialogue service does not need to be called, that is, the dialogue system can directly call a large model to generate a reply.

[0130] The following describes an example of the dialogue system calling the intelligent dialogue service to generate a reply in real time.

[0131] FIG. 4A illustrates a system architecture 400, according to some embodiments of the present application. As shown, the system architecture 400 is composed of two terminal devices (terminal 100A, terminal 100B) and one cloud device (cloud 200). It can be understood that the terminal device includes, but is not limited to, a mobile device such as a mobile phone, a tablet, etc. that can be used for communication, and the cloud device is composed of no less than one server and has a large model computing capability.

[0132] Specifically, as shown in FIG. 4A, the terminal 100A includes a dialogue system 101A, an intelligent dialogue service 102A, and a local search 103A. The dialogue system 101A is configured to provide communication services for users. The dialogue system 101A can be an application program on the terminal 100A that can perform instant communication, such as a voice assistant, an intelligent assistant, and the like. It can be understood that in some embodiments, the dialogue system 101A, the intelligent dialogue service 102A, and the local search 103A can be different applications, can be combined into one application, or any two of them can be combined into one application, and the present application does not make any requirements in this regard. The intelligent dialogue service 102A is configured to call the large model computing capability of the cloud 200 or the local search 103A according to dialogue scenario data to generate a reply. It can be understood that the specific process of generating a reply by the intelligent dialogue service 102A according to dialogue scenario data can refer to the steps S301-S305 shown in FIG. 3, and will not be described herein. For example, the intelligent dialogue service 102A can be a system application on the terminal 100A, and when the dialogue system 101A is configured to allow the intelligent dialogue service 102A to be called, the dialogue system 101A can call the intelligent dialogue service 102A to generate a reply in real time when detecting the operation of the user in the input box.

[0133] The local search 103A is configured to search in a local resource database according to local search information to obtain a local search result. The local search 103A searches in the local resource database according to the search information.

[0134] Similarly, the terminal 100B includes a dialogue system 101B, an intelligent dialogue service 102B, and a local search 103B. The functions of the dialogue system 101B, the intelligent dialogue service 102B, and the local search 103B are substantially the same as those of the dialogue system 101A, the intelligent dialogue service 102A, and the local search 103A, and will not be described herein.

[0135] The cloud 200 includes a large model module 201 and a search engine 202.

[0136]

[0137] ​The large model module 201 is configured to generate a reply based on the dialogue scenario data. Specifically, the large model module 201 can include a discriminative model, an intent recognition model, and a generation model. The large model module 201 is configured to acquire dialogue scenario data from the terminal 100B or the terminal 100A, use the discriminative model to determine whether retrieval is needed based on the dialogue scenario data, and use the intent recognition model to identify a specific retrieval intent in the dialogue scenario data in a case where retrieval is needed, so as to cause the search engine 202 or the local search 103B to perform retrieval based on the retrieval intent to obtain retrieval information. It can be understood that when the retrieval information is generated, the search engine or the third-party service can be called through a cloud service. At this time, the cloud service can send the retrieval information to an application programming interface (API) of the search engine 202 through a network protocol (such as HTTP, WebSocket, etc.), and the search engine 202 can also implement calling of the third-party service or direct calling of an API of the third-party service to obtain relevant retrieval results or service responses.

[0138] In addition, the large model module 201 is also configured to generate a reply by using the generation model. Specifically, in a case where retrieval is not needed, a reply is generated based on dialogue scenario data from the terminal 100A or the terminal 100B; in a case where retrieval is needed, a reply is generated based on dialogue scenario data and retrieval results. The specific process in which the large model module 201 generates a reply based on dialogue scenario data can refer to the description of the large model generating a reply in FIG. 3, and details are not described herein.

[0139] The search engine 202 is configured to obtain retrieval results based on retrieval information sent by the large model module 201 and feed back to the large model module 201. That is, the search engine 202 performs search in a cloud resource database based on the retrieval information, so as to obtain retrieval results.

[0140] It can be understood that the system architecture 400 in FIG. 4A is only an example, and more or fewer modules can be included in some embodiments, and the modules can be combined or split, and details are not described herein.

[0141] The following describes a specific process in which the terminal 100B interacts with the cloud 200 to generate a reply in a dialogue process.

[0142] FIG. 4B shows an interaction process between the terminal 100B and the cloud 200 according to some embodiments of the present application.

[0143] Specific steps are as follows:

[0144] S401, the terminal 100B detects the first operation, obtains the dialogue scene data, including the dialogue content.

[0145] Optionally, the dialogue scene data can also include user behavior or physiological indicators and other information.

[0146] In some embodiments, the terminal 100B obtains the dialogue scene data including the dialogue content, user behavior and physiological indicators and other information when the user allows it to run.

[0147] In some implementations, the terminal 100B can obtain the dialogue scene data including the dialogue content, user behavior and physiological indicators and other information in the following ways.

[0148] (1) Obtain dialogue content by calling API, wherein the dialogue content includes sent content and pre-sent content. Specifically, the terminal 100B can obtain the text, audio, picture and video information currently in the dialogue by calling the API of the chat application (such as dialogue system B1), which can include the chat content of the user and the other party, the timestamp of the dialogue text, the recording of the audio, the picture content and the video content, etc. In this way, the terminal 100B can obtain the basic content of the dialogue for subsequent analysis and processing.

[0149] (2) Obtain user behavior information by screen recording. Specifically, the terminal 100B needs to obtain the user's behavior information, for example, when the user communicates with the other party in the current dialogue system, the user's behavior information such as browsing and editing chat records can be obtained based on the current dialogue system. The terminal 100B can obtain the screen recording permission, record the user's browsing records and editing operations in the chat application, track the user's operation behavior, and thus obtain the user's behavior characteristics in the dialogue process, so as to better understand the current dialogue scene.

[0150] (3) Obtain physiological indicator information. Specifically, the terminal 100B can be connected with some other devices that collect user state, such as watches, bracelets, etc. At this time, the terminal 100B can interact with the application of the aforementioned collection device and obtain the corresponding physiological indicator data, such as the user's expression, heartbeat, etc. In addition, the terminal device 100B can also obtain some sensor data according to the built-in sensor, such as camera data, to obtain the user's current expression. These data can provide more information about the user's emotions and reactions, which helps to more accurately analyze and process the dialogue scene.

[0151] It can be understood that in order to ensure user privacy and data security, the terminal 100B will only collect and use necessary information and comply with relevant privacy protection regulations.

[0152] S402, the terminal 100B sends the integrated dialogue scene data to the cloud 200.

[0153] It can be understood that step S402 is an optional step. In addition, S402 can also be executed by the cloud 200.

[0154] In some embodiments, the terminal 100B can organize and integrate the collected dialogue scene data according to the time sequence relationship, and send it to the cloud 200 for processing.

[0155] It can be understood that the dialogue scene data includes various aspects of the dialogue. At this time, the terminal 100B can obtain the context of the dialogue completely, so that the large model in the cloud 200 can generate more accurate and personalized replies and suggestions, thereby enhancing the intelligence of the dialogue system and improving the user experience.

[0156] In some implementations, before the terminal 100B sends the dialogue scene data to the cloud 200, the collected dialogue scene data needs to be converted in format and organized into a transmittable data structure. The design of the specific data format will be determined according to the actual needs and system design, and common data exchange formats such as JSON (javascript object notation) or Protocol Buffers can be used. It can be understood that the actual data format can be adjusted according to specific needs and design, which is not limited here.

[0157] It can be understood that, as mentioned above, the dialogue scene includes but is not limited to (1) dialogue content, involving text, pictures, audio and video information, (2) user behavior information, such as browsing chat records, editing information, etc. (3) physiological index data, such as user expression, heartbeat and other physiological data, and these data will adopt different data formats and units according to specific devices and applications.

[0158] Exemplarily, the following introduces the data sent in combination with the dialogue scene corresponding to the interface 002 shown in FIG. 2A. The data sent includes three messages between user B-2 and user A-2 and a pre-reply of user B-2 as shown in FIG. 2A.

[0159] Specifically, the following shows the pseudo code:

[0160] #dialogue content

[0161] dialogue: ["user A-2", "user B-2"];

[0162] Conversation message, sent content: { "timestamp": 2023-10-31T17:30:00, "sender": "User A-2", "content": "What do you have planned for today after work?"} { "timestamp": 2023-10-31T17:30:10, "sender": "User B-2", "content": "I have nothing planned for the evening."} { "timestamp": 2023-10-31T17:30:20, "sender": "User A-2", "content": "Then?"};

[0163] Pre-sent content: { "timestamp": 2023-10-31T17:30:20, "sender": "User B-2", "content": "I want to go to the internet cafe by myself."}

[0164] # User behavior { "timestamp": 2023-10-31T17:31:30, "behavior type": "browsing history", "browsed data": "..."} { "timestamp": 2023-10-31T17:31:50, "behavior type": "editing sent content", "edited data": "..."}

[0165] # Psychological indicators

[0166] "Emotion": { "timestamp": 2023-10-31T17:32:30, "expression": "neutral"},

[0167] "heart rate": { "timestamp": 2023-10-31T17:32:30, "value": 80, "unit": bpm},

[0168] It can be understood that in the above example, the sent data includes the timestamp of the conversation, the sender and the content are recorded in the message list. In addition, the user's browsing history behavior and the user's psychological index data are also included. The user's browsing history data is not specifically shown in the current example. The user's psychological indicators include an expression of "neutral" and a heart rate of 80bpm (beats per minute). These information can be used for further processing and analysis to provide more personalized and accurate responses and suggestions.

[0169] S403, the cloud 200 determines whether retrieval is needed according to the conversation scene data.

[0170] If not, go to S404; if yes, go to S407. It can be understood that steps S404-S406 are optional steps.

[0171] In some embodiments, the cloud 200 can determine whether retrieval is needed through the large model. If the output of the large model is an identifier indicating that retrieval is not needed, S404 is entered, otherwise, S407 is entered.

[0172] For example, the dialogue scenario data corresponding to the dialogue scenario shown in FIG. 2A, the cloud 200 can use natural language processing (NLP) technology to analyze the dialogue, and then find that user A-2 is asking user B-2 about the arrangement, and user B-2 answers that there is no arrangement, and based on the content of this dialogue segment, it is analyzed that there is no retrieval demand, so retrieval is not needed. And because user KB does not explicitly reply to the content related to retrieval in the pre-sent message, it is unable to find the corresponding retrieval result in the corresponding resource database, for example, it is unable to find the third-party application service with a target, so retrieval is not needed, and S404 is entered.

[0173] For another example, referring to the scene shown in FIG. 2B, and the chat interface of the travel scene shown in FIG. 5A. Referring to the dialogue content shown in FIG. 2B, it can be found that there is a retrieval demand for watching movies between the two parties in the dialogue, at this time, the cloud 200 will determine that retrieval is needed according to the dialogue scenario data. Referring to the dialogue scenario involving the travel topic shown in FIG. 5A, in this scenario, user A-2 sends a message “I want to go on a trip”, user B-2 replies “Do you want to go to Dali?”, user A-2 continues “Or go to Hainan?”, user B-2 sends a message “Yes, when do we go?”, and user A-2 replies “Or book a flight to Dali on next Wednesday?”. According to the foregoing dialogue content, it can be found that the two parties in the dialogue need to go to Dali for a trip, and user A-2 is asking about the specific task of taking a flight to Dali on next Wednesday. At this time, the cloud 200 can determine that retrieval is needed according to the dialogue scenario data, and user B-2 needs to send information about taking a flight to Dali, and S407 is entered.

[0174] S404, the cloud 200 generates a reply according to the dialogue scenario data, including a dialogue reply and a reply reason.

[0175] In some embodiments, the cloud 200 can generate an accurate and coherent reply through a large model in combination with the dialogue context corresponding to the dialogue scenario data. The large model can understand the user's question in the dialogue scenario data through dialogue understanding technology and semantic understanding algorithm, recommend a reply that meets the user's intention and takes into account the other party's emotions, tone, and other factors, to provide a satisfactory answer.

[0176] For example, in the scenario shown in Figure 2A, Cloud 200 analyzes user A-2's question "So?" and user B-2's previous message "I have nothing to do tonight" to determine that user A-2's inquiry about user B-2's evening plans indicates an expectation of doing something together. Therefore, user B-2's pre-sent message contains a rejection clause and may cause the other party to feel uncomfortable. The generative model can generate the dialogue response: "Then let's go out and have some fun!" along with the reasoning: "The other party's inquiry about your evening plans indicates an expectation of doing something together; your current response may cause the other party to feel uncomfortable."

[0177] S405, Cloud 200 sends a reply to Terminal 100B.

[0178] In some embodiments, the cloud-based 200 can generate a reply based on relevant algorithms or rules, taking into account emotional judgment and factors to avoid offending the other party. After generating the reply, it can be processed according to a certain format to obtain formatted data for transmission to the terminal 100B.

[0179] In some implementations, the generated response and other relevant information can be combined into a JSON object. For example, a JSON object could contain the content of the response and a timestamp.

[0180] The following is pseudocode for sending data:

[0181] #reply

[0182] "Dialogue Reply": "Then let's go play together!"

[0183] Timestamp: 2023-10-31T08:51:26

[0184] "Reason for reply": "The other party asked about your plans for the evening because they are looking forward to doing something with you. Your current response may cause them to feel bad."

[0185] S406, Terminal 100B displays a reply.

[0186] In some embodiments, the terminal 100B displays the received response, which includes the dialogue reply and the reason for the reply.

[0187] In some implementations, terminal 100B parses the received JSON object and extracts the necessary fields, such as the dialogue response, timestamp, and reason for the response. This is then displayed on the interface to provide the user with appropriate feedback, thereby continuing the dialogue.

[0188] It can be understood that through the process, the cloud 200 can efficiently deliver the reply and related information to the terminal 100B, so that the application program or platform on the terminal 100B can flexibly process and present the response result.

[0189] It can be understood that in the process of conversation between people, replies are provided to both parties in real time according to the actual sending content and the pre-sending content. At the same time, with the consent of the user, the user's behavior, as well as psychological indicators such as observed expressions, tone of voice, body temperature, etc. are used to provide more accurate and smooth replies, so that the conversation can become more smooth and promote the conversation to develop in a positive direction.

[0190] S407, the cloud 200 identifies the retrieval intention.

[0191] In some embodiments, after the cloud 200 determines that the retrieval requirement is met, for example, the retrieval requirement is to book a ticket or watch a movie, the cloud 200 identifies the retrieval intention according to the dialogue scene data through the large model, so as to obtain the retrieval intention that can be used for retrieval.

[0192] For example, referring to the dialogue scene in FIG. 2B and FIG. 5A, the cloud 200 generates retrieval intentions related to watching a movie and booking a ticket according to the dialogue scene data, respectively.

[0193] S408, the cloud 200 generates retrieval information according to the retrieval intention.

[0194] In some embodiments, the cloud 200 can generate retrieval information, such as query parameters, which is convenient for search engines or native searches according to the retrieval intention. It can be understood that since the data resource library available for retrieval includes the data resource library of the terminal and the data resource library of the cloud, the generated retrieval information includes native retrieval information or cloud retrieval information, for example, the native retrieval information is native query parameters, and the cloud retrieval information is cloud query parameters.

[0195] It can be understood that the format of the retrieval information can be JSON format, or other formats, which are not required here.

[0196] It can be understood that the native retrieval information can include keywords related to the retrieval intention, search sentences related to the retrieval intention, and specific parameters can include: query intention, user input (the user input is a search sentence generated by the cloud 200 and can be directly used for search in native search), query object, and context information required when querying, such as user identity, platform and application program of terminal device, etc.

[0197] Exemplarily, a pseudo code of the native retrieval information corresponding to the terminal is shown as follows:

[0198] # "intent": "book a ticket";

[0199] "User input": "Want to book a flight from Beijing to Sanya"

[0200] "Query object": "Flight"

[0201] "Query context information": {"User identity": {"ID": "user123", "Name": "Zhang San"}, "Device information": {"platform": "Andriod", "app": "Flight query"}}

[0202] It can be understood that the above-mentioned user input is a search sentence used for searching in the terminal 100B.

[0203] Exemplarily, a pseudo code corresponding to the cloud retrieval information of the cloud is shown as follows:

[0204] "Intention": "Book a flight";

[0205] "User input": "Want to book a flight from Beijing to Sanya"

[0206] "Third-party application":

[0207] "Query context information": {"User identity": {"ID": "user123", "Name": "Zhang San"}, "Device information": {"platform": "Andriod", "app": "Flight query"}}

[0208] It can be understood that the above-mentioned user input is a search sentence used for searching in the cloud 200, for example, the search engine 202. Since there is a third application service in the cloud, for example, a third-party application program for tourism or booking a flight, the retrieval can be performed in the cloud service deployed in the application program related to tourism or booking a flight.

[0209] It can be understood that since the corresponding search tools and resources used for searching are different when searching on the terminal and searching in the cloud, the retrieval information generated is also different.

[0210] It can be understood that for the cloud retrieval part S409 and the local retrieval parts S410-S412, any one of the two parts can be executed, and the execution order is not limited. That is, step S409 and steps S410-S412 can be optional steps, respectively.

[0211] S409, the cloud 200 obtains a cloud retrieval result according to the cloud retrieval information.

[0212] ​In some embodiments, the cloud 200 retrieves the cloud search information, and searches in the cloud resource database to obtain the search result. Specifically, the cloud 200 analyzes and processes the cloud search information by using a natural language processing (NLP) technology to generate a specific query statement for searching, and obtains the cloud search result, which includes a relevant search result list, a total result quantity, a search engine response time, and the like.

[0213] S410, the cloud 200 sends the local search information to the terminal 100B.

[0214] In some embodiments, the cloud 200 can send the local search information to the terminal 100B through a network protocol (such as HTTP, WebSocket, and the like).

[0215] S411, the terminal 100B obtains the local search result according to the local search information.

[0216] In some embodiments, the terminal 100B searches in the local resource database according to the local search information to obtain the local search result. The local resource database can include local application programs and local data. The local application programs are various software programs installed on the device, and are usually stored in the local storage space of the device. The local data is data related to the device itself, such as device settings, user preferences, contacts, short messages, call records, photos, videos, and the like, and is usually stored in the local storage space of the device.

[0217] In some implementations, the terminal 100B searches and matches in the local resource database according to the local search information by using a database query language (such as SQL, NoSQL query, and the like) or a local resource search algorithm, and then obtains the local search result.

[0218] For example, in combination with FIG. 5B, for the dialogue scenario of taking a plane shown in FIG. 5A, the terminal 100B searches in the local application programs and the local data according to the local search information to obtain information related to a ticket from Beijing to Sanya.

[0219] S412, the terminal 100B sends the local search result to the cloud 200.

[0220] S413, the cloud 200 generates a reply according to the local and / or cloud search result and the dialogue scenario data, and the reply includes the fusion search result.

[0221] In some embodiments, the cloud 200 generates a reply according to the local and / or cloud search result and the dialogue scenario data by using a large model, and the generated reply includes the fusion search result corresponding to the search requirement.

[0222] For example, for the travel scenario shown in Figure 5A, the fusion search results could include: "The search system is finding travel and flight information for September 13th, including a flight link for 14:30-18:25, ¥1310, and a flight link for 06:30-10:20, ¥1680." and "Great! I'm really looking forward to this trip!"

[0223] In some implementations, the cloud 200 parses the return body corresponding to the cloud search results obtained in the cloud, such as the return body from the cloud search engine, and the information sent from the terminal 100B. Based on the parsed content, it constructs the input parameters of the large model and generates a response based on the input parameters.

[0224] Understandably, here's the specific process of generating a 200-number response in the cloud:

[0225] (1) The returned data corresponding to the cloud search results includes: the list of search results, the total number of results, and the search engine's response time.

[0226] Here's an example of pseudocode for the content returned by a search engine:

[0227] {"Flight Number": "CAxxx"; "Departure City": "Beijing"; "Departure Time": "2023-xx-xx 10:00"; "Arrival City": "Sanya"; "Arrival Time": "2023-xx-xx 13:00"; "Airline": "xxx"; "Price": "1000 yuan"}, {"Flight Number": "HUxxx"; "Departure City": "Beijing"; "Departure Time": "2023-xx-xx 14:00"; "Arrival City": "Sanya"; "Arrival Time": "2023-xx-xx 17:00"; "Airline": "xxx"; "Price": "1200 yuan"}; "Total Quantity": 2; "Response Time": "200".

[0228] (2) Parsing the return body corresponding to the cloud search results includes: parsing the return body corresponding to the cloud search results, extracting the search result list and other relevant information, such as the total number of results.

[0229] (3) The input parameters for constructing the large model based on the parsed content include: the input parameters for constructing the large model based on the return body corresponding to the cloud search results. The input parameters include the list of search results, the total number of results, and other necessary fields. The specific content and form of the input parameters can be adjusted according to the specific large model and application scenario.

[0230] Pseudo code of input parameters of an example large model:

[0231] Input parameters {

[0232] Retrieval results: {“Flight number”:“CAxxx”;“Departure city”:“Beijing”;“Departure time”:“2023-xx-xx10:00”;“Arrival city”:“Sanya”;“Arrival time”:“2023-xx-xx13:00”;“Airline”:“xxx”;“Price”:“1000 yuan”}, {“Flight number”:“HUxxx”;“Departure city”:“Beijing”;“Departure time”:“2023-xx-xx14:00”;“Arrival city”:“Sanya”;“Arrival time”:“2023-xx-xx17:00”;“Airline”:“xxx”;“Price”:“1200 yuan”}};“Total number”:2;“Response time”:“200”

[0233] Dialogue scenario data: {......}.

[0234] (4) Generating a reply according to the input parameters includes that the large model generates a recommended reply according to the constructed input parameters. It can be understood that the large model can generate relevant recommended reply content according to the retrieval results, the total number of results and other context information, combined with the model weight and the knowledge obtained through training. It can be understood that the recommended reply content can be a specific answer, suggestion, relevant resource link or other information for the user query, which is customized according to the specific application scenario.

[0235] Pseudo code of output results of an example large model:

[0236] Output results {

[0237] “Suggestion type”:“Flight reservation”,“Suggestion content”:“According to your query, I found the following flight information for you:”, “Specific suggestions”: {“Flight number”:“CAxxx”;“Departure city”:“Beijing”;“Departure time”:“2023-xx-xx10:00”;“Arrival city”:“Sanya”;“Arrival time”:“2023-xx-xx13:00”;“Airline”:“xxx”;“Price”:“1000 yuan”;“Booking interface link”:“xxxxx”}, {“Flight number”:“HUxxx”;“Departure city”:“Beijing”;“Departure time”:“2023-xx-xx14:00”;“Arrival city”:“Sanya”;“Arrival time”:“2023-xx-xx17:00”;“Airline”:“xxx”;“Price”:“1200 yuan”;“Booking interface link”:“xxxxx”}

[0238] “Suggestion type”:“Flight reservation”,“Suggestion content”:“According to your query, I found the following flight information for you:”, “Specific suggestions”: {“Flight number”:“CAxxx”;“Departure city”:“Beijing”;“Departure time”:“2023-xx-xx10:00”;“Arrival city”:“Sanya”;“Arrival time”:“2023-xx-xx13:00”;“Airline”:“xxx”;“Price”:“1000 yuan”;“Booking interface link”:“xxxxx”}, {“Flight number”:“HUxxx”;“Departure city”:“Beijing”;“Departure time”:“2023-xx-xx14:00”;“Arrival city”:“Sanya”;“Arrival time”:“2023-xx-xx17:00”;“Airline”:“xxx”;“Price”:“1200 yuan”;“Booking interface link”:“xxxxx”}

[0239] Parameter introduction: "flight number": indicates the unique identifier of the flight; "departure city": indicates the name or code of the departure city of the flight; "departure time": indicates the departure time of the flight; "arrival city": indicates the name or code of the arrival city of the flight; "airline": indicates the name or code of the airline carrying the flight; "price": indicates the price of the ticket. "Booking interface link": a link to the flight ticket booking of an application, through which the corresponding page of the application can be directly opened to perform the booking of the flight ticket, and the booking interface link is a link provided to the terminal side to display the relevant booking interface.

[0240] It can be understood that the generation model in the large model can use a template filling method to fill in some information when generating. For example, a pre-defined template structure is defined, which is filled into the corresponding position to generate a human-readable reply and suggestion.

[0241] It can be understood that the generation model in the large model can fuse the relevant search results (i.e., search results) obtained from the native (i.e., the terminal side) and the cloud and the original dialogue scene data, i.e., fuse search results from multiple different sources, so that the generated reply can not only include the corresponding accurate demand and be displayed to the user, but also provide high-quality reply content to the user, such as displaying App-related recommendation suggestions.

[0242] S414, the cloud 200 sends the reply to the terminal 100B.

[0243] S415, the terminal 100B displays the reply.

[0244] It can be understood that the terminal 100B receives and displays the reply. It can be understood that the terminal 100B can use the terminal side or cloud side resources and capabilities to provide the user with comprehensive and accurate replies.

[0245] It can be understood that the execution order of the above steps S401 to S415 is only an example, and in other embodiments, other execution orders can also be used, and part of the steps can also be split or combined, which is not limited herein.

[0246] It can be understood that the terminal 100A and the terminal 100B are both real-time display intelligent reply devices. For the travel conversation scenario shown in FIG. 5A, after the user conversation is displayed on each terminal device, the terminal device can display the intelligent reply. For example, the following describes the conversation of the terminal 100B in detail again. When the user B asks, as shown by user B-2 in the figure, “Do you want to go to Dali?” at this time, the terminal device 100B can call the intelligent conversation service 102B, at this time the intelligent conversation service 102B can understand the user KB's preference for the travel destination, and then provide travel information about Dali, such as scenic spot introduction and food recommendation, for the user KB to provide reference. When the user A sends, as shown by user A-2 in the figure, “How about going to Hainan?” at this time, it can be determined that the destination is Hainan, and the intelligent conversation service 102B can continue to converse with the user KA to understand more information about the travel plan, activity arrangement and order, etc., to provide more personalized suggestions and services. When the user KB indicates agreement and asks, as shown by user B-2 in the figure, “When do we go?” the intelligent conversation service 102B can use the context of the conversation to infer the user's interest in the departure time. According to this information, the intelligent conversation 102B can call the cloud service, such as a flight query interface, to provide the user with relevant information such as departure date, flight time and price. It can be understood that FIG. 5A shows the data flow of the intelligent conversation service 102B retrieving data from the cloud when it needs to be retrieved, which will not be described here.

[0247] Similarly, when the user KA indicates that he wants to go on a trip, the intelligent conversation service 102A will recognize the user's intention and further call the conversation understanding technology and intention recognition algorithm to retrieve relevant travel information from the local and cloud. By integrating travel planning tools, flight queries and hotel reservations, etc., the intelligent conversation service 102A can provide travel suggestions for the user KA, such as recommending suitable travel routes to Hainan, scenic spot introduction and destination recommendation, etc.

[0248] It can be understood that the interactive method proposed in the embodiments of the present application can monitor the conversation in real time, thereby avoiding the traditional reply according to the user's input speech. Moreover, the real-time monitoring of the various inputs of the conversation object, including the existing conversation content, the pre-sent content, and the physiological indicators, behaviors, etc. of the conversation person, through the comprehensive analysis and processing of the various inputs, combined with the previous conversation history information, the needs and intentions of the conversation participants can be more comprehensively understood, and the large model is combined for comprehensive reasoning. This enables the intelligent conversation service to provide ideal replies for the participants of the conversation and create a more ideal and harmonious conversation experience.

[0249] FIG. 6 shows a hardware module diagram of a terminal 100 and a cloud 200 according to some embodiments of the present application.

[0250] As shown in FIG. 6, the terminal 100 comprises a first display module 501, a first input module 502, a first storage module 503, a first calculation module 504, and a first communication module 505.

[0251] The cloud 200 comprises a second storage module 601, a second calculation module 602, and a second communication module 603.

[0252] Specifically, for the terminal 100:

[0253] The first display module 501 is configured to output and display interfaces. In some embodiments of the present application, the first display module 501 can display the reply generated by the intelligent dialogue service in a visual form to the user, which can be in the form of text, image, video, audio, and link data. It can be understood that through the first display module 501, the user can intuitively view and understand the reply provided by the intelligent dialogue service.

[0254] The first input module 502 is configured to interact with the dialogue system. The first input module 502 can be a keyboard, a touch screen, or a voice recognition device. Through the first input module 502, the user can ask questions, express needs, or provide relevant information to the intelligent dialogue service. The user can input text through the keyboard, click or gesture operation through the touch screen, or use voice input to dialogue with the dialogue system.

[0255] The first storage module 503 is configured to store various data and information required by the dialogue system, including relevant parameters of the intelligent dialogue service, local content, information of third-party applications in the local, local search results, etc. These data and information can be repeatedly called and referred to by the dialogue system or the intelligent dialogue service for analysis, reply generation, search, etc., so as to provide accurate and useful reply content.

[0256] The first calculation module 504 is configured to process, analyze, and calculate input data and information. It can be a central processing unit, including a processor and a memory, etc. It is also responsible for processing local search and other tasks.

[0257] The first communication module 505 is configured to realize connection and communication with other devices or networks, and can support wireless communication or wired communication. Through the first communication module 505, the terminal 100 can be connected with the Internet to call search capabilities, obtain the latest information and data, etc. The first communication module 505 can also transmit and interact data with other devices or systems to realize more extensive functions and services.

[0258] For the cloud 200:

[0259] The second storage module 601 is used to store and manage large amounts of data, models and other related information. It may include historical search data, user configuration files, system configuration files, model parameters, training datasets, etc. It can also be used for long-term data storage and backup, as well as support the training, updating and deployment of large models.

[0260] The second computing module 602 is the core computing resource of the cloud-based system, typically consisting of a cluster of high-performance computers or servers equipped with powerful processors and graphics processing units (GPUs). These computing resources are used for large-scale, complex computational tasks, such as training machine learning models, executing and inferring deep learning algorithms, and processing large-scale data. The second computing module 602 provides higher computing power and parallelism, enabling the system to perform advanced computational tasks such as intent recognition and recommendation responses more efficiently. Furthermore, the second computing module 602 can leverage its powerful computing capabilities and large-scale trained models to perform deeper analysis and processing of user questions, generating more advanced, accurate, and personalized recommendation responses.

[0261] The second communication module 603 is responsible for remote communication and data transmission with the terminal device. The second communication module 603 connects to the terminal device via the Internet to receive queries and requests from the terminal device in real time. Once a query or request is received, the communication device transmits it to a computing device in the cloud for processing and returns the processed result to the terminal device. Simultaneously, the communication device is also responsible for maintaining the connection with the terminal device, ensuring the stability and security of communication.

[0262] The following description, in conjunction with Figures 6 and 7, briefly illustrates a data flow in some embodiments where the terminal 100 interacts with the cloud 200 to generate a response. Figure 6 shows the data flow when no retrieval is required based on the dialogue scenario data; Figure 7 shows the data flow when a retrieval is required based on the dialogue scenario data.

[0263] Specifically, as shown in FIG. 6, the first input module 502 receives the input of the user to obtain the input information, for example, the user uses the keyboard, touch screen or voice recognition device to interact with the terminal device 100B, at this time, the first input module 502 receives the text, voice or video input information. The first storage module 503 stores the input information and stores the data collected by other collection modules, such as user behavior information and physiological index information, etc., to obtain the current dialogue scene data, and sends the current dialogue scene data to the first calculation module 504 for processing. The first calculation module 504 can preliminarily process the dialogue scene data to obtain the processed dialogue scene data, and then send the processed dialogue scene data to the first communication module 505. The first communication module 505 sends a request to the second communication module 603 in the cloud, and the request includes the dialogue scene data. The second communication module 603 sends the dialogue scene data to the second calculation module 602, and the second calculation module 602 directly generates a reply after judging that it is not necessary to search, and then sends the generated reply to the second communication module 603, which is sent to the first communication module 505. The first communication module 505 sends the reply to the first display module 501 under the control of the first calculation module 504, and the first display module 501 displays the reply.

[0264] As shown in FIG. 7, when the second calculation module 602 judges that it is necessary to search, it will generate a search intention, at this time, it can generate local search information and cloud search information, so as to search in the terminal and the cloud respectively. The second calculation module 602 obtains the second search result from the second storage module 601 according to the cloud search information. The second calculation module 602 sends the local search information to the first calculation module 504 through the second communication module 603 and the first communication module 505, and the first calculation module 504 searches from the first storage module according to the local search information to obtain the local search result. It can be understood that the result of the local search is mainly based on the existing data and information, which can provide some quick and simple answers.

[0265] Then it is sent to the second calculation module 602 through the first communication module 505 and the second communication module 603, and the second calculation module 602 fuses the local search result and the cloud search result to generate a reply.

[0266] It can be understood that the interaction method provided by the embodiments of the present application can select corresponding algorithms and technical tools according to specific needs and technical platforms in actual implementation. At the same time, privacy and data security should also be properly handled to protect the personal information of users, and more complex technical design and improvement may be required in actual application. For example, the first computing module 504 on the terminal side can also be used to fuse the retrieval result with the model reply obtained by the large model according to the dialogue scene data, so as to obtain the reply, and the specific implementation manner is not required here.

[0267] FIG. 8 shows a structural diagram of an apparatus 800 according to some embodiments of the present application. It can be understood that the apparatus 800 can be a terminal device or a cloud device. The apparatus 800 can include one or more processors 801, which can also be referred to as processing units, and can implement certain control functions. The processor 801 can be a general-purpose processor or a special-purpose processor, etc. For example, it can be a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control communication apparatuses such as base stations, baseband chips, terminals, terminal chips, DUs or CUs, execute software programs, and process data of software programs.

[0268] In an alternative design, the processor 801 can also store instructions and / or data 803, which can be run by the processor to enable the apparatus 800 to perform the interaction method described in the above method embodiments.

[0269] In another alternative design, the processor 801 can include a transceiver unit for implementing receiving and transmitting functions. For example, the transceiver unit can be a transceiver circuit, or an interface, or an interface circuit. The transceiver circuit, interface or interface circuit for implementing receiving and transmitting functions can be separate or integrated together. The above transceiver circuit, interface or interface circuit can be used for reading and writing of codes / data, or the above transceiver circuit, interface or interface circuit can be used for transmission or transfer of signals.

[0270] In yet another possible design, the apparatus 800 can include a circuit that can implement the functions of the interaction method in the foregoing method embodiments.

[0271] Optionally, the apparatus 800 can include one or more memories 802, which can store instructions / data 804. The instructions can be run on the processor to enable the apparatus 800 to perform the methods described in the above method embodiments. Optionally, the memory can also store data. Optionally, the processor can also store instructions and / or data. The processor and the memory can be separately arranged or integrated together. For example, the corresponding relationship described in the above method embodiments can be stored in the memory or in the processor.

[0272] Optionally, the apparatus 800 further includes a transceiver 805 and / or an antenna 806. The processor 801 can be referred to as a processing unit, which controls the apparatus 800. The transceiver 805 can be referred to as a transceiving unit, a transceiver, a transceiving circuit, a transceiving apparatus, an interface, an interface circuit, or a transceiving module, etc., which is used to realize the transceiving function.

[0273] Optionally, the apparatus 800 in the embodiments of the present application can be used to execute the interaction method described in FIG. 3 and FIG. 4B.

[0274] According to the method provided in the embodiments of the present application, the present application further provides a computer program product, which includes computer program codes, and when the computer program codes run on a computer, the computer is caused to realize the steps executed by the apparatus 800 in any one of the above embodiments.

[0275] According to the method provided in the embodiments of the present application, the present application further provides a computer readable medium, which stores program codes, and when the program codes run on a computer, the computer is caused to realize the steps executed by the apparatus 800 in any one of the above embodiments.

[0276] The embodiments disclosed in the present application can be implemented in hardware, software, firmware or a combination thereof. The embodiments of the present application can be implemented as computer programs or program codes running on programmable systems including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0277] The program codes can be applied to input instructions to execute the functions described in the present application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purpose of the present application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC) or a microprocessor.

[0278] The program codes can be implemented in a high-level programming language or an object-oriented programming language to communicate with the processing system. When necessary, the program codes can also be implemented in assembly language or machine language. In fact, the mechanisms described in the present application are not limited to the scope of any specific programming language. In any case, the language can be a compiled language or an interpreted language.

[0279] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can be implemented as instructions carried by or stored on one or more transitory or non-transitory machine- readable (e.g., computer-readable) media, which can be read and executed by one or more processors. For example, the instructions can be downloaded over the network or by another computer readable medium from a remote site. Thus, a machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including without limitation floppy disks, optical disks, optical fiber, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, flash memory, or tangible or other machine-readable media. Accordingly, a machine-readable medium includes any medium that is capable of storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0280] In the drawings, some structural or methodological features are shown in particular arrangements and / or orders. However, it should be understood that such particular arrangements and / or orders can not be required. Instead, in some embodiments, the features can be arranged differently than shown in the illustrative figures. Also, the inclusion of a structural or methodological feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features can not be included or can be combined with other features.

[0281] It should be noted that each unit / module referred in each apparatus embodiment of the present application is a logical unit / module, in physical, one logical unit / module can be one physical unit / module, also can be a part of one physical unit / module, also can be a combination of multiple physical unit / modules, the physical implementation of these logical units / modules is not the most important, the combination of the functions implemented by these logical units / modules is the key to solve the technical problems proposed in the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned each apparatus embodiment of the present application does not introduce the units / modules which are not closely related to solving the technical problems proposed in the present application, which does not mean that the above-mentioned apparatus embodiments do not have other units / modules.

[0282] It is to be understood that the phrases such as "first" and "second", and the like, used herein are used only to distinguish one from another, without necessarily implying an actual relationship or order between them. Also, the use of the term "including", "containing", or any other variant thereof, is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is not required to comprise only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. While the application has been illustrated and described in connection with certain preferred embodiments, it will be readily apparent to those of ordinary skill in the art that numerous modifications, substitutions, and alterations can be made without departing from the scope of the application.

Claims

1. An interaction method applied to an electronic device, the method comprising: displaying a first user conversation; in a case where the first user conversation is a user conversation requiring retrieval, displaying a first reply related to a retrieval result corresponding to the first user conversation, the retrieval result being obtained according to a retrieval intention of the first user conversation; in a case where the first user conversation is a user conversation not requiring retrieval, displaying a second reply to the first user conversation.

2. The method of claim 1, wherein, Further comprising: displaying the first reply or the second reply in the form of a card or a list, and the first reply or the second reply can include at least one data form of text, image, voice, video, and link.

3. The method according to claim 1 or 2, characterized in that, Further comprising: obtaining conversation scene data related to the first user conversation, the conversation scene data including at least one of conversation content of the first user conversation, behavior information of a user for the first user conversation, and a physiological indicator of the user for the first user conversation; calling a large language model, and the large language model determines whether the first user conversation is a user conversation requiring retrieval according to the conversation scene data; in a case where the first user conversation is a user conversation requiring retrieval, the large language model identifies a retrieval intention of the first user conversation according to the conversation scene data.

4. The method of claim 3, wherein, The first reply includes a fusion retrieval result corresponding to the retrieval result.

5. The method of claim 4, wherein, The fusion retrieval result is obtained by: the large language model generates the fusion retrieval result according to the input retrieval result and the conversation scene data of the first user conversation.

6. The method of claim 3, wherein, The retrieval result is obtained by: retrieving from a resource database according to retrieval information corresponding to the retrieval intention, wherein the retrieval information includes at least one of a keyword corresponding to the retrieval intention and a search sentence corresponding to the retrieval intention; the resource database includes a local resource database or a cloud resource database, and the local resource database includes local data or a local application program; the cloud resource database includes an application program or a service deployed in the cloud.

7. The method of claim 3, wherein, Further comprising: in a case where the first user conversation is a user conversation requiring retrieval, the large language model generates the second reply according to the conversation scene data.

8. The method of claim 1, wherein, The second reply includes a conversation reply and a reply reason corresponding to the conversation reply.

9. An interaction method applied to an electronic device, the method comprising: displaying a first user conversation; displaying a first reply of the first user conversation based on conversation scene data of the first user conversation, wherein the conversation scene data includes at least one of behavior information of a user for the first user conversation and a physiological indicator of the user for the first user conversation.

10. The method of claim 9, wherein, The first reply includes at least one of: a retrieval result corresponding to the first user conversation, a fusion retrieval result related to the retrieval result, a conversation reply, and a reply reason corresponding to the conversation reply.

11. The method of claim 10, wherein, Further comprising: calling a large language model, and the large language model determines, according to the dialogue scene data, whether the first user dialogue is a user dialogue that needs to be retrieved; in a case where the first user dialogue is a user dialogue that needs to be retrieved, displaying the retrieval result or the fusion retrieval result; in a case where the first user dialogue is a user dialogue that does not need to be retrieved, displaying the dialogue reply and a reply reason corresponding to the dialogue reply.

12. The method of claim 11, wherein, The fusion retrieval result is obtained by: The large language model identifies, according to the dialogue scene data, a retrieval intention of the first user dialogue; The large language model retrieves, according to retrieval information corresponding to the retrieval intention, from a resource database to obtain the retrieval result, wherein the retrieval information includes at least one of a keyword corresponding to the retrieval intention and a search sentence corresponding to the retrieval intention; The large language model generates the fusion retrieval result according to the input retrieval result and the dialogue scene data of the first user dialogue.

13. The method of claim 11, wherein, The dialogue reply and the reply reason corresponding to the dialogue reply are obtained by: in a case where the first user dialogue is a user dialogue that needs to be retrieved, the large language model generates the dialogue reply and the reply reason corresponding to the dialogue reply according to the dialogue scene data.

14. An interaction method applied to an electronic device, the method comprising: displaying a first user dialogue; displaying a first reply corresponding to the first user dialogue, the first reply including a dialogue reply and a reply reason corresponding to the dialogue reply.

15. The method of claim 14, wherein, Further comprising: obtaining dialogue scene data related to the first user dialogue, the dialogue scene data including at least one of dialogue content of the first user dialogue, behavior information of a user for the first user dialogue, and a physiological index of the user for the first user dialogue; calling a large language model, and the large language model generates the first reply according to the dialogue scene data.

16. The method of claim 15, wherein, Further comprising: The large language model determines, according to the dialogue scene data, whether the first user dialogue is a user dialogue that needs to be retrieved; in a case where the first user dialogue is a user dialogue that does not need to be retrieved, generating the first reply.

17. An interaction method applied to an electronic device, the method comprising: displaying a first user dialogue, the first user dialogue being a user dialogue that needs to be retrieved; displaying a first reply related to a retrieval result corresponding to the first user dialogue, the first reply including a fusion retrieval result corresponding to the retrieval result, wherein the retrieval result is obtained by retrieving according to a retrieval intention of the first user dialogue.

18. The method of claim 17, wherein, Further comprising: obtaining dialogue scene data related to the first user dialogue, the dialogue scene data including at least one of dialogue content of the first user dialogue, behavior information of a user for the first user dialogue, and a physiological index of the user for the first user dialogue; The large language model is invoked, and the large language model generates the first reply according to the dialogue scenario data and the search result.

19. The method of claim 18, wherein, Further comprising: The large language model determines whether the first user dialogue is a user dialogue that needs to be searched according to the dialogue scenario data. The large language model generates the first reply in the case that the first user dialogue is a user dialogue that needs to be searched.

20. An electronic device, comprising: The memory is configured to store instructions. The processor is configured to execute the instructions to implement the interaction method of any one of claims 1-19.

21. A readable storage medium, characterized by, The readable medium stores instructions, and the instructions, when executed on an electronic device, cause the electronic device to perform the interaction method of any one of claims 1-19.

22. A computer program product, characterised in that, The computer program product comprises computer program code, which, when executed on a computer, causes the computer to perform the interaction method of any one of claims 1-19.

Images