Interaction method, electronic device, readable storage medium, and program product

Abstract

The present application relates to the technical field of computers, and in particular to an interaction method, an electronic device, a readable storage medium, and a program product. The method comprises: first acquiring conversation scene data of two conversation parties; then determining whether the conversation scene data needs to be retrieved; if not, directly generating a reply on the basis of the conversation scene data; and if yes, performing accurate retrieval from a resource database to obtain a required retrieval result, thereby generating a reply on the basis of the obtained retrieval result. It can be understood that whether retrieval needs to be performed is determined, specific information is retrieved from the resource database when retrieval needs to be performed, and a reply is generated on the basis of the retrieval result, that is, the reply content comprises the content of the retrieval result, so that the generated reply is more accurate, thereby improving the experience of a user using the conversation system.

Description

Interaction method, electronic device, readable storage medium and program product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on February 26, 2024, with application number 202410211616.7 and application name “An interactive method, electronic device, readable storage medium and program product”. The entire contents of the above application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to an interaction method, an electronic device, a readable storage medium, and a program product. Background Art

[0003] Currently, artificial intelligence technology can be applied to scenarios where people exchange messages through electronic devices. For example, when people chat with each other through mobile applications, the application used by each user can help the user generate replies based on the messages sent by the other party, thereby assisting the user in replying to the other party.

[0004] Existing technologies utilize large models, such as artificial intelligence generated context (AIGC), due to their semantic analysis and logical reasoning capabilities. These models can generate responses based on the content of online communications between two parties, thereby assisting users in responding to messages sent by their counterparts. However, in some scenarios requiring retrieval, such as conversations about movie dates and event schedules, accurate responses cannot currently be generated.

[0005] Summary of the Invention

[0006] In order to solve the above problems, the embodiments of the present application provide an interactive method, an electronic device, a readable storage medium and a program product, which are used to solve the problem that in a person-to-person dialogue scenario, when a search is required, accurate responses cannot be provided to users in real time.

[0007] In a first aspect, an embodiment of the present application provides an interaction method, comprising: displaying a first user conversation; if the first user conversation is a user conversation that requires retrieval, displaying a first reply related to a retrieval result corresponding to the first user conversation, wherein the retrieval result is retrieved based on the retrieval intention of the first user conversation; if the first user conversation is a user conversation that does not require retrieval, displaying a second reply to the first user conversation.

[0008] As you can understand, during a chat, the electronic device will display the user conversation. During this process, the electronic device can display replies in real time to help users send messages to each other. Sometimes, during a chat, users may touch upon topics that require database retrieval, such as movie viewings, event schedules, and so on. In these cases, a search is necessary. Therefore, different replies can be displayed depending on whether the user conversation requires retrieval. If a search is required, for example, if the current conversation involves knowledge quizzes, personal event schedules, user information, movie viewings, shopping, and so on, a first reply related to the search results can be displayed. This means that the first reply includes content related to the search results. If a search is not required, for example, if the two parties are simply chatting, a second reply will be displayed, which does not include content related to the search results.

[0009] Understandably, by determining whether a search is necessary and using different methods to handle the situation when a search is needed and when it is not, the generated responses are more targeted. When a search is required, the generated responses include content related to the search results, making them more accurate and improving the user experience with the dialogue system.

[0010] It is understandable that since the retrieval results are obtained based on the retrieval intent of the first user conversation, in scenarios where no retrieval is required, that is, there is no need to identify the retrieval intent, time and computing resources are saved.

[0011] In a possible implementation of the first aspect above, the method further includes: displaying the first reply or the second reply in the form of a card or a list, and the first reply or the second reply may include at least one data form of text, image, voice, video, and link.

[0012] In a possible implementation of the first aspect, the method further includes:

[0013] Acquire conversation scene data related to the first user conversation, where the conversation scene data includes at least one of the conversation content of the first user conversation, user behavior information regarding the first user conversation, and physiological indicators of the user during the first user conversation; call a large language model, and the large language model determines, based on the conversation scene data, whether the first user conversation is a user conversation that needs to be retrieved; if the first user conversation is a user conversation that needs to be retrieved, the large language model identifies the retrieval intent of the first user conversation based on the conversation scene data.

[0014] It is understood that the content of the first user's conversation may include the current and historical conversation content between the two parties; the user's behavioral information regarding the first user's conversation may include user behavior information such as editing, browsing, and page scrolling in the conversation system; and the user's physiological indicators during the first user conversation may include heart rate, facial expressions, etc. It is understood that the use of multi-dimensional conversation scene data can improve the accuracy of determining whether a search is necessary, so that when a search is required, the need for a search can be reflected in a timely manner, thereby improving the accuracy and timeliness of conversation processing. Furthermore, because the conversation scene data includes multi-dimensional data, the elements used to generate replies are richer. At this time, when a reply is generated based on multi-dimensional data, the generated reply is more accurate.

[0015] In a possible implementation of the first aspect above, the first reply includes a fused retrieval result corresponding to the retrieval result.

[0016] It can be understood that the fused search results not only include the content corresponding to the search results, but also include the search replies corresponding to the search results that can be used to reply to the messages sent by the other party, and the search replies correspond to the search results, so that users can obtain more comprehensive, more relevant and more accurate replies.

[0017] In a possible implementation of the first aspect above, the fused retrieval result is obtained in the following manner: the large language model generates the fused retrieval result based on the input retrieval result and the conversation scene data of the first user conversation.

[0018] It can be understood that the retrieval results and the conversation scene data of the first user conversation are used as inputs of the large language model, so that the large model can be integrated with the retrieval results corresponding to the retrieval intent and the content analyzed in the conversation scene data, thereby providing users with high-quality responses.

[0019] In a possible implementation of the first aspect above, the retrieval results are obtained in the following manner: according to the retrieval information corresponding to the retrieval intent, a search is performed in the resource database to obtain the retrieval results, wherein the retrieval information includes at least one of the keywords corresponding to the retrieval intent and the search sentences corresponding to the retrieval intent; the resource database includes a local resource database, or a cloud resource database, and the local resource database includes local data or local applications; the cloud resource database includes: applications or services deployed in the cloud.

[0020] It is understandable that the search results obtained by searching various resource databases are very sufficient.

[0021] In a possible implementation of the first aspect, the method further includes: when the first user conversation is a user conversation that needs to be retrieved, the large language model generates a second reply based on the conversation scene data.

[0022] It can be understood that the large language model has semantic understanding and reasoning capabilities, and the large language model can effectively generate a second response based on the dialogue scenario data.

[0023] In a possible implementation of the first aspect above, the second reply includes a dialogue reply and a reply reason corresponding to the dialogue reply.

[0024] It is understood that when a reply is generated based on the conversation scene data without the need for retrieval, the generated reply may include the conversation reply and the reason for the reply. The reason for the reply is the analysis of the generated conversation reply, and the reason for the reply or the conversation reply takes into account the emotional reactions of both parties in the conversation, thereby improving the accuracy of the reply information and avoiding the problem of both parties being disgusted due to inappropriate reply content. It is understood that the electronic device can use a pre-trained large language model, take the conversation scene data as input, obtain the conversation reply and the reason for the reply, and the output reply needs to take into account the emotional tendencies of both parties, so that the large model can generate conversation replies and reasons for the reply that meet the current user needs based on the conversation scene data, and the output content can take into account the intentions and emotional tendencies of both parties.

[0025] In a second aspect, an embodiment of the present application provides an interaction method, applied to an electronic device, the method comprising: displaying a first user conversation; displaying a first reply to the first user conversation based on conversation scene data of the first user conversation, wherein the conversation scene data includes at least one of the user's behavioral information regarding the first user conversation and the user's physiological indicators of the first user conversation.

[0026] It is understood that the content of the first user's conversation may include the current and historical conversation content between the two parties; the user's behavior information regarding the first user's conversation may include user behavior information such as editing, browsing, and page scrolling in the conversation system; and the user's physiological indicators during the first user conversation may include heart rate, facial expressions, etc. It is understood that the use of multi-dimensional conversation scenario data enriches the elements used to generate responses, thereby making the generated responses more accurate.

[0027] In a possible implementation of the second aspect above, the first reply includes at least one of the following: a retrieval result corresponding to the conversation with the first user, a fused retrieval result related to the retrieval result, a conversation reply, and a reply reason corresponding to the conversation reply.

[0028] In a possible implementation of the second aspect above, it also includes: calling a large language model, and the large language model determines whether the first user conversation is a user conversation that needs to be retrieved based on the conversation scene data; if the first user conversation is a user conversation that needs to be retrieved, displaying the retrieval results or the fused retrieval results; if the first user conversation is a user conversation that does not need to be retrieved, displaying the conversation reply and the reply reason corresponding to the conversation reply.

[0029] It is understandable that whether a search is needed is determined based on the conversation scenario data, and different content is displayed when a search is needed and when a search is not needed, so that the displayed replies are more targeted and the user experience is improved.

[0030] In a possible implementation of the second aspect above, the fused retrieval result is obtained in the following manner: the large language model identifies the retrieval intent of the first user conversation based on the conversation scene data; the large language model searches the resource database based on the retrieval information corresponding to the retrieval intent to obtain the retrieval result, wherein the retrieval information includes at least one of the keywords corresponding to the retrieval intent and the search sentence corresponding to the retrieval intent; the large language model generates the fused retrieval result based on the input retrieval result and the conversation scene data of the first user conversation.

[0031] In a possible implementation of the second aspect above, the dialogue response and the response reason corresponding to the dialogue response are obtained in the following manner: when the first user dialogue is a user dialogue that needs to be retrieved, the large language model generates a dialogue response and the response reason corresponding to the dialogue response based on the dialogue scene data.

[0032] In a possible implementation of the second aspect above, it also includes: displaying the first reply in the form of a card or a list, and the first reply may include at least one data form of text, image, voice, video, and link.

[0033] In a third aspect, an embodiment of the present application provides an interaction method, applied to an electronic device, the method comprising: displaying a first user conversation; displaying a first reply corresponding to the first user conversation, the first reply comprising a conversation reply and a reply reason for the corresponding conversation reply.

[0034] As you can understand, when generating a reply based on the conversation scenario data, the generated reply may include the conversation response and the reason for the reply. The reason for the reply is an analysis of the generated conversation response, and the reason for the reply or the conversation response takes into account the emotional reactions of both parties in the conversation, thereby improving the accuracy of the reply information and avoiding the problem of inappropriate reply content causing resentment between both parties.

[0035] In a possible implementation of the third aspect above, it also includes: obtaining conversation scene data related to the conversation with the first user, the conversation scene data including at least one of the conversation content of the first user conversation, the user's behavioral information regarding the conversation with the first user, and the user's physiological indicators of the conversation with the first user; calling the large language model, and the large language model generating a first reply based on the conversation scene data.

[0036] It can be understood that electronic devices can use pre-trained large language models, take conversation scene data as input, obtain conversation replies and reasons for replies, and the output replies need to consider the emotional tendencies of both parties, so that the large model can generate conversation replies and reasons for replies that meet the current user needs based on the conversation scene data, and the output content can take into account the intentions and emotional tendencies of both parties.

[0037] In a possible implementation of the third aspect above, the method further includes: the large language model determining, based on the conversation scenario data, whether the first user conversation is a user conversation that needs to be retrieved; and generating the first reply if the first user conversation is a user conversation that does not need to be retrieved.

[0038] In a fourth aspect, an embodiment of the present application provides an interaction method, applied to an electronic device, the method comprising: displaying a first user conversation, the first user conversation being a user conversation that needs to be retrieved; displaying a first reply related to a retrieval result corresponding to the first user conversation, the first reply comprising a fused retrieval result corresponding to the retrieval result, wherein the retrieval result is obtained by retrieval based on the retrieval intention of the first user conversation.

[0039] In a possible implementation of the fourth aspect above, it also includes: obtaining conversation scene data related to the conversation with the first user, the conversation scene data including at least one of the conversation content of the first user conversation, the user's behavioral information regarding the conversation with the first user, and the user's physiological indicators of the conversation with the first user; calling the large language model, and the large language model generates the first reply based on the conversation scene data and the retrieval results.

[0040] In a possible implementation of the fourth aspect, the method further includes: the large language model determining, based on the conversation scenario data, whether the first user conversation is a user conversation that needs to be retrieved; and the large language model generating a first reply if the first user conversation is a user conversation that needs to be retrieved.

[0041] In a fifth aspect, an embodiment of the present application provides an electronic device comprising: a memory for storing instructions, and one or more processors. When the instructions are executed by one or more processors, the processor executes any one of the interaction methods in the first aspect and its various implementations, the second aspect and its various implementations, the third aspect and its various implementations, or the fourth aspect and its various implementations.

[0042] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored, which execute on an electronic device any interaction method such as the first aspect and any interaction method in its various implementations of the first aspect, the second aspect and any interaction method in its various implementations of the second aspect, the third aspect and any interaction method in its various implementations of the third aspect, or the fourth aspect and any interaction method in its various implementations of the fourth aspect.

[0043] In the seventh aspect, an embodiment of the present application provides a computer program product, which includes: computer program code, which, when the computer program code runs on a computer, enables the computer to execute any interaction method in the first aspect and its various implementations, any interaction method in the second aspect and its various implementations, any interaction method in the third aspect and its various implementations, or any interaction method in the fourth aspect and its various implementations.

[0044] Among them, the beneficial effects of the fifth to seventh aspects can refer to the relevant beneficial effects of the first, second, third and fourth aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] FIG1A shows a schematic diagram of a chat conversation scenario between people according to some embodiments of the present application;

[0046] FIG1B shows a schematic diagram of a chat interface 001 according to some embodiments of the present application;

[0047] FIG2A shows a schematic diagram of a chat interface 002 according to some embodiments of the present application;

[0048] FIG2B shows a schematic diagram of a chat interface 003 according to some embodiments of the present application;

[0049] FIG3 shows a schematic diagram of an interaction method according to some embodiments of the present application;

[0050] FIG4A shows a schematic diagram of a system architecture 400 according to some embodiments of the present application;

[0051] FIG4B shows a schematic diagram of interaction between a terminal 100B and a cloud 200 according to some embodiments of the present application;

[0052] FIG5A shows a schematic diagram of an interface 004 according to some embodiments of the present application;

[0053] FIG5B shows a schematic diagram of a search process according to some embodiments of the present application;

[0054] FIG6 shows a schematic diagram of modules of a terminal 100 and a cloud 200 according to some embodiments of the present application;

[0055] FIG7 shows another module schematic diagram of the terminal 100 and the cloud 200 according to some embodiments of the present application;

[0056] FIG8 shows a schematic diagram of a device according to some embodiments of the present application. DETAILED DESCRIPTION

[0057] The illustrative embodiments of the present application include, but are not limited to, an interactive method, an electronic device, a readable storage medium, and a program product.

[0058] As mentioned above, currently when two parties are chatting, if the content of the conversation involves some scenarios that require retrieval, they are unable to generate accurate responses.

[0059] The following describes the embodiments of the present application in conjunction with the accompanying drawings.

[0060] Figure 1A illustrates a chat conversation scenario between people, according to some embodiments of the present application. User KA and user KB communicate using the same chat application (e.g., instant messaging application 1 in the figure) on their respective terminals. It will be appreciated that the chat application is a conversation system that allows both parties to engage in a chat conversation. For ease of explanation, the conversation system on terminal 100A is referred to as "conversation system A0," and the conversation system on terminal 100B is referred to as "conversation system B0." At this point, user KA and user KB are communicating using conversation system A0 and conversation system B0, respectively.

[0061] Currently, the dialogue system can also recommend automatic reply content to users based on their chat records.

[0062] Figure 1B illustrates a chat interface 001 of terminal 100B based on dialogue system B0, during a chat between user KA and user KB, according to some embodiments of the present application. The content sent by "User A-1" on interface 001 is the chat content sent by user KA via terminal 100A, and the content sent by "User B-1" on interface 001 is the chat content sent by user KB via terminal 100B. Specifically, referring to the chat content on interface 001, user KA sends a message to terminal 100B, as shown in the figure, corresponding to user A-1, via terminal 100A, "What are your plans after get off work today?" User KB replies to user KA via terminal 100B, as shown in the figure, corresponding to user B-1, "I don't have anything to do tonight." User KA then sends a message to terminal 100B, as shown in the figure, corresponding to user A-1, "So?" User KB replies to user B-1 via terminal 100A, "Then let's go play together!" User KA then replies to user A-1, "Then let's go see the newly released movie 1." At this point, the dialogue system will generate a simple reply “OK!” in the prompt card K11 as shown in FIG1B .

[0063] Understandably, the conversation scenario shown in FIG1B includes content related to Movie 1, but the generated reply does not involve the movie content, and the generated reply is not accurate. Understandably, if the user needs to obtain accurate information about Movie 1, they need to search the application that allows purchasing movie tickets to obtain accurate information about Movie 1.

[0064] Therefore, to address the above-mentioned issues, the present application proposes an interactive method in which the conversation scenario data of the two parties is first obtained. A determination is then made as to whether the conversation scenario data requires retrieval. If retrieval is not required, for example, when the two parties are merely chatting, a reply is generated directly based on the conversation scenario data. If retrieval is required, for example, when the current conversation scenario involves knowledge questions and answers, personal activity schedules, user information, movie watching, shopping, and other content, a resource database (e.g., a local resource database or a cloud resource database) is searched accurately to obtain the required search results, and a reply is then generated based on the obtained search results. It can be understood that by determining whether a search is required and using different methods when a search is required and when it is not, the generated reply is more targeted. Furthermore, in scenarios where a search is required, specific information is retrieved from the resource database, and a reply is generated based on the search results. That is, the content of the reply includes the content of the search results, making the generated reply more accurate and improving the user experience of the conversation system. Furthermore, in scenarios where a search is not required, generating a reply directly based on the conversation scenario data saves time and computing resources.

[0065] For example, in the scenario shown in Figure 1B above, when a search is determined to be necessary, a large model can be used to understand and analyze the conversation, thereby obtaining keywords or phrases related to watching movies that can be used for search, such as the keyword "movie 1." This keyword can then be searched across various resource databases to obtain accurate information. A response related to the search results can then be generated based on the search results, making the generated response more accurate.

[0066] In some embodiments, in order to improve the accuracy of the judgment result of whether retrieval is needed, multi-dimensional data can be obtained as conversation scene data. For example, the conversation scene data can include not only the current and historical conversation content of the two parties, but also the user behavior information such as editing, browsing, page sliding, etc. of the current user in the conversation system, as well as the current user's physiological indicator data, such as heart rate, expression, etc., which are used to judge whether retrieval is needed, thereby improving the accuracy of judging whether retrieval is needed, so that when retrieval is needed, the need for retrieval can be reflected in a timely manner, improving the accuracy and timeliness of conversation processing. In addition, because the conversation scene data includes multi-dimensional data, the elements used to generate replies are richer. At this time, when generating replies based on multi-dimensional data, the generated replies are more accurate.

[0067] Furthermore, in some embodiments, when a response is generated based on the conversation scenario data without the need for retrieval, the generated response may include the conversation response and the reason for the response. The reason for the response is an analysis of the generated conversation response, and the reason for the response or the conversation response takes into account the emotional reactions of both parties to the conversation, thereby improving the accuracy of the response information and avoiding the problem of both parties being offended by inappropriate reply content. For example, the terminal device can use a pre-trained large model, take the conversation scenario data as input, obtain the conversation response and the reason for the response, and the output response needs to take into account the emotional tendencies of both parties, so that the large model can generate conversation responses and reasons for the response that meet the current user needs based on the conversation scenario data, and the output content can take into account the intentions and emotional tendencies of both parties.

[0068] FIG2A shows a schematic diagram of a chat interface 002 of a terminal 100B based on a dialogue system B1 during a chat between users KA and KB according to some embodiments of the present application. Interface 002 of terminal 100B displays the content sent by user A-2 to user B-2: "What?" ", user KB can perform some reply operations in the input box K21 of the dialogue system B1 in response to the content sent by user KA. For example, user KB clicks and enters the text "I am going to the Internet cafe to play by myself" in the input box K21. When the dialogue system B1 detects that the time it takes for the user to click and enter text reaches the preset reply time, it can call the big model and use the text entered by the current user and the previous conversation content between the two parties as dialogue scene data, so that the big model generates a reply based on the dialogue scene data; it can also collect the psychological indicator data of the current user KB and the behavioral information of user KB, so that the dialogue scene data includes the psychological indicator data of the current user KB and the behavioral information of user KB for the big model to generate a reply. It can be understood that the big model can analyze the intention of user KA to chat with the current user KB based on the dialogue scene data, which is actually the expectation to do something with user KB. The big model then automatically generates a reply, which includes the dialogue reply and the reason for the reply. The dialogue system B1 displays the generated reply in the form of a suggestion card K22.

[0069] As shown in the suggestion card K22 in Figure 2A, the reply includes a box K22A corresponding to the reason for the reply and a box K22B corresponding to the conversational response. Referring to Figure 2A, box K22A contains the specific reason for the reply: "The other party asked about your evening plans and is looking forward to doing something with you. Your current response may upset the other party." Furthermore, box K22B contains the conversational response, "Then let's go play together!", generated by the large model displayed by dialogue system B1. User KB can click the Send control K22C corresponding to the conversational response "Then let's go play together!" to send the conversational response directly to user KA. Furthermore, user KB can directly select and modify the conversational response "Then let's go play together!" in box K22B, then send the modified content by clicking the Send control K22C. In addition, user KB can also modify the original content in the input box based on the displayed dialogue reply "Then let's go play together!", for example, modifying it to the same content as the dialogue reply "Then let's go play together!", and then click the send control K23 corresponding to the content in the input box to send it. It is understandable that the send control can also be omitted in the suggestion card K22. The user can directly select the dialogue reply and reply reason in the suggestion card, and then the terminal 100B directly sends it to user KA. The specific sending method is not specified here.

[0070] Understandably, the specific reply reason displayed in suggestion card K22 analyzes user KA's intention in chatting with the current user KB, which is actually an expectation to do something together. Therefore, user KB's original reply may have caused user KA to feel unhappy. Furthermore, based on the psychological indicators collected for user KB, such as a normal heart rate and calm mood, user KB also does not feel any resentment towards user KA, thus enabling the appropriate response, "Then let's go play together!" The generated reply reason and the conversational response fully consider the emotional tendencies of both parties, providing the current user with a more accurate, friendly, and appropriate response. This ensures that when the user communicates with the other party based on accurate and reasonable responses, they will not cause resentment or discomfort.

[0071] In other embodiments, when a search is needed, various resource databases can be searched to obtain search results. The search results obtained from the various resource databases can then be integrated with the content analyzed from the conversation scenario data to provide users with high-quality responses. For example, the macro model can integrate the search results obtained from the various resource databases with the response content generated by the macro model based on the conversation scenario data, so that the generated response includes the integrated search results corresponding to the search results. It is understood that the integrated search results not only include the content corresponding to the search results, but also include a search reply corresponding to the search results that can be used to reply to the message sent by the other party, and the search reply corresponds to the search results. It is understood that the search reply reflects the response content generated by the macro model for the conversation scenario data. Moreover, since the search results obtained may include information from both local and cloud services, the generated and displayed integrated search results allow users to obtain more comprehensive, relevant, and accurate responses.

[0072] For example, according to some embodiments of the present application, FIG2B illustrates a chat interface 003 of terminal 100B based on dialogue system B1 during a chat between users KA and KB. The content of the conversation between "User A-2" and "B-2" on interface 003 is identical to the content already shared between "User A-1" and "B-1" in FIG1B . Furthermore, user KB enters "Let's see where to go" in input box K31. At this point, dialogue system B1 generates a response, as shown in suggestion card K32 in FIG2B . The response displayed in suggestion card K32 includes a fused search result. Referring to FIG2B , box K32A contains content corresponding to the search result: "Searching for ticket information for tonight's movie 1: XXX Cinema (XXX Store): ￥49.9, Showtimes Tonight: 7:30 PM | 8:00 PM | 9:15 PM." Furthermore, box K32B contains the search response generated by the dialogue system, "Why don't we go to this place?" As can be understood, the movie-watching content involved in the current conversation is strongly related to movie-related applications. At this point, the terminal can retrieve valid information from the corresponding application and present the generated reply in an optimized manner. This allows the user KB to quickly obtain the information they need without having to search other resource databases. Furthermore, the fused search results include content generated by the large model based on the conversation scenario data, eliminating the need for the user to enter a search reply related to the movie ticket search, thus improving the user experience. Similarly, the user can modify the content in box K32A or box K32B and then send it using the corresponding send controls K32C and K32D, respectively, or directly send the content in box K32A or box K32B using the send control. The user can also modify the content in input box K31 based on the reply in suggestion card K32 and then send it using the send control K33 corresponding to input box K31. The user can also directly select the content in suggestion card K32 and send it without using the send control. The specific sending method is not specified here.

[0073] It is understandable that large language models include but are not limited to artificial intelligence generated context (AIGC), ChatGPT, LaMDA, PaLM, and OPT-IML, which will not be detailed here.

[0074] It is understandable that the terminal can display the reply in the form of cards, lists or other forms, so that the user can intuitively obtain relevant information.

[0075] It can be understood that the dialogue system can be any application that can provide a chat function and can be deployed on any electronic device.

[0076] It can be understood that terminal 100A and terminal 100B can be any electronic device that runs a dialogue system, for example, it can include but is not limited to mobile phones, tablet computers, vehicle-mounted equipment, augmented reality (AR) / virtual reality (VR) devices, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), servers, server clusters, etc., without limitation here.

[0077] Figure 3 provides a schematic diagram of an interactive method according to an embodiment of the present application. In this method, a terminal device displays a user conversation and determines whether a search is required. Different responses are used depending on whether a search is required or not. For ease of explanation, the following description is based on the processor in the terminal device as the execution subject. The specific steps are as follows:

[0078] S301, obtaining the conversation scene data of the current user conversation.

[0079] In some embodiments, a terminal device displays a user conversation and can monitor the conversation process in real time on the dialogue system. When it is determined that a response needs to be generated, conversation scene data related to the current user conversation is collected. For example, upon detecting the user's first action, conversation scene data is obtained. It is understood that a user conversation is any conversation content displayed on the terminal device, such as chat messages sent by two parties displayed on the terminal device.

[0080] The first operation can be, for example, a user clicking on an input box on the interactive interface and entering text, emoticons, or the like, or a user clicking and entering content in the input box for a preset reply time, or a user selecting a microphone for voice input, or a user selecting a reply control (not shown) configured on the interface. It is understood that the first operation can be any of a variety of operations, as long as they indicate that the user wants to chat with the other party, and these operations will not be detailed here.

[0081] Conversational context data may include, but is not limited to, current and historical conversation content between the two parties, user behavior information, and the current user's physiological indicator data. Conversational content includes both sent and pre-sent content, and may include text, voice, video, images, and other information. User behavior information includes the current user's actions within the conversation system, such as editing, browsing, and scrolling. For example, if the current chat content involves a previously discussed topic, the user can return to the previous chat content to view and edit it. In this case, user behavior information includes the user's scrolling actions, editing actions on the previous chat content, and the viewed chat content data. The current user's physiological indicator data includes the current user's facial expressions, heart rate, and other information. For example, with the user's permission, the terminal device can capture the user's facial expressions through a camera. These expressions may correspond to emotions such as anger, peace, happiness, and sadness. For heart rate, the user's heart rate data can be obtained through internal devices capable of monitoring heart rate or other supporting equipment. It is understood that conversational context data may only include conversation content, and this is not a limitation. It is understandable that when using multi-dimensional dialogue scene data, the accuracy of the judgment results can be improved because the information of the dialogue scene data is more comprehensive, so that when retrieval is needed, it can be performed in real time, thereby improving the accuracy and timeliness of dialogue processing.

[0082] For example, for the scenario shown in Figure 2A above, when it is detected that user KB enters text in the input box K21, the 10 most recent sent contents of user A-2 and user B-2 in the current chat conversation and the pre-sent content entered by user KB in the input box K21 can be obtained; the user's expression captured by the camera, such as "peaceful", and the heart rate data of "75bpm" collected by the watch can also be obtained.

[0083] For another example, for the scenario shown in FIG2B above, when it is detected that user KB enters text in the input box K31, the 10 most recent sent contents of user A-2 and user B-2 in the current chat conversation, as well as the content entered by user KB in the input box K31 can be obtained; the user slides to the topic of "Movie 1" discussed the day before yesterday to view the behavioral information of the previous content; the user's expression captured by the camera, such as "peaceful", and the heart rate data of "75bpm" collected by the watch can also be obtained.

[0084] S302: Determine whether a search is required based on the conversation scene data.

[0085] If not, go to S303; if so, go to S304.

[0086] It is understandable that the current user conversation may involve some content that needs to be retrieved, such as movies, food, restaurants, buildings, names of people, trademarks, place names, customs, travel services, transportation, gossip information, event arrangements, and specific content of various disciplines such as law, music, painting, medicine, etc., which are not listed here. And there may be multiple contents that need to be retrieved in the current user conversation, so it is necessary to determine whether the current user conversation is a user conversation that needs to be retrieved. In the case that the current user conversation is a user conversation that needs to be retrieved, a search is required. In the case that the current user conversation is a user conversation that does not need to be retrieved, no search is required. Thus, when a search is required, the search intent can be accurately identified later to obtain the search results. Moreover, when a search is not required, there is no need to identify the search intent and subsequent search, and a reply is directly generated, thereby saving time and computing resources.

[0087] In some embodiments, the terminal device invokes a trained large model to determine whether the current conversation scenario data requires retrieval. For example, the large model can determine whether the current user conversation is a user conversation requiring retrieval, i.e., whether a retrieval is necessary, by extracting keywords or key sentences corresponding to the content to be retrieved from the current conversation scenario data. If the large model determines that a retrieval is necessary, the process proceeds to S304. If the large model determines that a retrieval is not necessary, the process proceeds to S303.

[0088] For example, for the scenario corresponding to small talk shown in FIG. 2A , the large model may be used to determine that no search is required, and the process proceeds to S303 .

[0089] For another example, in the scenario shown in FIG2B where there is content about watching movies, the large model can be used to identify keywords related to movies and other content, thereby determining that a search is to be performed and entering S304.

[0090] It is understood that the large model's ability to determine whether a search is necessary can be deployed on either the terminal device or the cloud device. When the large model's ability to determine whether a search is necessary is deployed on the cloud device, the terminal device can send the conversation scenario data to the cloud, which then specifically determines whether a search is necessary based on the conversation scenario data. That is, step S302 can also be executed on the cloud, with the cloud feeding back the large model's determination result to the terminal device.

[0091] Specifically, the large model may include a discriminant model to determine whether retrieval is required. It is understood that the discriminant model may include a neural network model, a convolutional network model, or other algorithms, although this is not required here. In some embodiments, the discriminant model in the large model can be pre-trained using any electronic device, such as a terminal device or a cloud device.

[0092] Taking the training of a large model on a cloud device as an example, the discriminant model can be trained as follows: Conversational scenario data related to various conversational scenarios is prepared in advance and used as training sample data. For ease of explanation, the conversational scenario data used for training is referred to as "conversational scenario sample data." Each conversational scenario sample data is assigned a corresponding label. Conversational scenario sample data that requires retrieval is labeled as "searchable," while conversational scenario sample data that does not require retrieval is labeled as "non-searchable." The training sample data and corresponding labels are then fed into the initial model to be trained, and the model is trained to obtain the discriminant model.

[0093] It is understandable that the conversation scene sample data includes not only conversation scenes that do not need to be retrieved, such as conversation scenes for small talk such as asking the other party how old they are this year, their home address, what they are wearing, and praising each other, but also conversation scenes that need to be retrieved. Among them, the conversation scene sample data that need to be retrieved can be generated by a pre-configured resource database to generate the required conversation scene sample data, so that the training sample data includes various types of content in multiple dimensions. It is understandable that the resource database configured in the training process is only used for the training process. When the large model training is completed and actually used, the resource database for retrieval may include but is not limited to the aforementioned resource database, or may not include the aforementioned resource database, and this is not required here.

[0094] For example, the pre-configured resource database for training the model can be: (1) Knowledge base: data containing common user questions and related answers, such as data with relevant answers such as the birth date of "Qin Shi Huang". (2) External data source: including external data sources such as online event calendars, social media platforms or user personal calendars that involve user security and other information. (3) Third-party application (APP) information: including registration information of third-party applications (APPs) and specific function information. It is understandable that third-party applications can provide various related functions. When it comes to functions, they also need to be retrieved, so they can be used as important data to determine whether to retrieve. Optionally, the conversation scene sample data used for training can include not only the conversation content, but also the user's behavior information and psychological indicator data.

[0095] It is understandable that the data content in the aforementioned data sources is the content that people often talk about in their daily lives. In this case, sample conversation scene data can be generated based on the aforementioned data sources, which is widely representative.

[0096] It is understandable that the discriminant model can also be obtained by other methods, which will not be described here.

[0097] S303: Generate a reply based on the conversation scenario data.

[0098] It can be understood that step S303 is an optional step.

[0099] It can be understood that after searching and judging based on the dialogue scene data, only the following steps S304-S306 may be executed.

[0100] In some embodiments, the terminal device generates a reply based on the conversation scenario data, and the generated reply may only include the conversation reply.

[0101] In other embodiments, the terminal device generates a reply based on the conversation scenario data, and the generated reply may include both the conversation reply and the reply reason corresponding to the conversation reply.

[0102] For example, referring to the interface 002 shown in FIG. 2A , the prompt card K22 shows the dialogue response and the reason for the response based on the current dialogue scene.

[0103] In some embodiments, the terminal device can generate responses based on the conversation scenario data using a large model. The large model can include a generative model for generating responses based on the conversation scenario data and other information. The generative model can be a recurrent neural network, a transformer model, or other generative model, though this is not a requirement.

[0104] It is understandable that the large model's ability to generate responses can be deployed on the terminal device or on the cloud device. When the large model's ability to generate responses is deployed on the cloud device, the terminal device can receive the response content sent by the cloud and thus obtain the response.

[0105] In some embodiments, any electronic device, such as any terminal device or cloud device, can be used to pre-train the generative model in the large model. Taking the cloud device training the large model as an example, the cloud device obtains the generative model through the Prompt fine-tuning training method.

[0106] Specifically, a large amount of sample conversation scenarios, obtained from a data source, is fed into the initial model for training. This allows the model to learn language skills and thus complete context, such as pre-configured templates with fixed structures. The model's parameters are then fine-tuned to ensure that the output includes reasonable responses and reasons for the responses, and that the responses take into account the intentions and emotional tendencies of both parties. For example, a pre-defined template structure is defined and filled in the corresponding positions to generate human-readable responses and suggestions. For example, the template could be: "The other party asks about your plans for the evening (where "evening" represents the time), and is looking forward to doing something with you ("doing something with you" represents the action). Your current response may cause the other party to be upset ("the other party's upset mood" represents the consequence)." The key information is then filled in the template to generate the corresponding response.

[0107] Understandably, other conversation understanding technologies and semantic understanding algorithms can also be used to understand the user's questions, recommend responses that are consistent with the user's intent and take into account the other party's emotions, tone and other factors to provide a satisfactory answer.

[0108] S304: Identify the search intent and obtain search information corresponding to the search intent.

[0109] In some embodiments, since it has been determined that a search is required, that is, the conversation scene data involves content that can be searched, the terminal device can call the large model at this time, so that the large model can identify the search intent based on the current conversation scene data. Then, the terminal device generates search information that can be searched in various resource databases supported by the current terminal device based on the search intent, such as query parameters, so as to perform a search based on the search information. It can be understood that the search intent is an intention that can be used to accurately search various resource databases to obtain accurate information. The search intent is a more precise expression of the intent when the search content is used for retrieval, which can involve various fields, including but not limited to music, law, writing, painting, learning, life and other aspects. The search information can be a search sentence, keyword, etc. corresponding to the search intent.

[0110] It is understandable that in some implementations, the intent recognition capability of the large model can be deployed on the terminal device or on the cloud device. When the intent recognition capability of the large model is deployed on the terminal device, at this time, the terminal device can identify the retrieval intent when a retrieval is required, and then generate retrieval information. When the intent recognition capability of the large model is deployed on the cloud device, at this time, the cloud device can continue to identify the retrieval intent when a retrieval is required, and then generate retrieval information, that is, step S304 can be executed on the cloud.

[0111] Specifically, the large model may also include an intent recognition model for identifying retrieval intent. The intent recognition model may be a neural network model, a convolutional network model, or other algorithms, etc., which are not limited here. The intent recognition model can understand and classify the user's intent in the conversation scenario and obtain a feature vector related to the intent. Using the classification algorithm and the probability model, the output feature vector is input into the classification algorithm or the probability model to determine the most likely retrieval intent. It is understandable that when a retrieval is required, the retrieval intent is further accurately identified through the large model, so that the required content can be accurately retrieved from various resource databases through the retrieval information corresponding to the retrieval intent.

[0112] For example, referring to interface 003 shown in Figure 2B above, the large model can identify the search intent involving movie 1, and then generate search information based on the identified search intent, for example, "Order 1 movie ticket for tonight's movie 1", or keywords such as "order", "movie 1", "watch", and "tonight". The specific form of the search information is not required here.

[0113] In some implementations, when training the intent recognition model, the cloud device can adopt the Prompt fine-tuning training method to obtain the intent recognition model: by training the model with training sample data including various corpora, the model learns the general language representation, and then fine-tunes the pre-trained model according to different downstream tasks. For example, the intent recognition model is used to output the preliminary intent, and then the preliminary intent is fine-tuned to obtain the target retrieval intent, and then the model is fine-tuned according to the target retrieval intent. Among them, the training sample data used to train the intent recognition model can be the dialogue scene sample data from the configured resource database used in the process of training the discriminant model. Please refer to the above step S302 for details, which will not be repeated here.

[0114] S305: Obtain search results based on the search information.

[0115] In some embodiments, based on the search information, a search is performed in various resource databases to obtain search results. The various resource databases may include a local resource database of the current terminal device and a cloud resource database connected to the current terminal device. For example, the data in the local resource database may be data stored in the local storage space, including device information, user preferences, contacts, text messages, call logs, photos, local videos, application data, etc.; the data in the cloud resource database may be third-party application or service data provided by the cloud, such as third-party services deployed in the cloud for positioning, ticket purchase, flight booking, reservations, etc. used by various applications.

[0116] For example, for the search information related to Movie 1, the user can search the local data corresponding to the ticket purchasing application on the terminal device and the cloud service corresponding to the ticket purchasing application based on the search information "Order a ticket for tonight's Movie 1" and other information, thereby obtaining local search results and cloud search results. For example, the cloud search result obtained is: XXX Cinema (XXX Store): ￥49.9, Tonight's Showtimes: 19:30|20:00|21:15.

[0117] S306: Generate a reply based on the search results.

[0118] In some embodiments, the search results are used as replies. For example, the local search results and the cloud search results obtained for the aforementioned movie 1 are combined and organized and displayed as a reply on the interface.

[0119] In other embodiments, the large model fuses the search results into the reply content derived from the conversation scenario data to generate a reply, which in this case includes the fused search results. Specifically, the large model includes a generative model. The large model inputs the conversation scenario data and the search results into a trained generative model, which then fuses the search results to generate a fused search result. The terminal device displays the fused search result generated by the large model to the user, providing a high-quality reply. It is understood that the method of fusion is not a requirement here.

[0120] In addition, in other embodiments, the large model is integrated according to the generated content of the dialogue scene data and the retrieval results to obtain a reply. The generated reply includes not only the fused retrieval results, but also the reasons for the reply corresponding to the fused retrieval results.

[0121] It is understandable that in some implementations, the reply generation capability of the large model can be deployed on the terminal device or on the cloud device. When the intent recognition capability of the large model is deployed on the terminal device, the terminal device can directly display the reply generated by the large model. When the reply generation capability of the large model is deployed on the cloud device, the cloud device can generate the reply, that is, the above step S306 can be executed on the cloud. Then, the cloud device sends the reply to the terminal device, and the terminal device displays the received reply.

[0122] For example, referring to Figure 2B , box K32A contains specific content corresponding to the search result: "Searching for ticket information for tonight's movie 1: XXX Cinema (XXX Store): ￥49.9, Tonight's Showtimes: 19:30 | 20:00 | 21:15." Furthermore, box K32B contains the search response displayed on the terminal device: "Why don't we go to this place?" It's understandable that the movie-watching content involved in the current conversation is strongly related to movie-related applications. At this point, the big model can retrieve valid information from the corresponding application, and the terminal device can present the response generated by the big model in a better way to prevent the user from receiving an accurate and detailed answer or solution.

[0123] It can be understood that in some implementations, taking the fusion of a large model as an example, the generative model used can be the same as the generative model used in the above step S303 when retrieval is not required. However, when training the generative model, the training samples used for training include conversation scene sample data involving retrieval intentions, and the obtained retrieval results and conversation scene samples are used as training samples input to the generative model for training, and then training is performed. The training method can be the same as the training method of the generative model in the above step S303, or it can be different, which is not required here.

[0124] It is understandable that the execution order of the above steps S301 to S306 is only an example. In other embodiments, other execution orders may be adopted, and some steps may be split or combined, which is not limited here.

[0125] It is understandable that the terminal device displays the generated reply on the screen, such as displaying the generated reply in the form of a card or a list. The generated reply may include at least one data form of text, image, voice, video, and link. The user can edit or send the displayed reply. For example, the user can directly modify the displayed reply, and after the modification is completed, click the send control on the screen for sending the displayed reply to send the modified reply. For example, the user does not modify the displayed reply and can directly send the displayed reply through the send control for sending the displayed reply. In addition, the user can also refer to the displayed reply and adaptively modify the input content in the original input box, and send the modified content in the input box through the send control for sending the content in the input box; or directly send the content in the input box through the send control for sending the content in the input box without modifying the input content in the original input box. Specifically, please refer to the description of user KB sending a reply or the content in the input box in Figures 2A and 2B above, which will not be repeated here.

[0126] It is understood that in other embodiments, for step S302 above, the terminal device obtains conversation scenario data, where the conversation scenario data includes the current and historical conversation content between the two parties, user behavior information, and the current user's physiological indicator data. At this time, the terminal device can directly generate a reply using the large model, without having to perform operations such as determining whether a search is required and searching the resource database to obtain search results. It is understood that because the conversation scenario data includes multidimensional data, the generated reply can be more accurate based on the multidimensional conversation scenario data. It is understood that the generated reply can be at least one of a search result, a fused search result, a conversation reply, and a corresponding reply reason.

[0127] It is understandable that the executor of the interactive method shown in FIG3 may also be an application, for example, the above-mentioned dialogue system, which may call other applications to generate replies, or may directly generate replies in real time without calling other applications.

[0128] It is understood that in some implementations, the real-time response generation function can be configured within the intelligent dialogue service (Agent) on the terminal device. When interacting with a user, the dialogue system on the terminal device can, with the user's permission, invoke the intelligent dialogue service on the terminal device and use it to provide the user with high-quality, accurate responses. Specifically, the intelligent dialogue service can operate based on the content of multiple rounds of dialogue. The intelligent dialogue service can invoke a large model and display the responses output by the large model. The large model can determine whether a search is required for the dialogue content, thereby determining whether a search is necessary. If a search is required, the large model will fuse the search results obtained from the local machine and cloud services with the responses generated by the large model based on the dialogue scenario data to generate a fused search result. The terminal can then display the response to the user in an appropriate manner and save it locally, providing the user with accurate, reasonable, and high-quality responses.

[0129] In other implementations, the dialogue system can also be directly configured with the function of generating a reply. In this case, there is no need to call the intelligent dialogue service, that is, the dialogue system can directly call the large model to generate a reply.

[0130] The following uses the example of a dialogue system calling an intelligent dialogue service for real-time replies to illustrate this.

[0131] Figure 4A shows a schematic diagram of a system architecture 400 according to some embodiments of the present application. As shown, system architecture 400 is composed of two terminal devices (terminal 100A and terminal 100B) and a cloud device (cloud 200). It is understood that the terminal devices include but are not limited to mobile devices such as mobile phones and tablets that can be used for communication. The cloud device is composed of at least one server and has large-scale model computing capabilities.

[0132] Specifically, as shown in FIG4A , the terminal 100A includes a dialogue system 101A, an intelligent dialogue service 102A, and a local search 103A. The dialogue system 101A is used to provide communication services to users. The dialogue system 101A can be an application that can perform instant communication on the terminal 100A, such as a voice assistant, an intelligent assistant, and It is understandable that in other embodiments, the dialogue system 101A, the intelligent dialogue service 102A, and the local search 103A may be different applications, or may be combined into the same application, or any two of them may be combined into the same application, which is not required here.

[0133] Intelligent dialogue service 102A is used to generate responses based on the dialogue scenario data, invoking the large-scale model computing capabilities of cloud 200 or local search 103A. It is understood that the specific process by which intelligent dialogue service 102A generates responses based on the dialogue scenario data can be referred to steps S301-S305 shown in Figure 3 above and will not be elaborated here. For example, intelligent dialogue service 102A can be a system application on terminal 100A. When dialogue system 101A is configured to allow invocation of intelligent dialogue service 102A, dialogue system 101A can invoke intelligent dialogue service 102A upon detecting a user's operation in the input box, and generate a response in real time.

[0134] The local search 103A is used to search the local resource database according to the local search information to obtain the local search result.

[0135] Similarly, terminal 100B includes a dialogue system 101B, an intelligent dialogue service 102B, and a local search engine 103B. The functions of dialogue system 101B, intelligent dialogue service 102B, and local search engine 103B are essentially the same as those of dialogue system 101A, intelligent dialogue service 102A, and local search engine 103A, and are not described in detail here.

[0136] The cloud 200 includes a large model module 201 and a search engine 202 .

[0137] The large model module 201 is used to generate responses based on the conversation scenario data. Specifically, the large model module 201 may include a discriminant model, an intent recognition model, and a generation model. The large model module 201 is used to obtain conversation scenario data from terminal 100B or terminal 100A, use the discriminant model to determine whether a search is required based on the conversation scenario data, and, if a search is required, use the intent recognition model to identify the specific search intent in the conversation scenario data, so as to obtain search information based on the search intent, thereby enabling the search engine 202 or the local search engine 103B to perform the search. It will be understood that after the search information is generated, the search engine or third-party service can be called through the cloud service. In this case, the cloud service can send the search information to the application programming interface (API) of the search engine 202 via a network protocol (such as HTTP, WebSocket, etc.). The search engine 202 can also call the third-party service, or directly call the third-party service API, to obtain relevant search results or service responses.

[0138] Furthermore, the large model module 201 is also used to generate responses using the generated model. Specifically, when retrieval is not required, a response is generated directly based on the conversation scene data from the terminal 100A or the terminal 100B; when retrieval is required, a response is generated based on the conversation scene data and the retrieval results. The specific process of the large model module 201 generating responses based on the conversation scene data can be referred to the description of the large model generating responses in FIG. 3 above, which will not be elaborated here.

[0139] The search engine 202 is used to obtain search results based on the search information sent by the large model module 201 and feed back the search results to the large model module 201. That is, the search engine 202 searches the cloud resource database based on the search information to obtain the search results.

[0140] It is understandable that the system architecture 400 in FIG. 4A is merely a schematic diagram. In other embodiments, more or fewer modules may be included, and the modules may be combined or split, which will not be elaborated herein.

[0141] Taking the terminal 100B as an example, the following describes the specific process of the terminal 100B interacting with the cloud 200 to generate a reply during the conversation.

[0142] FIG4B shows a schematic diagram of the interaction process between the terminal 100B and the cloud 200 according to some embodiments provided by the embodiments of the present application.

[0143] The specific steps are as follows:

[0144] S401: Terminal 100B detects a first operation and obtains conversation scene data, including conversation content.

[0145] Optionally, the conversation scene data may also include information such as user behavior or physiological indicators.

[0146] In some embodiments, terminal 100B detects the first operation and obtains the dialogue scene data when allowed to run by the user.

[0147] In some implementations, terminal 100B may obtain conversation scene data, including conversation content, user behavior, physiological indicators and other information, in the following manners.

[0148] (1) Obtaining the conversation content by calling an API, where the conversation content includes sent content and pre-sent content. Specifically, terminal 100B can obtain the text, audio, image, and video information of the current conversation by calling the API of the chat application (such as conversation system B1). The information may include the chat content between the user and the other party, the timestamp of the conversation text, the audio recording, the image content, and the video content. In this way, terminal 100B can obtain the basic content of the conversation for subsequent analysis and processing.

[0149] (2) Obtaining user behavior information through screen recording. Specifically, terminal 100B needs to obtain user behavior information. For example, when a user communicates with another party in the current dialogue system, the user's behavior information such as browsing and editing chat records can be obtained based on the current dialogue system. Terminal 100B can obtain screen recording permissions to record the user's browsing history and editing operations in the chat application to track the user's operation behavior, thereby obtaining the user's behavioral characteristics during the dialogue process, and thus more comprehensively understanding the current dialogue scene.

[0150] (3) Obtaining physiological indicator information. Specifically, the terminal 100B can be connected to some other devices that collect user status, such as watches, bracelets, etc. At this time, the terminal 100B can interact with the application of the aforementioned collection device and obtain corresponding physiological indicator data, such as the user's expression, heartbeat, etc. In addition, the terminal device 100B can also obtain some sensor data based on built-in sensors, such as camera data, to obtain the user's current expression. This data can provide more information about the user's emotions and reactions, which helps to more accurately analyze and process conversation scenarios.

[0151] It is understandable that in order to ensure user privacy and data security, terminal 100B will only collect and use necessary information and comply with relevant privacy protection regulations.

[0152] S402, terminal 100B sends the integrated conversation scene data to the cloud 200.

[0153] It is understandable that step S402 is an optional step. In addition, S402 can also be performed by the cloud 200.

[0154] In some embodiments, the terminal 100B may organize and integrate the collected conversation scene data according to a time sequence relationship, and send it to the cloud 200 for processing.

[0155] As you can understand, the conversation scene data includes information about all aspects of the conversation. Terminal 100B can now fully understand the context of the conversation, enabling the subsequent large model in cloud 200 to generate more accurate and personalized responses and suggestions, thereby enhancing the intelligence of the conversation system and improving the user experience.

[0156] In some implementations, before terminal 100B transmits the conversation scene data to cloud 200, it is necessary to convert the collected conversation scene data into a format that can be transmitted. The specific data format will be determined based on actual needs and system design. Common data exchange formats such as JSON (javascript object notation) or Protocol Buffers can generally be used. It is understood that the actual data format can be adjusted based on specific needs and designs and is not limited here.

[0157] It is understood that the conversation scenario mentioned above includes but is not limited to (1) conversation content, which involves text, pictures, audio, and video information, (2) user behavior information, such as browsing chat records, editing information, etc., and (3) physiological indicator data, such as user expressions, heartbeat, and other physiological data, and these data will use different data formats and units depending on the specific device and application.

[0158] For example, the following describes the sent data in conjunction with the conversation scenario corresponding to the interface 002 shown in Figure 2A. The sent data includes three messages between user B-2 and user A-2 as shown in Figure 2A, and a pre-reply from user B-2.

[0159] Specifically, the pseudo code is shown below:

[0160] #Conversation content

[0161] Interlocutors: [“User A-2”, “User B-2”];

[0162] Conversation message, sent content: {"timestamp":2023-10-31T17:30:00;"sender":"User A-2";"content":"What are your plans after get off work today?"}{"timestamp":2023-10-31T17:30:10;"sender":"User B-2";"content":"I don't have any plans tonight"}{"timestamp":2023-10-31T17:30:20;"sender":"User A-2";"content":"So?"};

[0163] Pre-sent content: {"timestamp":2023-10-31T17:30:20;"sender":"User B-2"";"content":"I want to go to the Internet cafe to play by myself"}

[0164] #User behavior {"timestamp":2023-10-31T17:31:30,"behavior type":"browsing history";"browsed data":"..."}{"timestamp":2023-10-31T17:31:50,"behavior type":"edit sent content";"edited data":"..."}

[0165] #Psychological Indicators

[0166] "emotion": {"timestamp":2023-10-31T17:32:30;"expression":"neutral"},

[0167] "heartrate": {"timestamp": 2023-10-31T17:32:30; "value": 80; "unit": bpm},

[0168] As you can understand, in the above example, the data sent includes the timestamp of the conversation, the sender, and the content, all of which are recorded in the message list. Furthermore, it also includes the user's browsing history and user psychological indicators. This browsing history data is not specifically displayed in the current example. The user's psychological indicators include a "neutral" expression and a heart rate of 80 beats per minute (bpm). This information can be further processed and analyzed to provide more personalized and accurate responses and suggestions.

[0169] S403, the cloud 200 determines whether a search is required based on the conversation scenario data.

[0170] If not, proceed to S404; if required, proceed to S407. It can be understood that steps S404-S406 are optional steps.

[0171] In some embodiments, the cloud 200 may determine whether a search is required through the large model. If the determination result output by the large model is an indicator indicating that a search is not required, the process proceeds to S404; otherwise, the process proceeds to S407.

[0172] For example, in the conversation scenario data corresponding to the conversation scenario shown in FIG2A , cloud 200 may use natural language processing (NLP) technology to analyze the conversation. It may then discover that user A-2 is inquiring about user B-2's schedule, to which user B-2 responds that she has no schedule. Based on the content of this conversation segment, cloud 200 may analyze that no search is required, and therefore no search is necessary. Furthermore, because user KB did not explicitly reply with search-related content in the pre-sent message, no search results can be found in the corresponding resource database. For example, no targeted search for third-party application services is possible, and therefore no search is required. The process proceeds to S404 .

[0173] For another example, refer to the scenario shown in Figure 2B and the chat interface for a travel scenario shown in Figure 5A. Referring to the conversation content shown in Figure 2B, it can be seen that both parties have a search request for movies. In this case, cloud 200 will determine the need for a search based on the conversation scenario data. Referring to the conversation scenario involving travel topics shown in Figure 5A, in this scenario, user A-2 sends the message "I want to go on a trip," to which user B-2 replies, "Going to Dali?" User A-2 continues, "How about Hainan?" User B-2 sends the message "Okay, when should we go?" User A-2 replies, "How about taking a flight next Wednesday?" Based on the conversation content, it can be seen that both parties want to travel to Dali, and user A-2 is inquiring about the specific task of taking a flight next Wednesday. At this point, cloud 200 can determine the need for a search based on the conversation scenario data, and that user B-2 subsequently needs to send a request regarding taking a flight to Dali, thus proceeding to S407.

[0174] S404, the cloud 200 generates a response based on the conversation scenario data, including the conversation response and the reason for the response.

[0175] In some embodiments, cloud 200 can generate accurate and coherent responses by combining the large model with the conversation context corresponding to the conversation scenario data. The large model can use conversation understanding technology and semantic understanding algorithms to understand the user's questions in the conversation scenario data, recommend responses that meet the user's intent and take into account factors such as the other party's emotion and tone, and provide a satisfactory answer.

[0176] For example, in the scenario shown in Figure 2A, cloud 200 analyzes user A-2's question "So?" and user B-2's previous message, "I don't have anything to do tonight," to determine that user A-2 is asking user B-2 about their evening plans, hoping to do something together. Therefore, user B-2's pre-sent message suggests a rejection, potentially irritating the other person. The generative model can generate a conversational response: "Then let's go play together!" and a reason for the response: "The other person is asking about your evening plans because they're looking forward to doing something together. Your current response may irritate the other person."

[0177] S405, the cloud 200 sends a reply to the terminal 100B.

[0178] In some embodiments, the cloud 200 can generate responses to the user's pre-sent content and existing chat content based on relevant algorithms or rules, and the generated responses take into account emotional judgment and factors that avoid causing resentment to the other party. After the response content is generated, it can be processed according to a certain format to obtain formatted transmission data and sent to terminal 100B.

[0179] In some implementations, the generated reply and other related information may be combined into a JSON object. For example, a JSON object may include the specific content of the reply and a timestamp.

[0180] The following is a pseudo code for sending data:

[0181] #reply

[0182] "Dialogue reply": "Then let's go play together!"

[0183] "Timestamp": 2023-10-31T08:51:26

[0184] "Reason for reply": "The other party asked about your evening plans because he or she is looking forward to doing something with you. Your current reply may make the other person feel bad."

[0185] S406: Terminal 100B displays a reply.

[0186] In some embodiments, terminal 100B displays the received reply, and the displayed reply includes the dialogue reply and the reason for the reply.

[0187] In some implementations, the terminal 100B parses the received JSON object and extracts the required fields, such as the conversation reply, timestamp, and reply reason in the JSON object, and then displays them on the interface to provide corresponding feedback to the user, thereby continuing the conversation.

[0188] It can be understood that through this process, the cloud 200 can efficiently transmit the reply and related information to the terminal 100B, so that the application or platform on the terminal 100B can flexibly process and present the response results.

[0189] Understandably, during a conversation between people, responses are provided to both parties in real time based on the actual content sent and the content that was previously sent. At the same time, with the user's explicit consent, user behavior and observed psychological indicators such as facial expressions, tone, and body temperature are utilized to provide more accurate and smooth responses, thereby making the conversation more fluid and driving the conversation in a positive direction.

[0190] S407, the cloud 200 identifies the search intent.

[0191] In some embodiments, after determining that the search requirements are met, for example, the search requirements are to book a flight or watch a movie, the cloud 200 performs intent recognition based on the conversation scenario data through a large model to obtain a search intent that can be used for retrieval.

[0192] For example, referring to the conversation scenarios in FIG. 2B and FIG. 5A , the cloud 200 generates search intentions related to watching a movie and booking a flight ticket, respectively, based on the conversation scenario data.

[0193] S408, the cloud 200 generates search information according to the search intent.

[0194] In some embodiments, the cloud 200 can generate search information, such as query parameters, to facilitate search as a search engine or local search based on the search intent. It is understood that since the data resource library available for search includes the terminal's data resource library and the cloud's data resource library, the generated search information includes local search information or cloud search information. For example, the local search information is the local query parameter, and the cloud search information is the cloud query parameter.

[0195] It is understandable that the format of the retrieved information can be JSON format or other formats, which is not required here.

[0196] It can be understood that the local search information may include keywords related to the search intent and search sentences related to the search intent. The specific parameters involved include: query intent, user input (the user input is a search sentence generated by the cloud 200 that can be directly used for local search), the object of the query, and the context information required for the query, such as user identity, terminal device platform and application, etc.

[0197] For example, the following is a pseudo code for retrieving information of a corresponding terminal:

[0198] # "intent": "book a flight";

[0199] "User input": "I want to book a flight from Beijing to Sanya"

[0200] "Query Object": "Air Ticket"

[0201] "Query context information": {"User identity": {"ID":"user123","Name":"Zhang San"}, "Device information": {"platform":"Android","app":"Flight query"}}

[0202] It can be understood that the above user input is the search sentence used for searching in the terminal 100B′.

[0203] For example, the following is a pseudo code for retrieving information from the cloud:

[0204] # "intent": "book a flight";

[0205] "User input": "I want to book a flight from Beijing to Sanya"

[0206] "Third-Party Applications":

[0207] "Query context information": {"User identity": {"ID":"user123","Name":"Zhang San"}, "Device information": {"platform":"iOS","app": }}

[0208] It is understood that the user input is a search phrase used to search in the cloud 200, for example, the search engine 202. Since the cloud has third-party application services, such as third-party applications for travel or flight booking, searches can be performed on services deployed in the cloud in applications related to travel or flight booking.

[0209] It is understandable that since the corresponding search tools and the resources used for searching are different when searching locally on the terminal and searching on the cloud, the generated search information will also be different.

[0210] It is understandable that the following cloud search part S409 and the local search part S410-S412 can be executed in any of the two parts, and the execution order is not limited. That is, step S409 and steps S410-S412 can be optional steps respectively.

[0211] S409, the cloud 200 obtains a cloud search result based on the cloud search information.

[0212] In some embodiments, the cloud 200 searches the cloud resource database based on the cloud search information to obtain search results. Specifically, the cloud 200 parses and processes the cloud search information using natural language processing (NLP) technology to generate a parsed query statement for use in the query to obtain cloud search results. The cloud search results include information such as a list of relevant search results, the total number of results, and the search engine's response time.

[0213] S410, the cloud 200 sends local search information to the terminal 100B.

[0214] In some embodiments, the cloud 200 may send local search information to the terminal 100B via a network protocol (such as HTTP, WebSocket, etc.).

[0215] S411, terminal 100B obtains local search results based on local search information.

[0216] In some embodiments, terminal 100B searches the local resource database based on the local search information to obtain local search results. The local resource database may include local applications and local data. Local applications are various software programs installed on a device and are typically stored in the device's local storage space. Local data, including data related to the device itself, such as device settings, user preferences, contacts, text messages, call logs, photos, videos, etc., is typically stored in the device's local storage space.

[0217] In some implementations, the terminal 100B uses a database query language (such as SQL, NoSQL query, etc.) or a local resource search algorithm based on the local retrieval information to search and match in the local resource database, and then obtains the local retrieval results.

[0218] For example, in conjunction with FIG5B , for the dialogue scenario of taking a flight shown in FIG5A , the terminal 100B searches for information related to air tickets from Beijing to Sanya in the local application and the local data according to the local search information.

[0219] S412 , the terminal 100B sends the local search result to the cloud 200 .

[0220] S413, the cloud 200 generates a reply based on the local and / or cloud search results and the conversation scene data, and the reply includes the fusion search results.

[0221] In some embodiments, the cloud 200 generates a response based on the local search results and / or cloud search results, as well as the conversation scenario data using a large model. The generated response includes a fused search result corresponding to the search requirements.

[0222] For example, for the travel scenario shown in FIG5A , the fused search results may include: “The search system is searching for travel information and air ticket information for September 13th, including a link for a flight from 2:30 PM to 6:25 PM, priced at ￥1,310, and a link for a flight from 6:30 AM to 10:20 AM, priced at ￥1,680.” and “Great! I’m really looking forward to this trip!”

[0223] In some implementations, the cloud 200 parses the return body corresponding to the cloud search result and the information sent by the terminal 100B based on the return body obtained from the cloud search, for example, the return body from the cloud search engine and the information sent from the terminal 100B, constructs the input parameters of the large model based on the parsed content, and generates a reply based on the input parameters.

[0224] As you can understand, the specific process of cloud 200 generating a reply is as follows:

[0225] (1) The return body corresponding to the cloud search results includes: the search result list, the total number of results and the response time of the search engine.

[0226] Here is a pseudo code example of what a search engine might return:

[0227] {“Flight Number”: “CAxxx”; “Departure City”: “Beijing”; “Departure Time”: “2023-xx-xx10:00”; “Arrival City”: “Sanya”; “Arrival Time”: “2023-xx-xx13:00”; “Airline”: “xxx”; “Price”: “1000 Yuan”}, {“Flight Number”: “HUxxx”; “Departure City”: “Beijing”; “Departure Time”: “2023-xx-xx14:00”; “Arrival City”: “Sanya”; “Arrival Time”: “2023-xx-xx17:00”; “Airline”: “xxx”; “Price”: “1200 Yuan”}; “Total Quantity”: 2; “Response Time”: “200”.

[0228] (2) Parsing the return body corresponding to the cloud search results includes: parsing the return body corresponding to the cloud search results, extracting the search result list and other related information, such as the total number of results.

[0229] (3) The input parameters for constructing a large model based on the parsed content include: constructing the input parameters of the large model based on the return body corresponding to the cloud search results, the input parameters include the search result list, the total number of results and other necessary fields, among which the specific content and form of the input parameters can be adjusted according to the specific large model and application scenario.

[0230] Here is a pseudo code example of the input parameters of a large model:

[0231] Input parameters {

[0232] Search results: {"flight number":"CAxxx";"departure city":"Beijing";"departure time":"2023-xx-xx10:00";"arrival city":"Sanya";"arrival time":"2023-xx-xx13:00";"airline":"xxx";"price":"1000 yuan"}, {"flight number":"HUxxx";"departure city":"Beijing";"departure time":"2023-xx-xx14:00";"arrival city":"Sanya";"arrival time":"2023-xx-xx17:00";"airline":"xxx";"price":"1200 yuan"};"total quantity":2;"response time":"200"}

[0233] Dialogue scene data: {......}.

[0234] (4) Generating a response based on input parameters includes: the large model generates a recommended response based on the constructed input parameters. It is understood that the large model can generate relevant recommended response content based on the search results, the total number of results, and other contextual information, combined with its own model weights and trained knowledge. It is understood that the recommended response content can be a specific answer to the user's query, suggestions, relevant resource links, or other information, customized according to the specific application scenario.

[0235] Pseudocode of the output of the example large model:

[0236] Output {

[0237] "Suggestion type": "Flight booking", "Suggestion content": "Based on your query, I found the following flight information for you:",

[0238] “Specific suggestions”: {“flight number”: “CAxxx”; “departure city”: “Beijing”; “departure time”: “2023-xx-xx10:00”; “arrival city”: “Sanya”; “arrival time”: “2023-xx-xx13:00”; “airline”: “xxx”; “price”: “1000 yuan”; “booking interface link”: “xxxxx”}, {“flight number”: “HUxxx”; “departure city”: “Beijing”; “departure time”: “2023-xx-xx14:00”; “arrival city”: “Sanya”; “arrival time”: “2023-xx-xx17:00”; “airline”: “xxx”; “price”: “1200 yuan”; “booking interface link”: “xxxxx”}

[0239] Parameter Description: "Flight Number": The unique identifier of the flight; "Departure City": The name or code of the departure city; "Departure Time": The departure time; "Arrival City": The name or code of the arrival city; "Airline": The name or code of the operating airline; "Price": The price of the ticket. "Booking Interface Link": A link to the flight booking app. This link directly opens the corresponding page in the app for booking tickets. The booking interface link is a link provided to the client-side to display the relevant booking interface.

[0240] It is understandable that when generating a generative model in a large model, a template filling method can be used to fill in some information. For example, a pre-defined template structure can be defined and filled in the corresponding positions to generate human-readable replies and suggestions.

[0241] It can be understood that the generative model in the large model can be integrated based on the relevant search results (i.e. search results) obtained from the local machine (i.e. the terminal side) and the cloud as well as the original conversation scenario data, that is, the search results from multiple different sources can be integrated together, so that the generated replies can not only include and display the content corresponding to the precise needs to the user, but also provide users with high-quality reply content, for example, displaying recommendations related to the App.

[0242] S414, the cloud 200 sends a reply to the terminal 100B.

[0243] S415, terminal 100B displays a reply.

[0244] It is understandable that the terminal 100B receives the reply and displays the reply. It is understandable that the terminal 100B can use the terminal side or cloud side resources and capabilities to provide the user with a comprehensive and accurate reply.

[0245] It is understandable that the execution order of the above steps S401 to S415 is only an example. In other embodiments, other execution orders may be adopted, and some steps may be split or combined, which is not limited here.

[0246] It is understood that both terminal 100A and terminal 100B are devices that display smart replies in real time. For the travel conversation scenario shown in Figure 5A above, after the user conversation is displayed on each terminal device, the terminal device can display smart replies. For example, the conversation on terminal 100B is further elaborated below. When user B asks "Are you going to Dali?" as shown by user B-2 in the figure, terminal device 100B can call smart conversation service 102B. At this time, smart conversation service 102B can understand user KB's preference for travel destinations and then provide travel information about Dali, such as attraction introductions and food recommendations, based on local and cloud data resources, to provide user KB with reference. When user A sends "How about going to Hainan?" as shown by user A-2 in the figure, the destination can be determined to be Hainan. Smart conversation service 102B can continue the conversation with user KA to learn more about itinerary plans, activity arrangements, orders, etc., to provide more personalized suggestions and services. When user KB agrees and asks, "When should we go?" as shown by user B-2, intelligent conversation service 102B can leverage the conversation context to infer the user's interest in departure time. Based on this information, intelligent conversation service 102B can invoke cloud services, such as a flight query interface, to provide the user with relevant information such as departure date, flight time, and price. As will be appreciated, Figure 5A illustrates the data flow that intelligent conversation service 102B retrieves from the cloud when a search is required, and further details will not be provided here.

[0247] Similarly, when user KA expresses a desire to travel, intelligent conversation service 102A recognizes the user's intent and further leverages conversation understanding technology and intent recognition algorithms to retrieve relevant travel information from the user's local device and the cloud. By integrating travel planning tools, flight search, and hotel reservations, intelligent conversation service 102A can provide user KA with travel recommendations, such as recommended itineraries, scenic spots, and destinations suitable for Hainan.

[0248] It is understandable that the interactive method proposed in the embodiments of this application can monitor conversations in real time, thereby avoiding the traditional practice of responding based on user input. Furthermore, by monitoring multiple inputs from the conversation partner in real time, including existing conversation content, pre-sent content, and factors such as the interlocutor's physiological indicators and behavior, comprehensive analysis and processing of multiple inputs, combined with previous conversation history information, can more comprehensively understand the needs and intentions of the conversation participants and perform comprehensive reasoning in conjunction with a large model. This enables intelligent conversation services to provide ideal responses to conversation participants, creating a more ideal and harmonious conversation experience.

[0249] FIG6 shows a hardware module diagram of a terminal 100 and a cloud 200 according to some embodiments of the present application.

[0250] As shown in FIG6 , the terminal 100 includes a first display module 501 , a first input module 502 , a first storage module 503 , a first calculation module 504 , and a first communication module 505 .

[0251] The cloud 200 includes a second storage module 601 , a second computing module 602 , and a second communication module 603 .

[0252] Specifically, for terminal 100:

[0253] The first display module 501 is used to output and display the interface. In some embodiments of the present application, the first display module 501 can display the responses generated by the intelligent dialogue service to the user in a visual form, which can be in the form of data such as text, images, videos, audio, and links. It can be understood that through the first display module 501, the user can intuitively view and understand the responses provided by the intelligent dialogue service.

[0254] The first input module 502 is used for user interaction with the dialogue system. The first input module 502 can be a keyboard, touch screen, or voice recognition device. Through the first input module 502, users can ask questions, express needs, or provide relevant information to the intelligent dialogue service. Users can communicate with the dialogue system by inputting text on the keyboard, using touch screen taps or gestures, or using voice input.

[0255] The first storage module 503 is used to store various data and information required by the dialogue system, including parameters related to the intelligent dialogue service, local content, information about third-party applications on the local machine, local search results, etc. This data and information can be repeatedly called and referenced by the dialogue system or intelligent dialogue service for analysis, response generation, search, etc., thereby providing accurate and useful responses.

[0256] The first computing module 504 is used to process, analyze and calculate input data and information. It can be a central processing unit, including components such as a processor and memory, and is also responsible for processing tasks such as local search.

[0257] The first communication module 505 is used to connect and communicate with other devices or networks, and can support wireless or wired communication. Through the first communication module 505, the terminal 100 can connect to the Internet to access search capabilities, obtain the latest information and data, etc. The first communication module 505 can also transmit and interact with other devices or systems to achieve a wider range of functions and services.

[0258] For Cloud200:

[0259] The second storage module 601 is used to save and manage large amounts of data, models and other related information, which may include historical search data, user profiles, system profiles, model parameters, training data sets, etc. It can also be used for long-term storage and backup of data, as well as support the training, updating and deployment of large models.

[0260] The second computing module 602 is the core computing resource of the cloud 200 and is typically composed of a group of high-performance computers or server clusters equipped with powerful processors and graphics processing units. These computing resources are used to perform large-scale complex computing tasks, such as training machine learning models, executing and inferring deep learning algorithms, and large-scale data processing. The second computing module 602 can provide higher computing power and parallelism, enabling the system to more efficiently perform advanced computing tasks such as intent recognition and recommended replies. In addition, the second computing module 602 can use its powerful computing power and large-scale trained models to conduct more in-depth analysis and processing of user questions, generating more advanced, accurate, and personalized recommended replies.

[0261] The second communication module 603 is responsible for remote communication and data transmission with the terminal device. The second communication module 603 connects to the terminal device via the internet to receive queries and requests from the terminal device in real time. Once a query or request is received, the communication device transmits it to the cloud computing device for processing and returns the processed results to the terminal device. The communication device is also responsible for maintaining the connection with the terminal device to ensure stable and secure communication.

[0262] The following, combined with Figures 6 and 7, briefly describes a data flow for generating responses by the terminal 100 interacting with the cloud 200 in some embodiments. Figure 6 illustrates the data flow when no retrieval is required based on the conversation scenario data; Figure 7 illustrates the data flow when retrieval is required based on the conversation scenario data.

[0263] Specifically, as shown in FIG6 , the first input module 502 receives user input and obtains input information. For example, when a user interacts with the terminal device 100B using a keyboard, touch screen, or voice recognition device, the first input module 502 receives input information such as text, voice, or video. The first storage module 503 stores the input information and the data collected by other acquisition modules, such as user behavior information and physiological indicator information, to obtain current conversation scene data and send the current conversation scene data to the first computing module 504 for processing. The first computing module 504 may perform preliminary processing on the conversation scene data to obtain processed conversation scene data, and then send the processed conversation scene data to the first communication module 505. The first communication module 505 sends a request to the second communication module 603 in the cloud, including the conversation scene data. The second communication module 603 sends the conversation scene data to the second calculation module 602. After determining that no retrieval is required, the second calculation module 602 directly generates a reply, and then sends the generated reply to the second communication module 603. The second communication module 603 sends it to the first communication module 505. The first communication module 505 sends the reply to the first display module 501 under the control of the first calculation module 504, and the first display module 501 displays the reply.

[0264] As shown in Figure 7, when the second computing module 602 determines that a search is needed, it will generate a search intent. At this time, local search information and cloud search information can be generated, so that searches can be performed on the terminal and the cloud respectively. The second computing module 602 obtains a second search result from the second storage module 601 based on the cloud search information. The second computing module 602 sends the local search information to the first computing module 504 through the second communication module 603 and the first communication module 505. The first computing module 504 searches from the first storage module based on the local search information to obtain a local search result. It can be understood that the results of the local search are mainly based on existing data and information, and can provide some quick and simple answers.

[0265] Then it is sent to the second calculation module 602 through the first communication module 505 and the second communication module 603. The second calculation module 602 integrates the local search results and the cloud search results to generate a reply.

[0266] It is understandable that the interaction method provided in the embodiment of the present application can select corresponding algorithms and technical tools according to specific needs and technical platforms during actual implementation. At the same time, privacy and data security should also be properly handled to protect the user's personal information. In actual applications, more complex technical designs and improvements may be required. For example, the first computing module 504 on the terminal side can also be used to fuse the search results with the model response obtained by the large model based on the dialogue scenario data to obtain a response. The specific implementation method is not required here.

[0267] FIG8 shows a structural diagram of an apparatus 800 according to some embodiments of the present application. It is understandable that the apparatus 800 can be a terminal device or a cloud device. The apparatus 800 may include one or more processors 801, which may also be referred to as a processing unit, and may implement certain control functions. The processor 801 may be a general-purpose processor or a dedicated processor, etc. For example, it may be a baseband processor or a central processing unit. The baseband processor may be used to process communication protocols and communication data, and the central processing unit may be used to control a communication device, such as a base station, a baseband chip, a terminal, a terminal chip, a DU or a CU, etc., to execute software programs and process data of the software programs.

[0268] In an optional design, the processor 801 may also store instructions and / or data 803, and the instructions and / or data 803 can be executed by the processor so that the device 800 executes the interaction method described in the above method embodiment.

[0269] In another alternative design, processor 801 may include a transceiver unit for implementing receiving and transmitting functions. For example, the transceiver unit may be a transceiver circuit, an interface, or an interface circuit. The transceiver circuit, interface, or interface circuit for implementing the receiving and transmitting functions may be separate or integrated. The transceiver circuit, interface, or interface circuit may be used for reading and writing code / data, or may be used for transmitting or delivering signals.

[0270] In another possible design, the apparatus 800 may include a circuit that can implement the functions of the interactive method in the aforementioned method embodiment.

[0271] Optionally, the device 800 may include one or more memories 802, on which instructions / data 804 may be stored. The instructions may be executed on a processor, causing the device 800 to perform the method described in the above method embodiment. Optionally, the memory may also store data. Optionally, the processor may also store instructions and / or data. The processor and memory may be provided separately or integrated. For example, the corresponding relationship described in the above method embodiment may be stored in the memory or in the processor.

[0272] Optionally, the apparatus 800 may further include a transceiver 805 and / or an antenna 806. The processor 801 may be referred to as a processing unit, and controls the apparatus 800. The transceiver 805 may be referred to as a transceiver unit, a transceiver, a transceiver circuit, a transceiver device, an interface, an interface circuit, or a transceiver module, and is configured to implement transceiver functions.

[0273] Optionally, the device 800 in the embodiment of the present application can be used to execute the interaction method described in Figures 3 and 4B in the embodiment of the present application.

[0274] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, enables the computer to implement the steps performed by the device 800 in any one of the above embodiments.

[0275] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable medium, which stores program code. When the program code runs on a computer, the computer implements the steps performed by the device 800 in any of the above embodiments.

[0276] The various embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0277] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0278] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0279] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy disks, optical disks, optical discs, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0280] In the accompanying drawings, some structural or method features are shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0281] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0282] It should be noted that in the examples and description of this patent, relational terms such as first and second are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. Without further restriction, an element defined by the phrase "comprising a" does not exclude the presence of other identical elements in the process, method, article or device comprising the element. Although the present application has been illustrated and described with reference to certain preferred embodiments of the present application, it should be understood by those skilled in the art that various changes can be made thereto in form and detail without departing from the scope of the present application.

Claims

1. An interaction method, applied to an electronic device, comprising: Display the first user conversation; If the first user conversation is a user conversation requiring retrieval, displaying a first reply related to a retrieval result corresponding to the first user conversation, the retrieval result being retrieved based on the retrieval intent of the first user conversation; If the first user conversation is a user conversation that does not need to be retrieved, a second reply to the first user conversation is displayed.

2. The method according to claim 1, characterized in that Also includes: The first reply or the second reply is displayed in a card or list format, and the first reply or the second reply may include at least one data form of text, image, voice, video, and link.

3. The method according to claim 1 or 2, characterized in that Also includes: Acquiring conversation scene data related to the first user conversation, the conversation scene data including at least one of the conversation content of the first user conversation, user behavior information regarding the first user conversation, and physiological indicators of the user during the first user conversation; Invoking a large language model, and the large language model determining, based on the conversation scene data, whether the first user conversation is a user conversation that needs to be retrieved; In the case where the first user dialogue is a user dialogue that needs to be retrieved, the large language model identifies the retrieval intention of the first user dialogue based on the dialogue scene data.

4. The method according to claim 3, characterized in that The first reply includes a fused search result corresponding to the search result.

5. The method according to claim 4, characterized in that The fusion search results are obtained by: The large language model generates the fused retrieval result based on the input retrieval result and the conversation scene data of the first user conversation.

6. The method according to claim 3, characterized in that The search results are obtained by: Searching a resource database according to search information corresponding to the search intent to obtain the search results, wherein the search information includes at least one of a keyword corresponding to the search intent and a search sentence corresponding to the search intent; The resource database includes a local resource database or a cloud resource database. Furthermore, the local resource database includes local data or local applications; The cloud resource database includes: applications or services deployed in the cloud.

7. The method according to claim 3, characterized in that Also includes: In the case where the first user dialogue is a user dialogue that needs to be retrieved, the large language model generates the second reply based on the dialogue scene data.

8. The method according to claim 1, characterized in that The second reply includes a dialogue reply and a reply reason corresponding to the dialogue reply.

9. An interaction method, applied to an electronic device, comprising: Display the first user conversation; Based on the conversation scene data of the first user conversation, a first reply to the first user conversation is displayed, wherein the conversation scene data includes at least one of the user's behavior information regarding the first user conversation and the user's physiological indicators during the first user conversation.

10. The method according to claim 9, characterized in that The first response includes at least one of the following: The retrieval results corresponding to the conversation with the first user, the fused retrieval results related to the retrieval results, the conversation reply, and the reply reason corresponding to the conversation reply.

11. The method according to claim 10, characterized in that Also includes: Invoking a large language model, and the large language model determining, based on the conversation scene data, whether the first user conversation is a user conversation that needs to be retrieved; In the case where the first user conversation is a user conversation requiring retrieval, displaying the retrieval result or the fused retrieval result; If the first user conversation is a user conversation that does not require retrieval, the conversation reply and the reply reason corresponding to the conversation reply are displayed.

12. The method according to claim 11, characterized in that The fusion search results are obtained in the following way: The large language model identifies the search intent of the first user's conversation based on the conversation scenario data; The large language model searches a resource database based on search information corresponding to the search intent to obtain the search results, wherein the search information includes at least one of a keyword corresponding to the search intent and a search sentence corresponding to the search intent; The large language model generates the fused retrieval result based on the input retrieval result and the conversation scene data of the first user conversation.

13. The method according to claim 11, characterized in that The dialogue response and the response reason corresponding to the dialogue response are obtained in the following manner: In the case where the first user dialogue is a user dialogue that needs to be retrieved, the large language model generates the dialogue response and the response reason corresponding to the dialogue response based on the dialogue scene data.

14. An interaction method, applied to an electronic device, comprising: Display the first user conversation; A first reply corresponding to the first user conversation is displayed, where the first reply includes a conversation reply and a reply reason corresponding to the conversation reply.

15. The method according to claim 14, characterized in that Also includes: Acquiring conversation scene data related to the first user conversation, the conversation scene data including at least one of the conversation content of the first user conversation, user behavior information regarding the first user conversation, and physiological indicators of the user during the first user conversation; A large language model is called, and the large language model generates the first reply according to the dialogue scenario data.

16. The method according to claim 15, characterized in that Also includes: The large language model determines, based on the conversation scene data, whether the first user conversation is a user conversation that needs to be retrieved; If the first user conversation is a user conversation that does not require retrieval, the first reply is generated.

17. An interaction method, applied to an electronic device, comprising: displaying a first user conversation, where the first user conversation is the user conversation that needs to be retrieved; A first reply related to the search result corresponding to the first user conversation is displayed, where the first reply includes a fused search result corresponding to the search result, wherein the search result is retrieved according to the search intention of the first user conversation.

18. The method according to claim 17, characterized in that Also includes: Acquiring conversation scene data related to the first user conversation, the conversation scene data including at least one of the conversation content of the first user conversation, user behavior information regarding the first user conversation, and physiological indicators of the user during the first user conversation; A large language model is called, and the large language model generates the first reply according to the conversation scenario data and the retrieval result.

19. The method according to claim 18, characterized in that Also includes: The large language model determines, based on the conversation scene data, whether the first user conversation is a user conversation that needs to be retrieved; The large language model generates the first reply when the first user dialogue is a user dialogue that needs to be retrieved.

20. An electronic device, characterized in that: The device comprises: a memory for storing instructions; A processor, configured to execute the instructions to implement the interaction method according to any one of claims 1 to 19.

21. A readable storage medium, characterized in that The readable medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the interaction method according to any one of claims 1 to 19.

22. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is run on a computer, the computer is caused to execute the interactive method according to any one of claims 1 to 19.

Images