Interactive canvas content generation method and device based on large model

Abstract

The invention discloses an interactive canvas content generation method and device based on a large model. The method comprises the steps of receiving a target question input by a user through an interactive canvas, wherein the interactive canvas comprises historical interaction data displayed in a card form; determining at least one to-be-processed target task according to the target problem; determining associated information of the target task, wherein the associated information comprises at least one of the following information: an associated card determined from the cards and reference information acquired from a preset information source; generating a target answer of the target question by using a large model according to the associated information of the target task; and displaying a target card generated based on the target question and the target answer on the interactive canvas, and displaying an association relationship between the target card and the association card when the association information comprises the association card.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent question answering, and more specifically, to a method and apparatus for generating interactive canvas content based on a large model. Background Technology

[0002] Currently, Large Language Models (LLMs) are rapidly developing, and their capabilities in natural language understanding, content generation, and dialogue are widely recognized and increasingly applied to more complex business scenarios. In some complex business scenarios, long-running, multi-turn dialogues are often involved, which are displayed as lengthy linear dialogue flows in the dialogue interface. As the dialogue extends, users find it difficult to quickly grasp the overall information within the limited dialogue display window. This is especially true when users need to summarize or follow up on specific past issues; excessively long dialogue chains require users to repeatedly search for context, resulting in significant inefficiency. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, this disclosure provides a method for generating interactive canvas content based on a large model, the method comprising: Receive the target question input by the user through an interactive canvas, which includes historical interaction data displayed in card format; Based on the target problem, at least one target task to be processed is identified; Determine the associated information of the target task, wherein the associated information includes at least one of the following: associated cards determined from the cards, and reference information obtained from a preset information source; Based on the association information of the target task, the target answer to the target question is generated using a large model; The interactive canvas displays target cards generated based on the target question and the target answer, and displays the association relationship between the target card and the associated card when the association information includes the associated card.

[0005] Secondly, this disclosure provides an interactive canvas content generation device based on a large model, the device comprising: A receiving module is used to receive the target question input by the user through an interactive canvas, wherein the interactive canvas includes historical interaction data displayed in the form of cards; The first determining module is used to determine at least one target task to be processed based on the target problem; The second determining module is used to determine the association information of the target task, wherein the association information includes at least one of the following: the associated card determined from the card, and the reference information obtained from the preset information source; The first generation module is used to generate the target answer to the target question using a large model based on the association information of the target task. The first display module is used to display target cards generated based on the target question and the target answer on the interactive canvas, and to display the association relationship between the target card and the associated card when the association information includes the associated card.

[0006] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect of this disclosure.

[0007] Fourthly, this disclosure provides an electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect of this disclosure.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect of this disclosure.

[0009] Through the above technical solution, after receiving the user's target question, at least one target task to be processed is first determined based on the target question. This enables automatic parsing and decomposition of the user's input target question, facilitating targeted responses to different intentions in complex questions, thereby effectively improving the accuracy and efficiency of handling complex problems. Next, associated information is determined for each target task. This associated information includes at least one of the associated cards determined from the interactive canvas or reference information obtained from preset information officers. This provides rich relevant context for each target task, offering a robust data foundation for subsequent answer generation. Then, a large model is used to generate the target answer based on the target task and its associated information. The interactive canvas then displays the target cards generated based on the target question and answer, as well as the relationships between the associated cards and the target cards. This allows for accurate and coherent answers to be obtained based on the associated information and the large model. The target cards display the target question and answer, along with the relationships between the target cards and the associated cards. Furthermore, the cards not only visually present the target question and answer but also identify the source and basis of the target answer through the relationships, providing users with a visual traceability path and improving the ease of access. Therefore, by transforming the traditional linear display and confusing dialogue interface into a visual interactive canvas, which displays historical interaction data through cards and identifies the relationships between cards, the problems of information confusion and difficulty in tracing can be effectively solved, thereby improving the efficiency and visualization of intelligent question answering.

[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart of an interactive canvas content generation method based on a large model, according to one embodiment of the present disclosure; Figure 2 This is an exemplary interface diagram of an interactive canvas based on the large model-based interactive canvas content generation method provided in this disclosure. Figure 3 This is another exemplary interface diagram of an interactive canvas based on the large model-based interactive canvas content generation method provided in this disclosure. Figure 4 This is a block diagram of an interactive canvas content generation apparatus based on a large model, according to one embodiment of the present disclosure. Figure 5 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0015] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0018] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0019] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0020] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0022] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] Figure 1 This is a flowchart of an interactive canvas content generation method based on a large model, according to one embodiment of this disclosure. Figure 1 As shown, the method may include steps 11 to 15.

[0024] In step 11, the target question is received by the user through the interactive canvas.

[0025] Optionally, the interactive canvas may display a first interactive component for users to input questions. The first interactive component may include a text input box and a question confirmation button. The text input box allows users to enter questions, and when the user triggers the question confirmation button, the real-time content in the text input box is uploaded as the question. For example, the first interactive component may be as follows: Figure 2 As shown in component K, it includes a text input box K1 and a question confirmation button K2.

[0026] The interactive canvas can include historical interaction data displayed in card format. Initially, the interactive canvas can be empty when no interactions have been generated. Optionally, each card can include a title and content, with the title derived from summarizing the content of the card (e.g., through a larger model summary).

[0027] Optionally, historical interaction data may include, but is not limited to, historical question and answer data, historical files, etc.

[0028] Historical question-and-answer data can be dialogue data from previous rounds. For example, a card can correspond to the dialogue data (including question and answer) of one round. Whenever a user enters a question and receives an answer, the question and answer will be displayed as a card on the interactive canvas.

[0029] Historical files can be files uploaded by users during past conversations. File types can include, but are not limited to, text files (e.g., doc, docx, txt, pdf) and image files (e.g., jpg, png). For example, one card can correspond to one file; each time a user uploads a file, that file can be displayed as a card on an interactive canvas.

[0030] Optionally, the interactive canvas may display a second interactive component for users to upload files. For example, the second interactive component may include a file upload button. When a user triggers the file upload button, a corresponding file upload window may be displayed. The file upload window may include a file upload area and an upload confirmation button. The file upload area allows users to upload text or image files by selecting or dragging files. When the user triggers the upload confirmation button, the file in the file upload area will be uploaded in real time. For example, there may be one text upload button, which can be triggered to upload different types of files. As another example, the text upload button may include upload buttons corresponding to different file types, with each upload button used to complete the upload of a file type, such as a first upload button for uploading text files and a second upload button for uploading image files. For example, the second interactive component may be as follows: Figure 2 As shown in J.

[0031] When a user uploads a file through the second interactive component, a card corresponding to the uploaded file can be generated, and the card can be displayed on the interactive canvas.

[0032] Optionally, when the uploaded file is a text file, the text file can be segmented into multiple fragments, and a vector corresponding to each fragment can be generated and stored in a preset vector library, and stored in association with the text file. For example, the above process can be accomplished using a large model.

[0033] Optionally, when the uploaded file is an image file, the descriptive information corresponding to the image file can be identified, and the descriptive information can be associated with and stored for storage. For example, a corresponding vector can be generated based on the descriptive information and stored in a preset vector library. For example, the descriptive information corresponding to the image file can be identified using a multimodal large model.

[0034] In this way, for images that cannot be directly recognized, a multimodal large model can be invoked to generate their descriptive information and associate it with image class cards, so that the image content can be quickly obtained based on the descriptive information when image understanding functions are not provided.

[0035] Optionally, before performing step 12, the method provided in this disclosure may further include the following steps: In response to receiving a target question, determine the question category of the target question.

[0036] The question categories can include a first question category representing chat and a second question category representing task processing. The first question category corresponds to questions raised by users during casual conversation, while the second question category corresponds to questions raised by users when they are not chatting and have task processing needs.

[0037] In one possible implementation, a judgment rule for determining the problem category can be preset, and the problem category of the target problem can be determined based on the judgment rule.

[0038] In another possible implementation, a large model can be used to identify the target problem and determine its category. A lightweight large model can be selected to determine the target problem category. For example, when classifying the target problem using a large model, a prefix caching technique can be introduced. By caching and reusing key-value caches with the same prompt prefix, redundant computations are reduced, thereby improving classification efficiency.

[0039] In another possible implementation, if the problem category of the target problem cannot be determined based on pre-set decision rules, the problem category of the target problem can be determined by using a large model.

[0040] After determining the problem category of the target problem, subsequent processing strategies can be determined based on the problem category.

[0041] Optionally, the step of determining at least one target task to be processed based on the target problem, namely step 12, can be executed when the target problem is classified as the second problem category.

[0042] Optionally, the method provided in this disclosure may also include: If the target question falls under the first question category, retrieve the background information associated with the user. Using the first preset model, response information corresponding to the target question is generated based on background information. The first preset model is a large model. Generate cards corresponding to the target question and answer information and display them on an interactive canvas.

[0043] If the target question falls under the first question category, which is more conversational, the model can be directly invoked to generate a response, enabling a rapid response to the user's question.

[0044] Optionally, the background information may include, but is not limited to, the user's corresponding persona information, the persona information requested by the user, and the model service configuration information.

[0045] Background information can be used as part of the prompt words and input into the first preset model along with the target question. Based on the powerful text processing and reasoning capabilities of the large model, the answer information corresponding to the target question can be obtained quickly.

[0046] After receiving the answer to the target question, this round of question and answer is complete. Cards can be generated based on the target question and the answer, and displayed through an interactive canvas.

[0047] Therefore, in casual conversation scenarios where users ask simple questions, it can quickly respond to users' questions and improve the user's real-time interactive experience.

[0048] In step 12, at least one target task to be processed is determined based on the target problem.

[0049] Optionally, the target problem may contain a single task. For example, the target problem may be "Please introduce plant A1", in which case the target problem only contains the single task of introducing plant A1.

[0050] Optionally, the target problem may contain two or more tasks. For example, the target problem may be "Please introduce plant A1 and then summarize the care of plant A2". In this case, the target problem contains two tasks: introducing plant A1 and summarizing the care of plant A2.

[0051] In one possible implementation, a large model can be used to break down the target problem into tasks to identify one or more target tasks within the target problem. For example, a prompt word template for text-based task recognition can be pre-set. By adding the target problem to the corresponding position in the prompt word template, a complete prompt word is generated and input into the large model to obtain the target task output by the large model.

[0052] In another possible implementation, step 12 may include the following steps: Perform intent identification on the target question to determine at least one question intent; For each problem intent, determine the target task corresponding to the problem intent.

[0053] Optionally, an intent recognition model (e.g., a large model) can be used to identify the intent of the target question to determine the question intent, which may be one or more. Then, based on the question intent, the target question can be broken down into sub-questions corresponding to each intent, thus forming the target task. For example, if the target question is "Please introduce plant A1 and then summarize the care of plant A2," intent recognition can identify two question intents: introducing plant A1 and summarizing the care of plant A2. Therefore, the target question can be broken down into two target tasks: introducing plant A1 and summarizing the care of plant A2.

[0054] In step 13, the associated information of the target task is determined. The associated information includes at least one of the following: associated cards identified from the cards, and reference information obtained from a preset information source.

[0055] For the target tasks identified in step 12, the associated information for each target task can be determined separately. Subsequent steps will focus on describing the processing procedure for each target task; the processing procedure for each target task can refer to this process.

[0056] In one possible implementation, associated cards can be identified from all cards displayed on the interactive canvas based on the target task. Optionally, associated cards can be identified based on semantic similarity. This involves determining the semantic similarity between each card and the target task, and identifying cards with semantic similarity greater than a preset threshold, or a preset number of cards with the highest semantic similarity ranking, as associated cards to add them to the association information of the target task. For example, the similarity between a card and the target task can be determined based on the text content corresponding to the card. Another example is the similarity between a card and the target task based on the vector corresponding to the card in a preset vector library (e.g., cards for text files, cards for image files, etc.). Optionally, associated cards can be identified directly from the cards on the interactive canvas based on a large model. For example, a prompt word template for finding associated cards can be pre-set. By adding the target task (e.g., the text corresponding to the target task) to the corresponding position in the prompt word template, a complete prompt word is generated and input into the large model to obtain the associated cards (or, no associated cards found) output by the large model.

[0057] In another possible implementation, reference information related to the target task can be obtained by searching within a preset information source. Optionally, the preset information source may include, but is not limited to, internet search engines and knowledge bases. For example, the knowledge base can be pre-built according to actual needs; for instance, whenever a user uploads a file via an interactive canvas, the uploaded file can be added to the knowledge base. For example, reference information in the preset information source can be obtained based on semantic similarity, such as when the semantic similarity exceeds a preset threshold. Optionally, reference information can be directly searched within the preset information source based on a large model. For example, a prompt word template for finding reference information can be pre-set. By adding the target task (e.g., the text corresponding to the target task) and the preset information source to the corresponding positions in the prompt word template, a complete prompt word is generated and input into the large model to obtain the reference information output by the large model.

[0058] In another possible implementation, an agent center can be used to determine the association information of the target task. The agent center may include multiple registered agents, each handling different tasks. Based on the target task (or, the target task and its associated cards), a target agent can be identified in the agent center according to pre-set matching rules, and the association information of the target task can be determined by invoking the target agent.

[0059] In another possible implementation, the target task may correspond to a task type, which may include a first type for generating answers to questions. For example, "Introduce plant A1" in the previous example.

[0060] In this embodiment, step 13 may include the following steps: For target tasks belonging to the first type, determine whether there are any related cards that are semantically related to the target task among the cards displayed on the interactive canvas; Obtain reference information related to the semantics of the target task by pre-setting information sources; If associated cards exist, add the associated cards and reference information to the associated information of the target task; If no associated card exists, add the reference information to the associated information of the target task.

[0061] For the first type of target task, it is possible to determine whether there are related cards semantically relevant to the target task in the cards of the interactive canvas, and to obtain reference information semantically relevant to the target task through a preset information source. If a related card can be found, both the related card and the reference information can be added to the target task's related information; if no related card can be found, only the reference information is added to the target task's related information. The implementation methods for determining related cards and determining reference information have been described above and will not be repeated here.

[0062] The task type corresponding to the target task can also include a second type used to generate a summary for the displayed cards. For example, "summarize the care of plant A2" in the previous example.

[0063] Accordingly, step 13 may include the following steps: For target tasks belonging to the second type, identify the associated cards that are semantically related to the target task from the cards displayed on the interactive canvas, and use them as the associated information of the target task.

[0064] For the second type of target task, since it is a summary task—that is, summarizing several cards in the target interactive canvas—the associated cards in the interactive canvas can be directly determined based on the target task as the association information of the target task. Optionally, the associated cards of the target task can be determined through semantic similarity; for specific implementation methods, please refer to the preceding text.

[0065] In one possible implementation, the cards in the interactive canvas can correspond to card types, which may include text, file, and image types. Specifically, text-type cards can be generated based on a user-input question and the interactive canvas's output answer to that question; file-type cards can be generated based on user-uploaded text files, as mentioned above; and image-type cards can be generated based on user-uploaded image files, as mentioned above.

[0066] In this embodiment, step 13 may include the following steps: If the interactive canvas includes a first card of the image type, obtain the first description information corresponding to the first card; Based on the first description information, determine whether there is a related card in the first card.

[0067] If the interactive canvas includes image-type cards (referred to as "first cards" for convenience), the first descriptive information corresponding to the first card can be obtained. This first descriptive information can be generated by a second preset model based on the original image corresponding to the first card, where the second preset model is a multimodal large model. As mentioned earlier, when a user uploads an image file, the descriptive information can be identified based on the multimodal large model and associated with the image file for storage. Therefore, for an image-type first card, its first descriptive information can be used to determine whether it is an associated card for the target task.

[0068] In one possible implementation, step 13 may include the following steps: In response to receiving a user's selection instruction for at least one card in the interactive canvas, the card indicated by the selection instruction is added to the associated cards.

[0069] Users can select any card on an interactive canvas to identify the cards they are interested in. The user's selection triggers a selection instruction. Therefore, based on the received user selection instruction, the card indicated by the instruction can be added to the associated cards. For example, a user can select several cards and enter "generate summary." The target problem is then identified as a second-type target task, with the associated information being the selected cards (associated cards). The subsequent step is to generate a summary for the user based on the content of the selected cards.

[0070] In one possible implementation, the cards in the interactive canvas can correspond to card types, including text, file, and image types. Accordingly, the method provided in this disclosure may further include the following steps: If a second card belonging to the image type is found among the identified associated cards, the third preset model is used to generate second descriptive information related to the target task based on the original image corresponding to the second card and the target task. The third preset model is a multimodal large model. Add the second descriptive information to the associated information.

[0071] If, after identifying the associated cards, there are second cards belonging to the image type (i.e., cards generated based on image files), a third pre-defined model (multimodal large model) can be used to generate second descriptive information related to the target task based on the original image corresponding to the second card and the target task. Since the original image contains rich information, while the target task may only focus on a portion of it, information highly relevant to the target task can be extracted from the original image based on the target task, making the extraction more targeted. In this way, based on the target task, the third pre-defined model can identify relevant content related to the target task in the original image to obtain second descriptive information. This second descriptive information will also serve as part of the target task's associated information, providing richer data support for subsequent answer generation.

[0072] It should be noted that in practical application scenarios, the corresponding target tasks for different target problems are diverse. Therefore, based on the aforementioned methods for determining the correlation between target tasks, any of the methods can be flexibly adopted according to actual needs to obtain the correlation information of the target tasks.

[0073] In step 14, the target answer to the target question is generated using a large model based on the associated information of the target task.

[0074] In one possible implementation, the target question, the associated information of each target task, and preset prompts can be combined to form the input text, which is then input into a large model to obtain the target answer output by the large model.

[0075] In another possible implementation, step 14 may include the following steps: Based on the target task, determine the target prompt word template; Based on the target prompt template and the association information of the target task, target prompts are generated; Input the target prompt word into the fourth preset model to obtain the target answer output by the fourth preset model. The fourth preset model is the large model.

[0076] The target prompt template allows you to set positions for the target task and related information. By filling these positions with the target task and its associated information, target prompts are generated. These prompts are then input into a fourth preset model, which leverages the powerful natural language understanding and reasoning capabilities of the large model to generate the target answer corresponding to the target question.

[0077] Optionally, prompt templates can be pre-set according to task type. For example, a first template can be set for the first type of task, and a second template can be set for the second type of task. For instance, the first template can include fields for the target task, associated cards, and reference information, while the second template can include fields for the target task and associated cards. Based on this, for each target task, a target prompt template can be determined according to the task type. Then, for each target task, prompts are generated based on the target prompt template and the associated information of the target task, and finally, these prompts are combined into a complete target prompt.

[0078] Optionally, prompt word templates can be pre-set based on the task types of the problem. For example, a third template can be set for problems containing only type 1 tasks, a fourth template for problems containing only type 2 tasks, and a fifth template for problems containing both type 1 and type 2 tasks. Based on this, target prompt word templates can be determined according to the task types of the target tasks contained in the target problem. For example, if the target task of the target problem contains only type 1, then the third template is used as the target prompt word template; if the target task of the target problem contains both type 1 and type 2, then the fifth template is used as the target prompt word template. After determining the target prompt word template, the target prompt words are obtained by adding the associated information of the target task to the corresponding positions in the target prompt word template.

[0079] After obtaining the target prompt, the target prompt is input into the fourth preset model. By utilizing the powerful natural language processing and reasoning capabilities of the large model, the target answer output by the fourth preset model is obtained.

[0080] In step 15, target cards generated based on the target question and target answer are displayed on an interactive canvas, and the relationship between the target cards and the associated cards is displayed when the associated information includes associated cards.

[0081] Once the target answer is determined, a target card can be generated based on the target question and the target answer.

[0082] The card generation method can be pre-set according to actual needs. For example, the card can be divided into a question area and an answer area, with the question area displaying the target question and the answer area displaying the target answer. Alternatively, the card can be divided into a title area and a body area, with the title area displaying a title generated based on the target answer and target question (e.g., obtained through a large model), and the body area displaying the target question and target answer.

[0083] When displaying the target card, if the associated information for the target question includes related cards, the relationship between the target card and the related cards can also be displayed.

[0084] Optionally, when the associated information includes associated cards, displaying the association relationship between the target card and the associated cards can be achieved in the following ways: Generate and display the connections between the target card and its associated cards.

[0085] For example, the association between the target card and each associated card can be represented by connecting them with lines.

[0086] Optionally, the attributes of the connection can be used to provide richer information about the relationship between the target card and related cards. For example, a correspondence can be set between the attributes of the connection and the meaning of the relationship between the target card and related cards to provide users with richer information through the connection. The attributes of the connection can include, but are not limited to: the direction of the connection, the solidity or dashedness of the connection, and the color of the connection.

[0087] For example, a line with an arrow pointing from the associated card to the target card can be used to indicate that the source of information for the target card is the associated card. Figure 3 As shown.

[0088] For example, the similarity between cards can be represented by solid and dashed lines. For instance, a reference threshold for similarity can be set. If the similarity (e.g., semantic similarity) between the target card and the associated card exceeds the reference threshold, the associated card and the target card are connected with a solid line. If the similarity between the target card and the associated card does not exceed the reference threshold, the associated card and the target card are connected with a dashed line.

[0089] Optionally, in the interactive canvas, users can drag any card. After a card is dragged, the lines will be redrawn based on the new position of the dragged card and its associated cards, according to the relationship.

[0090] Optionally, in the interactive canvas, users can zoom in on any card or any part of the canvas to achieve flexible viewing of any card.

[0091] Through the above technical solution, after receiving the user's target question, at least one target task to be processed is first determined based on the target question. This enables automatic parsing and decomposition of the user's input target question, facilitating targeted responses to different intentions in complex questions, thereby effectively improving the accuracy and efficiency of handling complex problems. Next, associated information is determined for each target task. This associated information includes at least one of the associated cards determined from the interactive canvas or reference information obtained from preset information officers. This provides rich relevant context for each target task, offering a robust data foundation for subsequent answer generation. Then, a large model is used to generate the target answer based on the target task and its associated information. The interactive canvas then displays the target cards generated based on the target question and answer, as well as the relationships between the associated cards and the target cards. This allows for accurate and coherent answers to be obtained based on the associated information and the large model. The target cards display the target question and answer, along with the relationships between the target cards and the associated cards. Furthermore, the cards not only visually present the target question and answer but also identify the source and basis of the target answer through the relationships, providing users with a visual traceability path and improving the ease of access. Therefore, by transforming the traditional linear display and confusing dialogue interface into a visual interactive canvas, which displays historical interaction data through cards and identifies the relationships between cards, the problems of information confusion and difficulty in tracing can be effectively solved, thereby improving the efficiency and visualization of intelligent question answering.

[0092] Figure 4 This is a block diagram of an interactive canvas content generation apparatus based on a large model, according to one embodiment of this disclosure. Figure 4 As shown, the device 40 may include: Receiving module 41 is used to receive the target question input by the user through an interactive canvas, wherein the interactive canvas includes historical interaction data displayed in the form of cards; The first determining module 42 is used to determine at least one target task to be processed based on the target problem; The second determining module 43 is used to determine the association information of the target task, the association information including at least one of the following: the association card determined from the card, and the reference information obtained from the preset information source; The first generation module 44 is used to generate the target answer to the target question using a large model based on the association information of the target task. The first display module 45 is used to display target cards generated based on the target question and the target answer on the interactive canvas, and to display the association relationship between the target card and the associated card when the association information includes the associated card.

[0093] Optionally, the device 40 further includes: The third determining module is configured to, in response to receiving the target question, determine the question category of the target question before the first determining module 42 determines at least one target task to be processed based on the target question, the question category including a first question category representing chat and a second question category representing task processing; The first determining module 42 is used to determine at least one target task to be processed based on the target problem when the problem category of the target problem is the second problem category.

[0094] Optionally, the device 40 further includes: The acquisition module is used to acquire background information associated with the user when the problem category of the target problem is the first problem category; The second generation module is used to generate response information corresponding to the target question based on the background information using the first preset model, wherein the first preset model is a large model. The second display module is used to generate cards corresponding to the target question and the response information and display them through the interactive canvas.

[0095] Optionally, the first determining module 42 includes: The first determining submodule is used to perform intent recognition on the target question and determine at least one question intent; The second determining submodule is used to determine the target task corresponding to each of the aforementioned problem intentions.

[0096] Optionally, the target task corresponds to a task type, and the task type includes a first type for generating answers to questions; The second determining module 43 includes: The third determining submodule is used to determine, for a target task belonging to the first type, whether there are any associated cards semantically related to the target task among the cards displayed on the interactive canvas; The first acquisition submodule is used to acquire reference information related to the semantics of the target task through a preset information source; The first adding submodule is used to add the associated card and the reference information to the association information of the target task if the associated card exists. The second adding submodule is used to add the reference information to the association information of the target task if the associated card does not exist.

[0097] Optionally, the target task corresponds to a task type, and the task type includes a second type for generating a summary for the displayed cards; The second determining module 43 includes: The fourth determination submodule is used to determine, for a target task belonging to the second type, an associated card semantically related to the target task from the cards displayed on the interactive canvas, as the associated information of the target task.

[0098] Optionally, the cards in the interactive canvas correspond to card types, including text type, file type, and image type; The second determining module 43 includes: The second acquisition submodule is used to acquire first description information corresponding to the first card if the interactive canvas includes a first card of image type. The first description information is generated by a second preset model based on the original image corresponding to the first card. The second preset model is a multimodal large model. The fifth determining submodule is used to determine whether the associated card exists in the first card based on the first description information.

[0099] Optionally, the second determining module 43 includes: The third addition submodule is used to add the card indicated by the selection instruction to the associated cards in response to receiving a user's selection instruction for at least one card in the interactive canvas.

[0100] Optionally, the cards in the interactive canvas correspond to card types, including text type, file type, and image type; The device 40 further includes: The third generation module is used to generate second descriptive information related to the target task if there is a second card of image type among the determined associated cards, using a third preset model, based on the original image corresponding to the second card and the target task. The third preset model is a multimodal large model. An add module is used to add the second description information to the associated information.

[0101] Optionally, the first generation module 44 includes: The sixth determining submodule is used to determine the target prompt word template based on the target task; The first generation submodule is used to generate target prompt words based on the target prompt word template and the association information of the target task; The second generation submodule is used to input the target prompt word into the fourth preset model to obtain the target answer output by the fourth preset model, wherein the fourth preset model is a large model.

[0102] Optionally, the first display module 45 includes: The third generation submodule is used to generate and display the connection between the target card and the associated card.

[0103] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0104] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device 600 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0105] like Figure 5As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0106] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0107] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0108] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0109] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0110] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0111] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: Receive the target question input by the user through an interactive canvas, which includes historical interaction data displayed in card format; Based on the target problem, at least one target task to be processed is identified; Determine the associated information of the target task, wherein the associated information includes at least one of the following: associated cards determined from the cards, and reference information obtained from a preset information source; Based on the association information of the target task, the target answer to the target question is generated using a large model; The interactive canvas displays target cards generated based on the target question and the target answer, and displays the association relationship between the target card and the associated card when the association information includes the associated card.

[0112] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0114] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not necessarily limiting in certain circumstances; for example, a receiving module can also be described as "a module that receives a target question input by the user through an interactive canvas".

[0115] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0116] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0117] According to one or more embodiments of this disclosure, a method for generating interactive canvas content based on a large model is provided, the method comprising: Receive the target question input by the user through an interactive canvas, which includes historical interaction data displayed in card format; Based on the target problem, at least one target task to be processed is identified; Determine the associated information of the target task, wherein the associated information includes at least one of the following: associated cards determined from the cards, and reference information obtained from a preset information source; Based on the association information of the target task, the target answer to the target question is generated using a large model; The interactive canvas displays target cards generated based on the target question and the target answer, and displays the association relationship between the target card and the associated card when the association information includes the associated card.

[0118] According to one or more embodiments of this disclosure, a method for generating interactive canvas content based on a large model is provided. Prior to the step of determining at least one target task to be processed based on the target problem, the method further includes: In response to receiving the target question, the question category of the target question is determined, the question category including a first question category representing chat and a second question category representing task processing; The step of determining at least one target task to be processed based on the target problem is performed when the problem category of the target problem is the second problem category.

[0119] According to one or more embodiments of this disclosure, a method for generating interactive canvas content based on a large model is provided, the method further comprising: If the problem category of the target problem is the first problem category, obtain the background information associated with the user; Using a first preset model, response information corresponding to the target question is generated based on the background information; the first preset model is a large model. Generate cards corresponding to the target question and the response information and display them through the interactive canvas.

[0120] According to one or more embodiments of this disclosure, an interactive canvas content generation method based on a large model is provided, wherein determining at least one target task to be processed based on the target problem includes: The target question is subjected to intent recognition to determine at least one question intent; For each of the stated problem intentions, determine the target task corresponding to the stated problem intention.

[0121] According to one or more embodiments of this disclosure, an interactive canvas content generation method based on a large model is provided, wherein the target task corresponds to a task type, and the task type includes a first type for generating answers to questions; The determination of the association information of the target task includes: For a target task belonging to the first type, determine whether there are any associated cards semantically related to the target task among the cards displayed on the interactive canvas; Obtain reference information related to the semantics of the target task by using a preset information source; If the associated card exists, add the associated card and the reference information to the association information of the target task; If the associated card does not exist, the reference information is added to the associated information of the target task.

[0122] According to one or more embodiments of this disclosure, an interactive canvas content generation method based on a large model is provided, wherein the target task corresponds to a task type, and the task type includes a second type for generating a summary for displayed cards; The determination of the association information of the target task includes: For a target task belonging to the second type, among the cards displayed on the interactive canvas, identify associated cards that are semantically related to the target task, as the associated information of the target task.

[0123] According to one or more embodiments of this disclosure, a method for generating interactive canvas content based on a large model is provided, wherein the cards in the interactive canvas correspond to card types, and the card types include text type, file type and image type; The determination of the association information of the target task includes: If the interactive canvas includes a first card of the image type, obtain the first description information corresponding to the first card. The first description information is generated by a second preset model based on the original image corresponding to the first card. The second preset model is a multimodal large model. Based on the first description information, determine whether the associated card exists in the first card.

[0124] According to one or more embodiments of this disclosure, an interactive canvas content generation method based on a large model is provided, which determines the association information of the target task, including: In response to receiving a user's selection instruction for at least one card in the interactive canvas, the card indicated by the selection instruction is added to the associated cards.

[0125] According to one or more embodiments of this disclosure, a method for generating interactive canvas content based on a large model is provided, wherein the cards in the interactive canvas correspond to card types, and the card types include text type, file type and image type; The method further includes: If a second card of image type exists among the identified associated cards, a second descriptive information related to the target task is generated using a third preset model based on the original image corresponding to the second card and the target task. The third preset model is a multimodal large model. Add the second description information to the associated information.

[0126] According to one or more embodiments of this disclosure, a method for generating interactive canvas content based on a large model is provided, wherein generating a target answer to the target question using a large model based on the association information of the target task includes: Based on the target task, determine the target prompt word template; Based on the target prompt word template and the association information of the target task, target prompt words are generated; The target prompt is input into the fourth preset model to obtain the target answer output by the fourth preset model, which is a large model.

[0127] According to one or more embodiments of this disclosure, a method for generating interactive canvas content based on a large model is provided, wherein displaying the association relationship between the target card and the associated card when the associated information includes the associated card includes: Generate and display the connection between the target card and the associated card.

[0128] According to one or more embodiments of this disclosure, an interactive canvas content generation apparatus based on a large model is provided, the apparatus comprising: A receiving module is used to receive the target question input by the user through an interactive canvas, wherein the interactive canvas includes historical interaction data displayed in the form of cards; The first determining module is used to determine at least one target task to be processed based on the target problem; The second determining module is used to determine the association information of the target task, wherein the association information includes at least one of the following: the associated card determined from the card, and the reference information obtained from the preset information source; The first generation module is used to generate the target answer to the target question using a large model based on the association information of the target task. The first display module is used to display target cards generated based on the target question and the target answer on the interactive canvas, and to display the association relationship between the target card and the associated card when the association information includes the associated card.

[0129] According to one or more embodiments of the present disclosure, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processing device, implements the steps of the interactive canvas content generation method based on a large model provided in any embodiment of the present disclosure.

[0130] According to one or more embodiments of this disclosure, an electronic device is provided, comprising: A storage device on which computer programs are stored; A processing device is configured to execute the computer program in the storage device to implement the steps of the interactive canvas content generation method based on a large model provided in any embodiment of the present disclosure.

[0131] According to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the interactive canvas content generation method based on a large model provided in any embodiment of the present disclosure.

[0132] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0133] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0134] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A method for generating interactive canvas content based on a large model, characterized in that, The method includes: Receive the target question input by the user through an interactive canvas, the interactive canvas including historical interaction data displayed in card form; Based on the target problem, at least one target task to be processed is identified; Determine the associated information of the target task, wherein the associated information includes at least one of the following: associated cards determined from the cards, and reference information obtained from a preset information source; Based on the association information of the target task, the target answer to the target question is generated using a large model; The interactive canvas displays target cards generated based on the target question and the target answer, and displays the association relationship between the target card and the associated card when the association information includes the associated card.

2. The method according to claim 1, characterized in that, Prior to the step of determining at least one target task to be processed based on the target problem, the method further includes: In response to receiving the target question, the question category of the target question is determined, the question category including a first question category representing chat and a second question category representing task processing; The step of determining at least one target task to be processed based on the target problem is performed when the problem category of the target problem is the second problem category.

3. The method according to claim 2, characterized in that, The method further includes: If the problem category of the target problem is the first problem category, obtain the background information associated with the user; Using a first preset model, response information corresponding to the target question is generated based on the background information; the first preset model is a large model. Generate cards corresponding to the target question and the response information and display them through the interactive canvas.

4. The method according to claim 1, characterized in that, The step of determining at least one target task to be processed based on the target problem includes: The target question is subjected to intent recognition to determine at least one question intent; For each of the stated problem intentions, determine the target task corresponding to the stated problem intention.

5. The method according to claim 1, characterized in that, The target task corresponds to a task type, and the task type includes a first type for generating answers to questions; The determination of the association information of the target task includes: For a target task belonging to the first type, determine whether there are any associated cards semantically related to the target task among the cards displayed on the interactive canvas; Obtain reference information related to the semantics of the target task by using a preset information source; If the associated card exists, add the associated card and the reference information to the association information of the target task; If the associated card does not exist, the reference information is added to the associated information of the target task.

6. The method according to claim 1, characterized in that, The target task corresponds to a task type, and the task type includes a second type for generating a summary for the displayed cards; The determination of the association information of the target task includes: For a target task belonging to the second type, among the cards displayed on the interactive canvas, identify associated cards that are semantically related to the target task, as the associated information of the target task.

7. The method according to claim 1, characterized in that, The cards in the interactive canvas correspond to card types, which include text type, file type and image type; The determination of the association information of the target task includes: If the interactive canvas includes a first card of the image type, obtain the first description information corresponding to the first card. The first description information is generated by a second preset model based on the original image corresponding to the first card. The second preset model is a multimodal large model. Based on the first description information, determine whether the associated card exists in the first card.

8. The method according to claim 1, characterized in that, Determining the associated information of the target task includes: In response to receiving a user's selection instruction for at least one card in the interactive canvas, the card indicated by the selection instruction is added to the associated cards.

9. The method according to claim 1, characterized in that, The cards in the interactive canvas correspond to card types, which include text type, file type and image type; The method further includes: If a second card of image type exists among the identified associated cards, a second descriptive information related to the target task is generated using a third preset model based on the original image corresponding to the second card and the target task. The third preset model is a multimodal large model. Add the second description information to the associated information.

10. The method according to claim 1, characterized in that, The step of generating the target answer to the target question using a large model based on the association information of the target task includes: Based on the target task, determine the target prompt word template; Based on the target prompt word template and the association information of the target task, target prompt words are generated; The target prompt is input into the fourth preset model to obtain the target answer output by the fourth preset model, which is a large model.

11. The method according to claim 1, characterized in that, The step of displaying the association relationship between the target card and the associated card when the associated information includes the associated card includes: Generate and display the connection between the target card and the associated card.

12. An interactive canvas content generation device based on a large model, characterized in that, The device includes: A receiving module is used to receive the target question input by the user through an interactive canvas, wherein the interactive canvas includes historical interaction data displayed in the form of cards; The first determining module is used to determine at least one target task to be processed based on the target problem; The second determining module is used to determine the association information of the target task, wherein the association information includes at least one of the following: the associated card determined from the card, and the reference information obtained from the preset information source; The first generation module is used to generate the target answer to the target question using a large model based on the association information of the target task. The first display module is used to display target cards generated based on the target question and the target answer on the interactive canvas, and to display the association relationship between the target card and the associated card when the association information includes the associated card.

13. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processing device, the computer program performs the steps of the method described in any one of claims 1-11.

14. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-11.

Images