Intelligent question and answer method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides an intelligent question and answer method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining question data input by a user; decomposing the problem data to determine a plurality of subtasks corresponding to the problem data; for each subtask, determining a processing data type of the subtask, and matching an intelligent agent corresponding to the processing data type; generating a target code for solving the subtask through the intelligent agent, and executing the target code to obtain target information corresponding to the subtask; and fusing the target information to obtain target output data corresponding to the problem data. According to the method, the subtasks are allocated to the intelligent agents of the corresponding types on the basis of the types of the processed data, so that the multi-modal data can be supported, meanwhile, the target codes are generated on the basis of the specific subtasks, the subtasks are completed through the target codes, the limitation that only specific data can be supported is solved, and the performability of intelligent question answering is greatly expanded.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to an intelligent question answering method and device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] The current question answering system completes the question answering task by calling pre-set tools such as API and database, and the user provides a file in a specified format, and the system can only execute according to the pre-set tools or functions, and when the user introduces new knowledge forms or needs to process the knowledge, the corresponding tool function needs to be redeveloped, and the task executability is relatively limited. SUMMARY

[0003] The main purpose of the present application is to provide an intelligent question answering method, device, electronic device and computer readable storage medium, which aims to solve the problem of limited task executability of the existing question answering system.

[0004] To achieve the above purpose, the present application provides an intelligent question answering method, which comprises the following steps:

[0005] Obtaining problem data input by a user;

[0006] Decomposing the problem data to determine a plurality of subtasks corresponding to the problem data;

[0007] For each subtask, determining the processing data type of the subtask and matching an agent corresponding to the processing data type;

[0008] Generating target code for solving the subtask through the agent, and executing the target code to obtain target information corresponding to the subtask;

[0009] Fusing each target information to obtain target output data corresponding to the problem data.

[0010] Optionally, the generating target code for solving the subtask through the agent comprises:

[0011] Obtaining target processing data corresponding to the subtask;

[0012] Obtaining a preset prompt word and a large model corresponding to the agent, wherein the preset prompt word corresponds to the processing data type of the agent;

[0013] Generating the target code based on the preset prompt word through the large model, wherein the target code indicates that the target processing data is parsed.

[0014] Optionally, the problem data comprises supplementary information; and the obtaining of the target processing data corresponding to the subtask comprises:

[0015] determining type data corresponding to the subtask in the supplementary information, wherein the type data corresponds to a processing data type of the subtask;

[0016] taking the type data as the target processing data corresponding to the subtask.

[0017] Optionally, the problem data comprises a target question, and the obtaining of the target processing data corresponding to the subtask comprises:

[0018] performing network search on the target question to obtain a network search result;

[0019] taking the network search result as the target processing data corresponding to the subtask.

[0020] Optionally, the executing of the target code to obtain the target information corresponding to the subtask comprises:

[0021] starting a target container corresponding to the subtask, wherein the target containers corresponding to the subtasks are isolated from each other;

[0022] determining a target running environment of the target code, and building the target running environment in the target container;

[0023] executing the target code in the target container to obtain the target information.

[0024] Optionally, the fusing of the target information to obtain the target output data corresponding to the problem data comprises:

[0025] converting each of the target information into vectorized data;

[0026] matching historical similar data corresponding to the vectorized data in a historical knowledge base;

[0027] fusing each of the target information and the historical similar data to obtain the target output data corresponding to the problem data.

[0028] Optionally, the fusing of the target information to obtain the target output data corresponding to the problem data comprises:

[0029] for each of the target information, determining a weight of the subtask corresponding to the target information, wherein the weight of the subtask is positively correlated with a correlation degree between the subtask and the problem data;

[0030] The target information is fused based on the weight corresponding to each target information to obtain the target output data corresponding to the question data.

[0031] To achieve the above object, the application further provides an intelligent question answering device, which comprises:

[0032] The first obtaining module is configured to obtain question data input by a user.

[0033] The first determining module is configured to decompose the question data to determine a plurality of sub-tasks corresponding to the question data.

[0034] The second determining module is configured to determine, for each sub-task, a processing data type of the sub-task and match an agent corresponding to the processing data type.

[0035] The first generating module is configured to generate target code for solving the sub-task by the agent and execute the target code to obtain target information corresponding to the sub-task.

[0036] The first fusing module is configured to fuse each target information to obtain target output data corresponding to the question data.

[0037] To achieve the above object, the application further provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program implements the steps of the intelligent question answering method when executed by the processor.

[0038] To achieve the above object, the application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the intelligent question answering method when executed by a processor.

[0039] The application provides an intelligent question answering method and device, an electronic device and a computer readable storage medium. Question data input by a user is acquired. The question data is decomposed to determine a plurality of subtasks corresponding to the question data. For each subtask, a processing data type of the subtask is determined, and an agent corresponding to the processing data type is matched. The agent generates target code for solving the subtask, and executes the target code to obtain target information corresponding to the subtask. The target information is fused to obtain target output data corresponding to the question data. The question data is decomposed into a plurality of subtasks, and the subtasks are distributed to agents of corresponding types based on processing data types, so that the intelligent question answering method can support multi-modal data. In addition, the agent generates target code, so that the tool for solving the subtask does not need to be set in advance, but target code is generated based on the specific subtask, and the subtask is completed through the target code, thereby solving the limitation of supporting only specific data and greatly expanding the executability of the intelligent question answering method. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0042] Figure 1 A flowchart of a first embodiment of the intelligent question answering method of the present application;

[0043] Figure 2 A schematic diagram of an application environment of the intelligent question answering method of the present application;

[0044] Figure 3 A schematic diagram of the overall execution process of the intelligent question answering method of the present application;

[0045] Figure 4 A schematic diagram of the overall principle implementation of the intelligent question answering method of the present application;

[0046] Figure 5 A schematic diagram of the module structure of the electronic device of the present application. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. In order to enable a person skilled in the art to better understand the present application, the technical solutions in the present application will be described clearly and completely in the following with reference to the accompanying drawings of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, and are not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0048] The present application provides an intelligent question and answer method, referring to Figure 1 , Figure 1 is a flowchart of the first embodiment of the intelligent question and answer method of the present application, the method comprising the steps of:

[0049] Step S10, obtaining question data input by a user;

[0050] The question data is the relevant data input by the user when asking a question; the question data can include question text input by the user, uploaded pictures, videos, and other types of files. The specific content contained in the question data is based on the actual input of the user, for example, if the user only inputs a question in the form of text in the dialogue box, the question data only contains the question text; for example, if the user only uploads a PDF file, the question data only contains the PDF file; for example, if the user inputs a question in the form of text in the dialogue box and also uploads pictures, videos, and a PDF file, the question data contains question text, pictures, videos, and a PDF file.

[0051] It should be noted that when the subsequent subtask execution fails, the user can be asked whether to introduce more guiding information, and when the user further provides other data, this part of data also belongs to the question data.

[0052] Step S20, decomposing the question data to determine a plurality of subtasks corresponding to the question data;

[0053] After obtaining the question data, the question that needs to be answered by the user can be determined based on the question data, and the operation that needs to be performed to answer the question of the user can be determined; the subtask indicates an independent operation, and the subtask is the smallest task that cannot be further divided, for example, the subtask can be parsing a document, parsing an image, parsing a video, searching network data, crawling network data, organizing network data, calling an external data source API, and retrieving relevant knowledge information from a historical knowledge base according to the parsed content.

[0054] Taking the walking fault diagnosis of a new energy vehicle assembly robot as an example, the user input problem data includes the question text "According to the provided fault phenomenon video and report, tell me the assembly robot walking fault type, analyze the possible causes and recommend the repair scheme", the fault phenomenon video and the report. The subtasks obtained by decomposing the problem data can include analyzing the user-provided fault video, converting the video into a corresponding multi-dimensional vector, analyzing the user question, converting the question into a corresponding multi-dimensional vector, retrieving local historical fault videos, retrieving local repair manuals, searching for fault-related network data, fusing data, giving fault judgment and solution.

[0055] The decomposition method of the problem data can be set based on actual needs. For example, a question path planning agent can be set. The question path planning agent can specifically decompose the problem data through reasoning-action ReAct. Specifically, a large model and a preset prompt word can be set in the question path planning agent. The large model is prompted by the prompt word to decompose the problem data into executable subtasks and optimize the decision path through self-reflection. The prompt word can be set based on actual needs on the basis of ensuring decomposition.

[0056] The question path planning agent can also display the subtasks to the user in the form of a to-do list, and update the to-do list in real time according to the progress of the subtasks, marking the completed or uncompleted subtasks.

[0057] Step S30, for each subtask, determine the processing data type of the subtask, and match the agent corresponding to the processing data type;

[0058] The processing data type is the type of data processing required by the subtask. For example, the processing data type can be document analysis, image analysis, video analysis, network data search, network data crawling, network data organization, calling external data source API, and retrieving relevant knowledge information from the historical knowledge base according to the analyzed content.

[0059] Corresponding agents are set in advance for different processing data types. The number of agents of the same processing data type can be multiple, so that multiple subtasks of the same processing data type can be processed in parallel.

[0060] Specifically, the specific method of matching the agent can be set based on actual needs. For example, a task path planning agent is set. The task path planning agent determines the data processing type of each subtask and specifies the agent with the same data processing type to execute the corresponding subtask.

[0061] By setting corresponding agents for different processing data types, different types of sub-tasks can be executed by the agents, that is, support for different modal data is realized, the ability of multi-modal data processing is achieved, and the limitation that only single modal knowledge data can be processed in the prior art is solved.

[0062] In step S40, the agent generates target code for solving the sub-task, and executes the target code to obtain target information corresponding to the sub-task.

[0063] The target code is used to solve the sub-task when executed, so as to obtain the target information corresponding to the completion of the sub-task.

[0064] It can be understood that in the embodiment, specific task implementation tools are not pre-set in the agent, such as in the prior art, when text needs to be processed, a specific text processing tool is set, and when a picture needs to be processed, a specific picture processing tool is set. In this case, if the user needs to process or obtain data formats for which no specific tool is set, the related information cannot be processed. In the embodiment, the target code for solving the sub-task is generated, so that the sub-task can generate target code for processing any type of data, thereby realizing the execution of the sub-task, eliminating the dependence on specific tools, avoiding the problem that specific data cannot be processed due to the absence of tools in the prior art, and supporting the use of data of different modalities, different types, and different needs by the user, thereby improving the success rate and accuracy of task completion.

[0065] The generation of specific target code can be set based on actual application needs. For example, if the processing data type corresponding to the sub-task is text, the agent can generate VLM (Vision-Language Model) code for parsing document text; if the processing data type corresponding to the sub-task is a table, the agent can generate Python code for parsing table content by Paddle-OCR; if the processing data type corresponding to the sub-task is a picture, the agent can generate VLM code for parsing document pictures; if the processing data type corresponding to the sub-task is a PDF, the agent can generate Python code for parsing PDF documents by Paddle-OCR; and if the processing data type corresponding to the sub-task is a word, excel, or other type of text file, the agent can generate code for calling the corresponding Python toolkit.

[0066] The target information is the related result information obtained by executing the sub-task. For example, data obtained by parsing pictures, videos, PDFs, and the like.

[0067] To improve support for multi-modal data, after obtaining the target information, the target information can be converted into a structured format, such as JSON format, with the paragraph title as the key and the paragraph content as the value; thereby realizing the unification of the multi-modal data format. The specific structuring method can be set based on actual needs, such as by writing data Pandas code through an intelligent agent.

[0068] Taking the walking fault diagnosis of a new energy vehicle assembly robot as an example, the intelligent agents corresponding to the sub-tasks include a text analysis intelligent agent, a video analysis intelligent agent, and a crawling analysis intelligent agent. The video analysis intelligent agent extracts key frames in the fault phenomenon video to detect abnormal parts of the robot and phenomena. The text analysis intelligent agent retrieves local maintenance manuals. The crawling analysis intelligent agent obtains the latest industry safety standards and standard solutions by simulating the use of browser search, click, page turning, and other behaviors; after analyzing the knowledge, the relevant maintenance videos, maintenance step processes, industry standard SOPs, and other knowledge are recalled to obtain the target information.

[0069] Step S50, the target information is fused to obtain the target output data corresponding to the problem data.

[0070] After obtaining the target information corresponding to each sub-task, the target information can be fused to obtain the target output data corresponding to the problem data.

[0071] The target output data corresponds to the answer to the problem data. The specific way of fusing the target information can be set based on actual needs.

[0072] After obtaining the target output data, the target output data can be displayed to the user.

[0073] In other embodiments, after the target information is fused, the task path planning intelligent agent can also call the VLM large model to generate a website in the form of operable multimedia combined with the target information, and return the website content as the target output data to the user.

[0074] Taking the walking fault diagnosis of a new energy vehicle assembly robot as an example, the task path planning intelligent agent can call the large model to write code to generate the final answer containing the 3D heat map comparison, maintenance steps, maintenance solution guidance video, and compliance suggestions, and display it in the form of a dynamic use manual website, providing more interactivity.

[0075] The embodiment decomposes the problem data into multiple sub-tasks, and distributes the sub-tasks to agents of corresponding types based on the processing data types, so that the multi-modal data can be supported, and the target code is generated by the agent, so that the tool for solving the sub-tasks does not need to be set in advance, but the target code is generated based on the specific sub-tasks, and the sub-tasks are completed through the target code, solving the limitation of only supporting specific data, and greatly expanding the executability of the intelligent question and answer.

[0076] Further, in the second embodiment of the intelligent question and answer method of the present application based on the first embodiment of the present application, the step S40 comprises the steps of:

[0077] Step S41, obtaining target processing data corresponding to the sub-task;

[0078] The target processing data is data that needs to be processed by the sub-task.

[0079] For example, if the sub-task is to analyze the fault video provided by the user, the target processing data corresponding to the sub-task is the fault video uploaded by the user; for another example, if the sub-task is to analyze the user question, the target processing data corresponding to the sub-task is the text question input by the user. Specifically:

[0080] Further, the problem data includes supplementary information; and the step S41 comprises the steps of:

[0081] Step S411, determining type data corresponding to the sub-task in the supplementary information, wherein the type data corresponds to the processing data type of the sub-task;

[0082] Step S412, taking the type data as the target processing data corresponding to the sub-task.

[0083] The supplementary information is other files uploaded by the user in addition to the input text question, such as pictures, videos, documents, etc. Since the agent itself may lack knowledge related to the user question or the user needs to obtain the answer based on the information provided by himself, the user can provide supplementary information related to the question.

[0084] The type data is data in the supplementary information that needs to be processed by the sub-task; it can be understood that the processing data type of the target processing data corresponding to the sub-task is the same as the sub-task, so the data with the same processing data type as the sub-task is determined as the type data in the supplementary information, and the type data is taken as the target processing data.

[0085] For example, if the supplementary information contains pictures, the pictures in the supplementary information can be taken as the target processing data corresponding to the sub-task with the processing data type of analyzing pictures.

[0086] For example, if the supplementary information includes a video, the video can be converted into multiple frames of images, and the multiple frames of images can be used as the target processing data corresponding to the subtask of analyzing pictures.

[0087] Further, the question data includes a target question, and the step S41 includes the steps of:

[0088] Step S413, performing network search on the target question to obtain a network search result.

[0089] Step S414, using the network search result as the target processing data corresponding to the subtask.

[0090] The target question is a text question input by the user into a dialog box.

[0091] To expand the information source, in this embodiment, network search is performed on the target question to obtain more related data through the network search result, and then the answer is obtained based on the more data, thereby improving the accuracy of the answer. In addition to performing network search on the target question, network search can also be performed on the supplementary information after analysis, based on the related content obtained by analysis, to obtain the corresponding target processing data.

[0092] When the subtask corresponds to a network detection intelligent agent, information retrieval can be implemented by calling a browser operation open source framework through a large model, and the network search result can be obtained by searching, browsing, clicking, and crawling data in the browser.

[0093] In other embodiments, a subtask for obtaining external data source information can also be set, and an external data source information intelligent agent can also be set. The intelligent agent can obtain external information by using a large model to write code to call a system-accessed external data source API, and the external information can be used as the target processing data corresponding to the subtask.

[0094] Step S42, obtaining a preset prompt word corresponding to the intelligent agent and a large model, wherein the preset prompt word corresponds to the processing data type of the intelligent agent.

[0095] Step S43, generating the target code based on the preset prompt word through the large model, wherein the target code indicates that the target processing data is analyzed.

[0096] The preset prompt word is a prompt word used to instruct the large model to generate the target code. It can be understood that the preset prompt word is set based on the processing data type corresponding to the intelligent agent. For example, if the processing data type corresponding to the intelligent agent is to analyze pictures, the preset prompt word should instruct the analysis of picture files, and the large model can generate code for analyzing pictures based on the preset prompt word.

[0097] In the embodiment, the preset prompt words and the large model corresponding to the different intelligent agents are set based on the processing data type, so that the execution of the corresponding code implementation subtasks can be generated for different types of data.

[0098] Further, in the third embodiment of the intelligent question and answer method of the application based on the first embodiment of the application, the step S40 comprises the steps of:

[0099] In step S44, the target container corresponding to the subtask is started, wherein the target containers corresponding to the subtasks are isolated from each other.

[0100] In step S45, the target running environment of the target code is determined, and the target running environment is built in the target container.

[0101] In step S46, the target code is executed in the target container to obtain the target information.

[0102] The target container Docker can provide an independent running environment. In the embodiment, different subtasks correspond to different target containers, and because the target containers are isolated from each other, the codes and data in other containers cannot be accessed, so that the subtasks can be independently run, and the information is not polluted by other containers.

[0103] The target running environment is the required running environment of the target code, which can be set based on the actual target code requirements; for example, the Python code for generating Paddle-OCR table content needs to install related Python tool packages in the target container.

[0104] After the target running environment is built, the target code can be executed to obtain the target information.

[0105] When the target code fails to execute, the intelligent agent can call the large model to solve the problems in the container.

[0106] The communication of the intelligent agent corresponding to the subtask is scheduled by the task path planning intelligent agent, and the intelligent agents cannot access each other. The task path planning intelligent agent and the intelligent agent corresponding to the subtask communicate through gRPC-TLS encryption and provide special encryption volumes for data, isolate internal network data and external data, and prevent the data of the intelligent agent from being intercepted and data leakage.

[0107] In the embodiment, the target running environment is built, so that the execution of the target code can be realized.

[0108] Further, in the fourth embodiment of the intelligent question and answer method of the application based on the first embodiment of the application, the step S50 comprises the steps of:

[0109] Step S51, converting each of the target information into vectorized data;

[0110] Step S52, matching the historical similar data corresponding to the vectorized data in the historical knowledge base;

[0111] Step S53, fusing each of the target information and the historical similar data to obtain the target output data corresponding to the problem data.

[0112] By converting the target information into vectorized data, it is possible to align multi-modal data using a unified feature vector, and provide multi-modal data retrieval capability for RAG (Retrieval-augmented Generation).

[0113] The specific vectorization method can be set based on actual needs, such as setting a vectorization agent, calling a multi-modal vectorization large model through the vectorization agent, writing a code for generating a multi-modal feature vector through the VLM, and converting the target information corresponding to the multi-modal data such as pictures, videos, network search results, and external data sources into vectorized data through the execution of the code.

[0114] The historical knowledge base contains historical stored related information. For example, in historical question and answer, the user uploads problem data, and the target information obtained through the problem data is converted into vectorized data, which can be stored in the historical knowledge base. Specifically, when the user uploads the problem data, the user can indicate data storage. When the user indicates storage, the vectorized data corresponding to the problem data is stored in the historical knowledge base, otherwise, it is not stored in the historical knowledge base.

[0115] In this embodiment, by converting the target information into vectorized data, it is possible to recall knowledge data of multiple modalities in the historical knowledge base at the same time, i.e. historical similar data, through a unified vector feature, thereby realizing data expansion. By combining the historical similar data for fusion, it is possible to improve the accuracy of the target output data. For multi-modal input of the user, such as a question described using voice or video, it is also possible to perform retrieval in a unified feature space in the historical knowledge base through vectorized data.

[0116] The historical similar data is data similar to the target information in the historical knowledge base. Specifically, a similarity threshold can be set, and the vector similarity of the vectorized data corresponding to each target information and the data stored in the historical knowledge base is determined, and the data with a vector similarity greater than the similarity threshold in the historical knowledge base is taken as the historical similar data.

[0117] Further, in the fifth embodiment of the intelligent question and answer method of the present application based on the first embodiment of the present application, the step S50 comprises the steps of:

[0118] In step S54, for each target information, the weight of the subtask corresponding to the target information is determined, wherein the weight of the subtask is positively correlated with the correlation degree between the subtask and the problem data.

[0119] In step S55, the target information is fused based on the weight corresponding to each target information to obtain the target output data corresponding to the problem data.

[0120] It can be understood that different target information is obtained through different data via different agents, and therefore, the content indicated by different target information and the influence on the target output data are different.

[0121] In this embodiment, the weight of the subtask is set; when the subtask is generated, the weight corresponding to the subtask can be determined based on the correlation degree between the subtask and the problem data; the determination of the specific weight can be realized by the task path generation agent, such as determining the correlation degree between the subtask and the problem data by the large model in the task path generation agent, and then obtaining the corresponding weight.

[0122] For example, taking the new energy automobile assembly robot walking fault diagnosis as an example, the problem data input by the user includes the problem text “according to the provided fault phenomenon video and report, tell me the fault type of the assembly robot, analyze the possible reasons and recommend the maintenance scheme”, the fault phenomenon video and the report, the task path generation agent calls and assigns the weight of the video analysis subtask 50%, the text analysis subtask 40% and the crawling analysis subtask 10%.

[0123] When different target information has conflicting content, the target information with higher weight can be used for fusion, and the target information with lower weight can be discarded, so that the accuracy of the target output data can be improved.

[0124] The information fusion can be realized by setting the information fusion judgment agent, such as the information fusion judgment agent which sorts and fuses multiple target information according to the weight given by the large model.

[0125] In other embodiments, if there is a conflict between different target information, a user real-time feedback mechanism can be introduced, such as displaying inquiry information to enable the user to select between the conflicting content, and then fusing based on the target information selected by the user. For example, the calling subtask intelligent agent stores historical interaction information, and dynamically adjusts the subsequent strategy by observing the results of the execution of the intelligent agent corresponding to the subtask. For problems that cannot be handled, a request is initiated to the user to seek user feedback and take over; if the subtask fails, the subtask is retried, and if the number of retries reaches a retry threshold and still fails, for the subtask that affects the execution of the task path, the user is asked whether to introduce more guiding information to improve accuracy; for the subtask that does not affect the execution of the task path and has a lower weight, the subtask can be discarded to improve task efficiency.

[0126] Referring to Figure 2 , Figure 2 is a schematic diagram of the application environment of the intelligent question and answer method of the application; the client communicates with the server through the network, the server accepts the question data issued by the user through the client, and the server executes the intelligent question and answer method of the application based on the question data after accepting the question data.

[0127] Referring to Figure 3 , based on Figure 3 the overall execution process of the application is described:

[0128] 1. The user inputs question data;

[0129] 2. The question data is decomposed to determine a plurality of subtasks corresponding to the question data, and the weights of the subtasks are determined based on the correlation degree between the subtasks and the question data;

[0130] 3. The task path planning intelligent agent determines the data processing type of each subtask, and specifies the intelligent agent of the same data processing type to execute the corresponding subtask; wherein, for the subtasks executed in parallel, a plurality of intelligent agents can be used to realize synchronous calling, and for the subtasks executed in sequence, the calling and execution are performed in sequence based on the execution sequence;

[0131] 4. The subtasks are displayed to the user in the form of a to-do list, the to-do list is updated in real time according to the progress of the completion of the subtasks, and the completed or uncompleted subtasks are marked; whether the to-do list is executed completely is judged;

[0132] 5. When the to-do list is not executed completely, the uncompleted subtasks are executed;

[0133] 6. Whether the subtask is executed successfully is judged;

[0134] 7. If the execution is not successful, it is judged whether the retry number reaches the retry threshold, if not, the subtask is retried, if yes, the retry is stopped;

[0135] 8. It is judged whether each subtask is completed, if there is a subtask which reaches the retry threshold but is still not successful, the user is asked whether more guiding information is introduced;

[0136] 9. If all subtasks are completed, it is returned to the fourth step;

[0137] 10. When the completion list execution is completed, the information fusion judgment agent sorts and fuses multiple target information according to the weight given by the large model to obtain target output information;

[0138] 11. According to the recalled multi-modal knowledge, the VLM large model is called to generate the corresponding answer to the question.

[0139] Referring to Figure 4 , based on Figure 4 the overall principle of the implementation of the application is described:

[0140] The large language model is set to realize the path planning agent to decompose the problem data, and the large language model communicates with the intelligent question and answer device;

[0141] The intelligent question and answer device communicates with the intelligent agent management system corresponding to the subtask;

[0142] A plurality of intelligent agents are set in the intelligent agent management system, and based on the processing data type, text analysis intelligent agents, image analysis intelligent agents, video analysis intelligent agents, network data crawling intelligent agents and external API acquisition intelligent agents are included.

[0143] Each intelligent agent starts an independent target container when running, and correspondingly, the container includes a text analysis container, an image analysis container, a video analysis container, a network data crawling container and an external API acquisition container; the target container can provide an independent running environment. In the embodiment, different subtasks correspond to different target containers, and since the target containers are isolated and cannot access the codes and data in other containers, the subtasks can be independently run, and the information is ensured not to be polluted by other containers.

[0144] When the intelligent agent executes the subtask, the large model is called in the target container to write target code and configure a running environment or call the large model to analyze data;

[0145] The target code is executed to obtain target information corresponding to the subtask; the target information is fused to obtain target output data corresponding to the problem data, and the target output data is returned to the user through the intelligent question and answer device.

[0146] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0147] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device) to execute the method described in each embodiment of the present application.

[0148] The present application also provides an intelligent question answering device for implementing the above intelligent question answering method, the intelligent question answering device comprising:

[0149] A first acquisition module for acquiring user input question data;

[0150] A first determination module for decomposing the question data to determine a plurality of sub-tasks corresponding to the question data;

[0151] A second determination module for determining, for each sub-task, a processing data type of the sub-task and matching an agent corresponding to the processing data type;

[0152] A first generation module for generating target code for solving the sub-task through the agent and executing the target code to obtain target information corresponding to the sub-task;

[0153] A first fusion module for fusing each target information to obtain target output data corresponding to the question data.

[0154] The intelligent question answering device decomposes the question data into a plurality of sub-tasks, and assigns the sub-tasks to agents of corresponding types based on processing data types, so that it can support multi-modal data. At the same time, by generating target code through the agent, it is not necessary to set the tool for solving the sub-task in advance, but to generate the target code based on the specific sub-task, and to complete the sub-task through the target code, which solves the limitation of only supporting specific data and greatly expands the executability of intelligent question answering.

[0155] It should be noted that the first obtaining module in this embodiment can be used to execute step S10 in the embodiments of the present application, the first determining module in this embodiment can be used to execute step S20 in the embodiments of the present application, the second determining module in this embodiment can be used to execute step S30 in the embodiments of the present application, the first generating module in this embodiment can be used to execute step S40 in the embodiments of the present application, and the first fusing module in this embodiment can be used to execute step S50 in the embodiments of the present application.

[0156] Further, the first generating module comprises:

[0157] a first obtaining unit, configured to obtain target processing data corresponding to the subtask;

[0158] a second obtaining unit, configured to obtain a preset prompt word and a large model corresponding to the agent, wherein the preset prompt word corresponds to a processing data type of the agent;

[0159] a first generating unit, configured to generate the target code based on the preset prompt word by using the large model, wherein the target code indicates that the target processing data is parsed.

[0160] Further, the problem data comprises supplementary information, and the first obtaining unit comprises:

[0161] a first determining subunit, configured to determine type data corresponding to the subtask in the supplementary information, wherein the type data corresponds to a processing data type of the subtask;

[0162] a first executing subunit, configured to take the type data as the target processing data corresponding to the subtask.

[0163] Further, the problem data comprises a target problem, and the first obtaining unit comprises:

[0164] a first searching subunit, configured to perform network searching on the target problem to obtain a network searching result;

[0165] a second executing subunit, configured to take the network searching result as the target processing data corresponding to the subtask.

[0166] Further, the first generating module comprises:

[0167] a first starting unit, configured to start a target container corresponding to the subtask, wherein the target containers corresponding to the subtasks are isolated from each other;

[0168] The first determining unit is configured to determine a target running environment of the target code, and to build the target running environment in the target container.

[0169] The first executing unit is configured to execute the target code in the target container to obtain the target information.

[0170] Further, the first fusion module comprises:

[0171] The first converting unit is configured to convert each of the target information into vectorized data.

[0172] The first matching unit is configured to match historical similar data corresponding to the vectorized data in a historical knowledge base.

[0173] The first fusion unit is configured to fuse each of the target information and the historical similar data to obtain target output data corresponding to the problem data.

[0174] Further, the first fusion module comprises:

[0175] The second determining unit is configured to determine, for each of the target information, a weight of the subtask corresponding to the target information, wherein the weight of the subtask is positively correlated with a correlation degree between the subtask and the problem data.

[0176] The second fusion unit is configured to fuse the target information based on the weight corresponding to each of the target information to obtain target output data corresponding to the problem data.

[0177] Reference Figure 5 In terms of hardware structure, the electronic device can comprise a communication module 10, a memory 20, a processor 30 and the like. In the electronic device, the processor 30 is connected with the memory 20 and the communication module 10 respectively, the memory 20 stores a computer program, the computer program is executed by the processor 30, and the computer program realizes the steps of the above method embodiments when executed.

[0178] The communication module 10 can be connected with external communication devices through a network. The communication module 10 can receive a request sent by the external communication device, and can also send a request, an instruction and information to the external communication device. The external communication device can be other electronic devices, servers or Internet of Things devices, such as televisions and the like.

[0179] Memory 20 can be used to store software programs and various data. Memory 20 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as obtaining user-entered question data). The data storage area may include a database and may store data or information generated based on system usage. Furthermore, memory 20 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0180] The processor 30 is the control center of the electronic device. It connects all parts of the electronic device using various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 20 and accessing data stored in the memory 20, it performs various functions of the electronic device and processes data, thereby providing overall monitoring of the electronic device. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 30.

[0181] although Figure 5 Although not shown, the electronic device may further include a circuit control module, which is used to connect to the power supply to ensure the normal operation of other components. Figure 5 The electronic device structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0182] The present invention also provides a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium may be Figure 5 The memory 20 in the electronic device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a television, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

[0183] In the present application, the terms "first", "second", "third", "fourth", "fifth" are only for the purpose of description, and cannot be understood as indicating or implying relative importance, and for those skilled in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0184] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms is not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0185] Although the embodiments of the present application have been shown and described above, the scope of protection of the present application is not limited thereto, and it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications and replacements to the above embodiments within the scope of the present application, and these changes, modifications and replacements shall be covered within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims.

Claims

1. An intelligent question-answering method, characterized in that: The intelligent question-answering method comprises: Get the question data entered by the user; Decomposing the problem data to determine a plurality of subtasks corresponding to the problem data; For each of the subtasks, determining a processing data type of the subtask, and matching an agent corresponding to the processing data type; Generate a target code for solving the subtask through the agent, and execute the target code to obtain target information corresponding to the subtask; The target information is fused to obtain target output data corresponding to the problem data.

2. The intelligent question-answering method according to claim 1, wherein: The generating of the target code for solving the subtask by the agent includes: Obtaining target processing data corresponding to the subtask; Obtaining a preset prompt word and a large model corresponding to the agent, wherein the preset prompt word corresponds to a processing data type of the agent; The target code is generated based on the preset prompt word through the large model, wherein the target code indicates parsing of the target processing data.

3. The intelligent question-answering method according to claim 2, wherein: The problem data includes supplementary information; obtaining the target processing data corresponding to the subtask includes: Determining type data corresponding to the subtask in the supplementary information, wherein the type data corresponds to the type of processing data of the subtask; The type data is used as the target processing data corresponding to the subtask.

4. The intelligent question-answering method according to claim 2, wherein: The problem data includes a target problem, and obtaining target processing data corresponding to the subtask includes: Performing a web search on the target question to obtain web search results; The network search results are used as the target processing data corresponding to the subtask.

5. The intelligent question-answering method according to claim 1, wherein: The executing the target code to obtain the target information corresponding to the subtask includes: Starting the target container corresponding to the subtask, wherein the target containers corresponding to the subtasks are isolated from each other; Determine a target operating environment for the target code and build the target operating environment in the target container; The target code is executed in the target container to obtain the target information.

6. The intelligent question-answering method according to claim 1, wherein: The target output data corresponding to the problem data obtained by fusing the target information includes: Convert each of the target information into vectorized data; Matching historical similar data corresponding to the vectorized data in a historical knowledge base; The target information and the historical similar data are fused to obtain target output data corresponding to the problem data.

7. The intelligent question-answering method according to claim 1, wherein: The target output data corresponding to the problem data obtained by fusing the target information includes: For each piece of target information, determining a weight of the subtask corresponding to the target information, wherein the weight of the subtask is positively correlated with the degree of relevance between the subtask and the problem data; The target information is fused based on the weights corresponding to the respective target information to obtain target output data corresponding to the problem data.

8. An intelligent question-answering device, characterized in that: The intelligent question-answering device comprises: The first acquisition module is used to obtain question data input by the user; A first determining module, configured to decompose the problem data to determine a plurality of subtasks corresponding to the problem data; A second determining module is configured to determine, for each of the subtasks, a processing data type of the subtask and match an agent corresponding to the processing data type; A first generation module is configured to generate a target code for solving the subtask through the agent, and execute the target code to obtain target information corresponding to the subtask; The first fusion module is used to fuse the target information to obtain target output data corresponding to the problem data.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the intelligent question-answering method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the intelligent question-answering method according to any one of claims 1 to 7.

Images