Data processing method, data processing system, and computing device

Abstract

This application discloses a data processing method, data processing system, and computing device, relating to the fields of large-scale model technology and multimodal data processing. The method includes: acquiring text data; performing intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; assigning the at least one processing task to at least one second intelligent agent, and using the at least one second intelligent agent to execute the corresponding processing task to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; and integrating the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data. This application solves the technical problems of poor real-time performance and low accuracy in processing user-input text data by intelligent assistant systems in related technologies.

Description

Technical Field

[0001] This application relates to the field of large model technology and multimodal data processing, and more specifically, to a data processing method, a data processing system, and a computing device. Background Technology

[0002] With the rapid development of artificial intelligence technology, multimodal intelligent assistant systems are gradually becoming a bridge connecting humans and the digital world. These systems integrate information processing capabilities across multiple modalities, such as text, voice, and images, to provide users with a richer and more natural interactive experience. However, current intelligent assistant systems suffer from poor real-time processing and low accuracy when handling user-input text data.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a data processing method, a data processing system, and a computing device to at least solve the technical problems of poor real-time performance and low accuracy in processing user-input text data by intelligent assistant systems in related technologies.

[0005] According to one aspect of the embodiments of this application, a data processing method is provided, comprising: acquiring text data; performing intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; assigning the at least one processing task to at least one second intelligent agent, and using the at least one second intelligent agent to execute the corresponding processing task respectively, to obtain processing results corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; and integrating the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data.

[0006] According to another aspect of the embodiments of this application, a data processing method is also provided, comprising: responding to an input command applied to an operation interface, determining text data corresponding to the input command; performing intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; assigning the at least one processing task to at least one second intelligent agent, and using the at least one second intelligent agent to execute the corresponding processing task respectively, obtaining processing results corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; integrating the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data; and outputting the target data.

[0007] According to another aspect of the embodiments of this application, a data processing method is also provided, comprising: acquiring text data by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the text data; performing intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; assigning the at least one processing task to at least one second intelligent agent, and using the at least one second intelligent agent to execute the corresponding processing task respectively to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; integrating the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data; and outputting the target data by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target data.

[0008] According to another aspect of the embodiments of this application, a data processing system is also provided, including: a client for sending text data; a server connected to the client for performing intent analysis on the text data using a first intelligent agent to generate at least one processing task, assigning the at least one processing task to at least one second intelligent agent, and executing the corresponding processing task using the at least one second intelligent agent to obtain the processing result corresponding to the at least one processing task, and integrating the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data, wherein different processing tasks are used to perform different types of processing on the text data, and at least one processing task corresponds one-to-one with at least one second intelligent agent; the client is also used to output the target data.

[0009] According to another aspect of the embodiments of this application, a data processing system is also provided, comprising: a plurality of intelligent agents, including a first intelligent agent and at least one second intelligent agent; wherein the first intelligent agent is used to perform intent analysis on text data, generate at least one processing task, and assign the at least one processing task to at least one second intelligent agent, wherein different processing tasks are used to perform different types of processing on the text data; at least one second intelligent agent is used to execute the corresponding processing task respectively to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; the first intelligent agent is also used to integrate the processing results corresponding to the at least one processing task to generate target data.

[0010] According to one aspect of the embodiments of this application, a data processing apparatus is provided, comprising: an acquisition module for acquiring text data; an analysis module for performing intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; a processing module for assigning the at least one processing task to at least one second intelligent agent, and using the at least one second intelligent agent to execute the corresponding processing task respectively to obtain processing results corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; and an integration module for integrating the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data.

[0011] According to another aspect of the embodiments of this application, a data processing apparatus is also provided, comprising: a determining module, configured to determine text data corresponding to an input command applied to an operation interface; an analysis module, configured to perform intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; a processing module, configured to assign the at least one processing task to at least one second intelligent agent, and use the at least one second intelligent agent to execute the corresponding processing task respectively to obtain a processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; an integration module, configured to integrate the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data; and an output module, configured to output the target data.

[0012] According to another aspect of the embodiments of this application, a data processing apparatus is also provided, comprising: an acquisition module, configured to acquire text data by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the text data; an analysis module, configured to perform intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; an allocation module, configured to allocate the at least one processing task to at least one second intelligent agent, and use the at least one second intelligent agent to execute the corresponding processing task respectively to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; an integration module, configured to integrate the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data; and an output module, configured to output the target data by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target data.

[0013] According to another aspect of the embodiments of this application, a computing device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor connected to the memory via a bus for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0016] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0018] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0019] In this embodiment, the method involves: acquiring text data; using a first intelligent agent to perform intent analysis on the text data to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; assigning the at least one processing task to at least one second intelligent agent, and using the at least one second intelligent agent to execute the corresponding processing task to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; and using the first intelligent agent to integrate the processing results corresponding to the at least one processing task to generate target data. It is readily apparent that by having at least one second intelligent agent work collaboratively to execute corresponding processing tasks, the goal of efficiently processing text data is achieved, improving the real-time performance of text data processing. Furthermore, by having multiple intelligent agents process multiple processing tasks separately, the problem of low accuracy in text data processing by a single intelligent agent is avoided, achieving the goal of more accurate text data processing and improving the accuracy of text data processing. This achieves the technical effect of more real-time and more accurate text data processing, thereby solving the technical problems of poor real-time performance and low accuracy in processing user-input text data by intelligent assistant systems in related technologies.

[0020] It is worth noting that the general description above and the detailed description that follow are merely for illustrative purposes and do not constitute a limitation on this application. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a schematic diagram illustrating an application scenario of a data processing method according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of a data processing method according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of an agent-based interaction according to an embodiment of this application;

[0025] Figure 4 This is a flowchart of a data processing method according to an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of an operation interface according to an embodiment of this application;

[0027] Figure 6This is a flowchart of a data processing method according to an embodiment of this application;

[0028] Figure 7 This is a schematic diagram of a data processing system according to an embodiment of this application;

[0029] Figure 8 This is a schematic diagram of a data processing system according to an embodiment of this application;

[0030] Figure 9 This is a schematic diagram of a data processing apparatus according to an embodiment of this application;

[0031] Figure 10 This is a schematic diagram of a data processing apparatus according to an embodiment of this application;

[0032] Figure 11 This is a schematic diagram of a data processing apparatus according to an embodiment of this application;

[0033] Figure 12 This is a structural block diagram of a computing device according to an embodiment of this application;

[0034] Figure 13 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] The technical solution provided in this application is mainly implemented using large-scale model technology. Here, "large-scale model" refers to a deep learning model with a large number of parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of parameters. Large-scale models can also be called foundational models. They are pre-trained using large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and Multimodal Pre-training Models.

[0038] It should be noted that, in practical applications, large models can be fine-tuned using a small number of samples to adapt them to different tasks. For example, large models can be widely used in Natural Language Processing (NLP), computer vision, and speech processing. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. Therefore, the main application scenarios for large models include, but are not limited to, digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design. In this embodiment, the use of a large model for data processing in an intelligent assistant system is taken as an example for explanation.

[0039] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0040] Agent (Intelligent Agent): In computer science, an agent is an entity that possesses autonomy, responsiveness, sociality, and initiative. It can be a software program or a hardware device used to perform specific tasks.

[0041] Multimodal: refers to technologies or systems that can process and integrate multiple types of data or information (e.g., text, audio, video, etc.).

[0042] Natural Language Processing (NLP): NLP is a branch of artificial intelligence that aims to enable computers to understand, interpret, and generate human natural language.

[0043] According to an embodiment of this application, a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0044] Considering the large number of model parameters in large models and the limited computing resources of mobile terminals, the method provided in this application embodiment can be applied to, for example, Figure 1 The application scenarios shown are not limited to these. Figure 1 This is a schematic diagram illustrating an application scenario of a data processing method according to an embodiment of this application. Figure 1 In the application scenario shown, the large model is deployed on server 10. Server 10 can connect to one or more client devices 20 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Client devices 20 can interact with users through a graphical user interface to access the large model, thereby implementing the method provided in this embodiment.

[0045] In this embodiment, the system consisting of a client device and a server can perform the following steps: The client device displays the input text data and target data in a graphical user interface; the server acquires the text data; performs intent analysis on the text data using a first intelligent agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; assigns the at least one processing task to at least one second intelligent agent, and uses the at least one second intelligent agent to execute the corresponding processing task to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; and integrates the processing results corresponding to the at least one processing task using the first intelligent agent to generate the target data.

[0046] It should be noted that with the rapid development of high-performance computing units, the methods provided in this application embodiment can also be applied to model-in-machine systems in other application scenarios. In one optional embodiment, the model-in-machine system has multiple built-in models, and users can select one model to adjust as needed to obtain their own model. The high-performance computing unit built into the model-in-machine system can then directly call the adjusted model to execute the methods provided in this application embodiment. In another optional embodiment, the large model-in-machine system has a pre-trained model built-in, and the high-performance computing unit built into the model-in-machine system can then directly call that model to execute the methods provided in this application embodiment.

[0047] Furthermore, when users need to train their own models, they can upload their own datasets via the client. These datasets are then sent to the server, allowing the server to adjust the pre-trained model using the dataset to obtain the user's customized model, which can then be deployed to the production environment. To facilitate users' model adjustment needs, the server provides complete adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies. This allows the adjusted model to better adapt to different application domains and achieve a high degree of customization.

[0048] Under the aforementioned operating environment, this application provides the following: Figure 2 The data processing method shown. Figure 2 This is a flowchart of a data processing method according to an embodiment of this application. Figure 2 As shown, the method may include the following steps:

[0049] Step S202: Obtain text data.

[0050] The aforementioned text data can be data input by the user in the interactive interface, containing user intent. Users can input text in the dialog box of the interactive interface to obtain text data, or they can input voice in the dialog box, which will then recognize the voice to obtain text data, but this is not limited to these methods. The application scenarios of the aforementioned interactive interface can include, but are not limited to: e-commerce platforms, social media platforms, online learning platforms, smart home systems, mobile applications, health management applications, smart assistant systems, and intelligent driving systems. Technical personnel can define the specific application scenarios according to actual usage needs. In this embodiment, the personal assistant interactive interface (hereinafter referred to as the interactive interface) in a smart assistant system is used as an example for explanation, but it is not limited to this. The aforementioned user intent reflects the user's needs. The types of user intent can include, but are not limited to: information acquisition intent, problem-solving intent, entertainment intent, and social interaction intent. In this embodiment, information acquisition intent is used as an example for explanation, but it is not limited to this. Information acquisition intent can include, but is not limited to: search intent, document intent, text-to-image intent, schema selection, and knowledge enhancement intent. Among these, search intent indicates a need to search the internet or knowledge base using a search engine's Application Programming Interface (API) based on text data to obtain the user's desired data. Document intent indicates a need to generate documents based on text data. Text-to-image intent indicates a need to generate images based on text data. Primitive model selection indicates a need to select the appropriate processing model based on text data to obtain the accurate processing results required by the user. Knowledge enhancement intent indicates a need to process or improve text data to obtain higher quality, more detailed, and richer data about the text data required by the user.

[0051] In one optional embodiment, during user interaction through the intelligent assistant system, the user inputs text data with their intent in a dialog box on the interactive interface. After receiving the text data, the intelligent assistant system analyzes it to obtain a result that matches the user's intent. However, current intelligent assistant systems suffer from poor real-time performance and low accuracy in processing text data. To address these issues, this embodiment first acquires the text data when the user inputs it into the interactive interface. For example, if the user inputs the text "How tall is the pine tree?" into the dialog box on the interactive interface, this text data can be acquired first.

[0052] Step S204: The first intelligent agent performs intent analysis on the text data to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data.

[0053] The first intelligent agent described above performs intent analysis on the acquired text data to obtain the target intent corresponding to the text data. The processing tasks described above instruct the intelligent assistant system to perform different types of processing on the text data. Among these, the processing tasks and the intent... Figure 1 One-to-one correspondence can include, but is not limited to, at least one of the following: search task, document task, text-to-graph task, schema selection task, and knowledge enhancement task.

[0054] In one optional embodiment, upon receiving text data input by the user, the intelligent assistant system first performs intent analysis on the text data using a first intelligent agent to obtain at least one processing task. For example, the first intelligent agent can first segment the text data into words, dividing continuous text data into individual words and / or phrases. Next, the first intelligent agent can extract features from the words and / or phrases to obtain word features and / or phrase features. Then, the first intelligent agent can call a pre-trained intent recognition model to recognize the intent from the word features and / or phrase features, obtaining at least one intent from the text data. Finally, the first intelligent agent can generate at least one processing task based on at least one intent. Different processing tasks are used to perform different types of processing on the text data. For another example, the intelligent assistant system can first perform search determination, knowledge base determination, base model determination, and text-to-image determination on the text data using the first intelligent agent. If it is determined that a search and knowledge base are required, the first intelligent agent can determine the existence of a search intent and a document intent. At this point, the first intelligent agent can generate tasks to be processed as a search task and a document task based on the search intent and document intent. When it is determined that a search and text-to-image processing are required, the first agent can determine that there is a search intent and a text-to-image processing intent. At this time, the first agent can generate the tasks to be processed as the search task and the text-to-image processing task based on the search intent and the text-to-image processing intent.

[0055] In another optional embodiment, when the user inputs the text data "How tall is a pine tree?", the intelligent assistant system first performs intent analysis on the text data through a first intelligent agent to determine the user's intent as a search intent. Then, the intelligent assistant system can generate a processing task based on the search intent: a search task. As another example, when the user inputs the text data "Please help me generate a video containing a kitten", the intelligent assistant system first performs intent analysis on the text data through a first intelligent agent to determine the user's intent as a search intent and a video generation intent. Then, the intelligent assistant system can generate multiple processing tasks based on the search intent and the video generation intent: a search task and a video generation task. The first intelligent agent is used to perform intent analysis on the text data to obtain the intent corresponding to the text data.

[0056] In another alternative embodiment, when the user input text data is "How tall is the pine tree?", the intelligent assistant system first analyzes the text data using a pre-trained intent classification model to determine the user's intent as a search intent. Then, based on this search intent, the intelligent assistant system generates a processing task: a search task. As another example, when the user input text data is "Please help me generate a video containing a kitten", the intelligent assistant system first analyzes the text data using a pre-trained intent classification model to determine the user's intent as a search intent and a video generation intent. Then, based on these two intents, the intelligent assistant system generates multiple processing tasks: a search task and a video generation task.

[0057] Step S206: Assign at least one processing task to at least one second agent, and use at least one second agent to execute the corresponding processing task to obtain the processing result corresponding to at least one processing task, wherein at least one processing task corresponds one-to-one with at least one second agent.

[0058] The aforementioned second intelligent agent is used to execute corresponding processing tasks and obtain the processing results corresponding to those tasks. That is, different processing tasks correspond one-to-one with different second intelligent agents. Different intelligent agents are created to execute different processing tasks. The types of second intelligent agents can include, but are not limited to: search intelligent agents, text-to-image intelligent agents, and document intelligent agents. It should be noted that an intelligent agent is an entity capable of perceiving its environment and making decisions to achieve a specific goal. Intelligent agents can be in software form, such as chatbots, recommendation systems, and control software for autonomous vehicles; or in hardware form, such as robots and drones. The main functions of intelligent agents include the following:

[0059] Automation and Efficiency Enhancement: Intelligent agents can automate tasks, reducing manpower requirements and improving work efficiency. Decision Support: Intelligent agents can make reasonable decisions based on environmental information and preset goals, assisting humans in solving complex problems. Learning and Adaptation: Many intelligent agents possess learning capabilities, enabling them to learn from experience and adapt to environmental changes, improving their performance. Interaction and Communication: Intelligent agents can interact with humans or other intelligent agents to provide services or complete tasks. Monitoring and Early Warning: In some applications, intelligent agents are used to monitor the state of the environment or system, issuing timely warnings upon detecting anomalies. Personalized Services: Intelligent agents can provide personalized services or product recommendations based on user behavior and preferences. Exploration and Discovery: In scientific research and exploration, intelligent agents can perform tasks that are difficult or impossible for humans, such as deep-sea exploration and space exploration. Simulation and Prediction: Intelligent agents can be used to simulate the behavior of complex systems, predict future trends, and provide a basis for decision-making.

[0060] In one optional embodiment, when at least one processing task is obtained, the intelligent assistant system can first assign the at least one processing task to at least one first agent. Secondly, the intelligent assistant system can generate target prompts corresponding to at least one first agent based on text data, and determine a target processing model based on at least one task to be processed. Then, the intelligent assistant system can input the target prompts into the target processing model corresponding to the at least one first agent, and the target processing model generates the processing result corresponding to the at least one processing task based on the corresponding processing task. For example, when the processing task is a search task, the intelligent assistant system can first assign the search task (i.e., the processing task) to the corresponding search agent (i.e., the second agent). Secondly, the intelligent assistant system can generate target prompts corresponding to the search agent based on text data (e.g., including but not limited to: text processing model, text search), and determine a target processing model based on the search task (e.g., a large language model (LLM)). Then, the intelligent assistant system can input the target prompts into the large language model corresponding to the search agent, and the large language model generates the search result (i.e., the processing result) corresponding to the search agent based on the corresponding search task (i.e., the processing task). For example, given that the processing tasks are a search task and a text-to-image task, the intelligent assistant system can first assign the search task and the text-to-image task (i.e., multiple processing tasks) to the search agent and the text-to-image agent (i.e., multiple second agents). Next, the intelligent assistant system can generate target prompts for the search agent and the text-to-image agent based on text data, and determine the target processing model (e.g., a visual language model (VLMs)) based on the search task and the text-to-image task. Then, the intelligent assistant system can input the target prompts into the visual language models corresponding to the search agent and the text-to-image agent. The visual language models then generate the search results for the search agent and the text-to-image results (i.e., the processing results) for the text-to-image agent based on the corresponding search task and the text-to-image task (i.e., multiple processing tasks).

[0061] It should be noted that the aforementioned at least one task to be processed can determine the target processing model corresponding to at least one second agent, and the aforementioned target prompts are used to help the target processing model generate the processing results corresponding to at least one processing task. Target prompts may include, but are not limited to, document prompts and search prompts. The aforementioned target processing models may include, but are not limited to, large language models and visual language models. Different target processing models have different network structures.

[0062] In another optional embodiment, when at least one processing task is obtained, the intelligent assistant system can first assign the at least one processing task to at least one second intelligent agent. Then, the at least one second intelligent agent can execute the corresponding at least one pending task to obtain the processing result corresponding to the at least one processing task. For example, when multiple processing tasks are obtained, namely a search task and a text-to-image task, the intelligent assistant system can first assign the search task to a search intelligent agent and the text-to-image task to a text-to-image intelligent agent. Then, the search intelligent agent can execute the search task to obtain search results, and simultaneously, the text-to-image intelligent agent can execute the text-to-image task to obtain text-to-image results.

[0063] Step S208: The first intelligent agent integrates the processing results corresponding to at least one processing task to generate target data.

[0064] The target data mentioned above can be data that is ultimately returned to the user and conforms to the user's intent. The target data includes data in at least two modalities: text, images, video, and audio. For example, it can be data containing text and images, data containing text, images, and video, or data containing text, images, video, and audio. The specific number and type of modal data included in the target data are determined according to the user's intent and are not limited in this embodiment.

[0065] In one optional embodiment, upon obtaining the processing results corresponding to at least one processing task, the intelligent assistant system can integrate the processing results corresponding to at least one task by invoking a target processing model through a first intelligent agent to generate target data. For example, upon obtaining the search results corresponding to the search task and the text-to-image result corresponding to the text-to-image task, the intelligent assistant system can integrate the search results and the text-to-image result by invoking a visual language model through a first intelligent agent, thereby obtaining target data containing text and video.

[0066] In this embodiment, the method involves: acquiring text data; using a first intelligent agent to perform intent analysis on the text data to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; assigning the at least one processing task to at least one second intelligent agent, and using the at least one second intelligent agent to execute the corresponding processing task to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; and using the first intelligent agent to integrate the processing results corresponding to the at least one processing task to generate target data. It is readily apparent that by having at least one second intelligent agent work collaboratively to execute corresponding processing tasks, the goal of efficiently processing text data is achieved, improving the real-time performance of text data processing. Furthermore, by having multiple intelligent agents process multiple processing tasks separately, the problem of low accuracy in text data processing by a single intelligent agent is avoided, achieving the goal of more accurate text data processing and improving the accuracy of text data processing. This achieves the technical effect of more real-time and more accurate text data processing, thereby solving the technical problems of poor real-time performance and low accuracy in processing user-input text data by intelligent assistant systems in related technologies.

[0067] In the above embodiments of this application, the first intelligent agent is used to perform intent analysis on text data to generate at least one processing task, including: using the first intelligent agent to perform intent analysis on text data to determine at least one target intent corresponding to the text data; and generating a processing task that matches the target intent based on any one of the target intents.

[0068] The first intelligent agent described above performs intent analysis on the text data input by the user to obtain the target intent corresponding to the text data. The types of intent analysis for text data can include, but are not limited to: search determination, knowledge base determination, base model determination, and text-to-image determination. Search determination is used to decide whether a web search is needed, and the specific content and scope of the search. Knowledge base determination is used to decide whether an internal knowledge base needs to be queried, and the specific content and scope of the query. Base model determination is used to determine whether the base model used is LLM or VLM. Text-to-image determination is used to decide whether an image needs to be generated, and the specific content and style of the generated image. Each determination corresponds to a specific determination task, which can be determined based on the specific functions of the intelligent agent.

[0069] In one optional embodiment, upon acquiring text data, the intelligent assistant system can perform intent analysis on the text data using a first intelligent agent. This includes performing search determination, knowledge base determination, base model determination, and text-to-image determination to obtain the target intent corresponding to the text data. The intelligent assistant system can then generate at least one processing task based on the target intent. For example, if the acquired text data is "I need to view the historical update records of product ××", the intelligent assistant system can first use the first intelligent agent to perform intent analysis on the text data, obtaining the target intent as a search intent and a document intent, i.e., searching for the historical update records of product ×× and generating a document. In this case, the intelligent assistant system can generate processing tasks as a search task and a document task based on the target intent. As another example, if the acquired text data is "Generate a training video about employee training", the intelligent assistant system can first use the first intelligent agent to perform intent analysis on the text data, obtaining the target intent as a search intent and a video generation intent, i.e., searching for content about employee training and generating a video. In this case, the intelligent assistant system can generate processing tasks as a search task and a video generation intent based on the target intent, but this is not limited to these examples.

[0070] It should be noted that the first intelligent agent performs both intent analysis and requirements analysis simultaneously. Specifically, the requirements analysis for search determination analyzes user input to determine if external information support is needed. This analysis includes: search strategy selection and result verification. Search strategy selection chooses a suitable search engine and search parameters, while result verification verifies the accuracy and relevance of the search results. Similarly, the requirements analysis for knowledge base determination analyzes user input to determine if internal knowledge support is needed. This analysis includes: query strategy selection and result verification. Query strategy selection chooses a suitable query method and query parameters, while result verification verifies the accuracy and relevance of the knowledge base query results. Finally, the requirements analysis for text-to-image generation determination analyzes user input to determine if image generation is necessary. This analysis includes: generation strategy selection and result verification. Generation strategy selection chooses a suitable image generation model and parameters, while result verification verifies the quality and relevance of the generated image.

[0071] It should be noted that the determination content and requirement analysis content of the first intelligent agent described above are only examples in this application embodiment, and the actual function of the first intelligent agent is not limited to the above description.

[0072] In the above embodiments of this application, the first intelligent agent performs intent analysis on text data to determine the target intent corresponding to the text data, including at least one of the following: using the first intelligent agent to identify whether the text data contains a preset instruction; if the text data contains a preset instruction, determining the intent corresponding to the preset instruction as the target intent; using the first intelligent agent to determine whether the text data matches a preset template; if the text data matches the preset template, determining the intent corresponding to the preset template as the target intent; if the text data does not contain a preset instruction, or the text data does not match the preset template, identifying the text data to determine the target intent.

[0073] The aforementioned preset commands can be interactive commands pre-deployed by technicians within the interactive interface, each corresponding to a specific intent. Different preset commands have different specific intents. For example, when the preset command is "deep search," the intent is "deep search intent," while when the preset command is "PPT template," the intent is "generate PPT template intent." When a user selects a preset command, it will be displayed as text in a dialog box within the interactive interface. The user can then add further text to the corresponding text to obtain the desired interactive data.

[0074] The aforementioned preset templates can be text templates with specific intentions set in advance by technical personnel. The specific template type and format can be set by technical personnel according to actual usage needs, and are not limited in this embodiment. Different preset templates correspond to different intentions. For example, when the preset template is for depth search, the intention corresponding to the depth search template is the depth search intention, while when the preset template is for text-to-image, the intention corresponding to the text-to-image template is the text-to-image intention.

[0075] In one optional embodiment, during the intent analysis of text data by the first intelligent agent, the first intelligent agent can also simultaneously identify the text data to determine whether it contains a preset instruction and / or a preset template. If the first intelligent agent determines that the text data contains a preset instruction, the intent corresponding to the preset instruction can be determined as the target intent of the text data. If the first intelligent agent determines that the text data contains a preset template, the intent corresponding to the preset template can be determined as the target intent of the text data. If the first intelligent agent determines that the text data contains both a preset instruction and a preset template, both the intent corresponding to the preset instruction and the intent corresponding to the preset template can be determined as the target intent of the text data. However, if the text data does not contain a preset instruction, or the text data does not match a preset template, the first intelligent agent can identify the text data to determine the target intent.

[0076] In the above embodiments of this application, at least one second intelligent agent is used to perform corresponding processing tasks to obtain processing results corresponding to at least one processing task, including: generating a target prompt word corresponding to any second intelligent agent based on text data; inputting the target prompt word into a second processing model within the second intelligent agent; and using the second processing model to generate processing results corresponding to the processing tasks of the second intelligent agent.

[0077] The target prompts mentioned above are used to help the target processing model generate processing results corresponding to at least one processing task. Target prompts may include, but are not limited to, document prompts and search prompts. The types of the second processing models mentioned above may include, but are not limited to, large language models and visual language models. Different second processing models have different network structures. The processing results mentioned above are the execution results of the second processing model after performing at least one processing task. For example, if the processing task is a search task, the processing result may be the searched text; if the processing tasks are a search task and a text-to-image task, the processing result may be the searched text and an image generated from the text, but is not limited to these.

[0078] In one optional embodiment, the intelligent assistant system can generate target prompt words corresponding to any second intelligent agent based on text data. For example, the intelligent assistant system can first extract keywords from the text data to generate target prompt words corresponding to any second intelligent agent, or it can input text data into a target prompt word template to obtain target prompt words corresponding to any second intelligent agent. Simultaneously, the intelligent assistant system can determine a second processing model based on at least one task to be processed. For example, the intelligent assistant system can first determine the second processing model based on the type of at least one task to be processed. If at least one task to be processed is a text task, the second processing model can be determined to be an LLM (Limited Language Model). If at least one task to be processed includes both text tasks and image, video, or voice tasks, the second processing model can be determined to be a VLM (Video Model Model). Then, the intelligent assistant system can input the target prompt words into the second processing model within at least one second intelligent agent, and the second processing model will generate processing results corresponding to at least one processing task based on the corresponding processing task.

[0079] In the above embodiments of this application, generating a target prompt word corresponding to any second intelligent agent based on text data includes: determining a prompt word template corresponding to the second intelligent agent; and generating a target prompt word based on text data and the prompt word template.

[0080] The aforementioned prompt word template can be a template pre-set by the technician for the corresponding intelligent agent, used to obtain target prompt words from text data. The specific type and format of the prompt word template can be set by the technician according to immediate needs, and is not limited in this embodiment. It should be noted that different intelligent agents correspond to different prompt word templates. The technician can first define multiple slots corresponding to any second intelligent agent, where different slots correspond to different types of key information in the text data. Secondly, the technician can set a prompt word template for any second intelligent agent based on these multiple slots, and use placeholders in the prompt word template to dynamically fill in specific information during actual use.

[0081] In one optional embodiment, given text data and at least one second agent, the intelligent assistant system first obtains a prompt word template corresponding to the second agent. Then, the intelligent assistant system inputs the text data into the prompt word template to generate a target prompt word. For example, the intelligent assistant system can first recognize the text data to obtain at least one key piece of information. Then, the intelligent assistant system can dynamically fill the at least one key piece of information into the corresponding slot in the prompt word template to obtain the target prompt word.

[0082] In the above embodiments of this application, the method further includes: determining a first processing model from multiple data processing models based on the type of text data being processed by at least one processing task, wherein the network structures of different data processing models are different.

[0083] The first processing model described above is used to process at least one processing task to obtain the processing result corresponding to at least one processing task. The network structure differs for different data processing models.

[0084] In an optional embodiment, the intelligent assistant system can further determine a first processing model from multiple data processing models based on the type of text data being processed by at least one processing task. For example, if the processing task is a text task, the first processing model can be determined to be an LLM model, while if the processing task includes both text tasks and image, video, or voice tasks, the first processing model can be determined to be a VLM model.

[0085] In the above embodiments of this application, the first intelligent agent integrates the processing results corresponding to at least one processing task to generate target data, including: inputting the processing results corresponding to at least one processing task into a first processing model within the first intelligent agent, and generating target data using the first processing model, wherein the target data includes data in at least two modalities: text, image, video, and audio.

[0086] The target data mentioned above can be the final result required by the user. The first processing model described above is used to integrate at least one processing result to obtain the target data. The type of the first processing model may include, but is not limited to, large language models and visual language models.

[0087] In one optional embodiment, upon obtaining the processing results corresponding to at least one processing task, the intelligent assistant system can further input the processing results of at least one processing task into a first processing model within the first intelligent entity. The first processing model then integrates the processing results of the at least one processing task to obtain the target data. For example, upon obtaining the searched text corresponding to a search task and the generated image corresponding to a text-to-image task, the intelligent assistant system can input the searched text and the generated image into a first processing model (VLMs) within the first intelligent entity. The first processing model then integrates the searched text and the generated image to obtain a text introducing the video content and a video, i.e., the target data, but is not limited to this. As another example, upon obtaining a PPT template corresponding to a PPT task and the searched text corresponding to a search task, the intelligent assistant system can input the PPT template and the searched text into a first processing model (VLMs) within the first intelligent entity. The first processing model then integrates the PPT template and the searched text to obtain the final PPT required by the user.

[0088] In the above embodiments of this application, the method further includes: during the process of processing text data using at least one second agent, outputting prompt information corresponding to at least one second agent according to a first output method; and when generating target data, determining a second output method based on the modality of the target data, and outputting the target data according to the second output method.

[0089] The first output method described above can be displaying text through a dialog box. The aforementioned prompt message is used to inform the user that text data is being processed. The second method described above can be displaying text through a dialog box, or it can be displaying both text and cards through a dialog box, where the specific content displayed in the cards can be data in other modalities such as video, images, or audio.

[0090] In one alternative embodiment, during the processing of text data using at least one second agent, the intelligent assistant system may output prompts from at least one second agent in the interactive interface in the form of a dialog box displaying text. For example, the intelligent assistant system may display a dialog box containing "Processing" in the interactive interface to prompt the user that the text data is being processed.

[0091] In another alternative embodiment, upon obtaining the target data, the intelligent assistant system can first determine the modality of the target data. If the target data contains only text, the intelligent assistant system can display the target data as text in a dialog box within the interactive interface. If the target data contains multiple modalities, the intelligent assistant system can display the text data as text in a dialog box and display other modal data besides text data, such as video, images, or voice, in card format.

[0092] This application aims to propose a technical solution for a multimodal intelligent assistant system based on the Agent paradigm. This solution fully utilizes the latest Large Language Models (LLMs) and Visual Language Models (VLMs) technologies to construct an efficient and intelligent multimodal interaction platform. The multimodal intelligent assistant system aims to possess the following functions: Efficiency: Achieving rapid response and reducing user waiting time. Accuracy: Improving the precision of information processing and understanding, and reducing misjudgments. Naturalness: Providing a smooth and natural interactive experience, making users feel as if they are communicating with a real person. Personalization: Providing customized services based on user habits and preferences.

[0093] Figure 3 This is a schematic diagram of an agent-based interaction according to an embodiment of this application, such as... Figure 3 As shown, this includes the server-side and the algorithm-side. The specific interaction process is as follows:

[0094] After the server obtains the text data, it first identifies the text data to determine whether the text data contains a preset instruction (i.e., ...). Figure 3 (mandatory instructions in the text) and / or preset templates (i.e.) Figure 3 In the system prompt, the server can send text data and recognition results to the algorithm. After obtaining the text data, the algorithm can first perform intent analysis through the first intelligent agent to obtain the target intent, such as... Figure 3 As shown, target intents include: search intent, document intent, text-to-graph intent, schema selection, and knowledge enhancement intent.

[0095] Secondly, at least one task to be processed can be generated based on the target intent, and then the at least one task to be processed can be input into the corresponding at least one second agent, such as Figure 3 As shown, at least one second intelligent agent includes: a search intelligent agent, a text-to-image intelligent agent, a code intelligent agent, and a document intelligent agent, etc. Then, the second processing model corresponding to at least one second intelligent agent processes at least one task to be processed. At this time, the intelligent assistant system can display the prompt information corresponding to at least one first intelligent agent in the interactive interface of the server in a first output manner.

[0096] Before the target processing model corresponding to at least one second agent processes at least one task to be processed, it is also necessary to generate at least one target prompt word corresponding to the second agent based on the text data, such as Figure 3 As shown, target suggestions can include text suggestions, search suggestions, etc. Additionally, a second processing model needs to be determined based on at least one task to be processed, such as... Figure 3 As shown, the second processing model can be either a large language model or a visual language model.

[0097] Finally, having obtained the processing results for at least one task, the target data can be obtained through the first processing model, and can be displayed in the server's interactive interface in a second output manner, such as... Figure 3 As shown, the second output method is to display text and cards through a dialog box.

[0098] In an agent-based multimodal intelligent assistant system, each agent is responsible for processing a specific type of task or modal information. Through collaboration among agents, the fusion and processing of multimodal information are achieved. The following is a detailed explanation of the main agents included in the system and their functions:

[0099] Search Agent: Function: The search agent is responsible for retrieving relevant information from the internet or other data sources to answer user questions or meet user needs. Specific tasks: Keyword extraction: Extracting key information from user input to generate search keywords. Search engine invocation: Using search engine APIs (such as Google Search API, Bing Search API, etc.) to perform searches. Result filtering: Filtering the most relevant information from search results and removing irrelevant or low-quality content. Result presentation: Presenting the filtered information to the user in a user-friendly way, such as summaries, link lists, etc.

[0100] Knowledge Base Agent: Functions: The Knowledge Base Agent manages and queries the internal knowledge base, providing structured and semi-structured knowledge information. Specific Tasks: Knowledge Base Maintenance: Regularly updates and maintains the knowledge base to ensure the accuracy and timeliness of information. Knowledge Query: Retrieves relevant information from the knowledge base based on user input. Knowledge Reasoning: Utilizes technologies such as knowledge graphs to perform logical reasoning and relational deduction, providing more in-depth answers. Knowledge Recommendation: Recommends relevant knowledge items based on the user's query history and preferences.

[0101] Planning Capabilities: Planning capability is an important function of the system, involving collaboration among multiple agents to formulate and execute complex tasks. It mainly includes the following decision-making and planning capabilities:

[0102] Search Determination: Functions: Determines whether a web search is needed, and the specific content and scope of the search. Requirements Analysis: Analyzes user input to determine if external information support is required. Search Strategy Selection: Selects an appropriate search engine and search parameters. Result Validation: Verifies the accuracy and relevance of search results.

[0103] Knowledge Base Determination: Function: Determines whether to query the internal knowledge base, and the specific content and scope of the query. Requirements Analysis: Analyzes user input to determine if internal knowledge support is needed. Query Strategy Selection: Selects the appropriate query method and parameters. Result Validation: Verifies the accuracy and relevance of the knowledge base query results.

[0104] Base Model Determination: Function: Determines whether the base model used is LLM or VLM.

[0105] Image Generation Decision: Function: Determines whether an image needs to be generated, and the specific content and style of the generated image. Requirements Analysis: Analyzes user input to determine if image generation is necessary. Generation Strategy Selection: Selects an appropriate image generation model and parameters. Result Validation: Verifies the quality and relevance of the generated image.

[0106] To achieve the fusion and processing of multimodal information, an effective collaboration mechanism is needed between different agents: Task allocation: Assign tasks to the appropriate agents based on the type and content of user input. Information sharing: Exchange of information between agents through message passing or shared memory. Result integration: Integrate the results processed by different agents to form the final output.

[0107] Here is an example, suppose a user asks: "How tall are pine trees usually?"

[0108] Search Agent: Extract keywords: "pine tree" and "height". Call the search engine API to perform a search. Filter and present the most relevant information, such as "The height of Korean pine is 30-40 meters, and the height of Masson pine is 20-30 meters".

[0109] Knowledge Base Agent: Queries the internal knowledge base to confirm the height information of different types of pine trees. If the relevant information is found in the knowledge base, it returns the answer directly.

[0110] Base model determination: Select text LLM.

[0111] Planning Capabilities: Search Decision: Analyze user input to determine if a web search is needed. Knowledge Base Decision: Check if relevant information exists in the knowledge base. Image Decision: If the user further requests an image, generate an image of the Eiffel Tower.

[0112] Through this agent-based architecture, the system can efficiently process multimodal information and provide an accurate and natural interactive experience.

[0113] This application's embodiments enable efficient search and knowledge base management: Intelligent Search: The search agent combines keyword extraction and search engine APIs to provide accurate search results. Knowledge Base Maintenance: The knowledge base agent regularly updates and maintains the knowledge base to ensure the accuracy and timeliness of information. Knowledge Reasoning: Utilizing large model technology, logical reasoning and relational deduction are performed to provide more in-depth answers.

[0114] The innovations of this application's embodiments include: introducing intelligent planning capabilities, formulating and executing complex tasks through collaboration among multiple agents, and dynamically adjusting based on user feedback. Specifically: Search determination: analyzing user input to determine whether a web search is needed, and selecting a suitable search engine and search parameters. Knowledge base determination: checking if relevant information exists in the knowledge base, and selecting a suitable query method and query parameters. Image generation determination: analyzing user input to determine whether an image needs to be generated, and selecting a suitable image generation model and parameters.

[0115] According to an embodiment of this application, a data processing method is also provided. Figure 4 This is a flowchart of a data processing method according to an embodiment of this application. Figure 4 As shown, the method may include the following steps:

[0116] Step S402: Respond to the input command applied to the operation interface and determine the text data corresponding to the input command;

[0117] Step S404: Use the first intelligent agent to perform intent analysis on the text data and generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data.

[0118] Step S406: Assign at least one processing task to at least one second agent, and use at least one second agent to execute the corresponding processing task to obtain the processing result corresponding to at least one processing task, wherein at least one processing task corresponds one-to-one with at least one second agent.

[0119] Step S408: The first intelligent agent integrates the processing results corresponding to at least one processing task to generate target data.

[0120] Step S4010: Output the target data.

[0121] Figure 5 This is a schematic diagram of an operation interface according to an embodiment of this application. Figure 5 As shown, the operation interface 51 includes a display area 52 and an input area 53. During user interaction through the intelligent assistant system, the user can first input text data containing their intent in the input area of ​​the operation interface. The operation interface responds to the received text data by generating an input command. Based on this command, the interface determines the corresponding text data. Next, a first intelligent agent performs intent analysis on the text data, generating at least one processing task. This task is then assigned to at least one second intelligent agent, which executes the corresponding task to obtain the processing result. The intelligent assistant system then integrates the processing results from the at least one processing task using the first intelligent agent to generate target data. Finally, the target data is output and displayed in the display area.

[0122] According to an embodiment of this application, a data processing method is also provided. Figure 6 This is a flowchart of a data processing method according to an embodiment of this application. Figure 6 As shown, the method may include the following steps:

[0123] Step S602: Obtain text data by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes text data;

[0124] Step S604: Use the first intelligent agent to perform intent analysis on the text data and generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data.

[0125] Step S606: Assign at least one processing task to at least one second intelligent agent, and use at least one second intelligent agent to execute the corresponding processing task to obtain the processing result corresponding to at least one processing task, wherein at least one processing task corresponds one-to-one with at least one second intelligent agent.

[0126] Step S608: The first intelligent agent integrates the processing results corresponding to at least one processing task to generate target data.

[0127] Step S6010: Output target data by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target data.

[0128] The first interface mentioned above can be an interface for obtaining text data from the user interface of the intelligent assistant system. The second interface mentioned above can be an interface for outputting target data to the user interface.

[0129] In one optional embodiment, during the interaction between the user and the intelligent assistant system, the user can first call a first interface to obtain text data, then use a first intelligent agent to perform intent analysis on the text data to generate at least one processing task, then assign the at least one processing task to at least one second intelligent agent, and use the at least one second intelligent agent to execute the corresponding processing task to obtain the processing result corresponding to the at least one processing task, then use the first intelligent agent to integrate the processing results corresponding to the at least one processing task to generate target data, and finally output the target data by calling a second interface.

[0130] The first interface includes a first parameter, the value of which includes text data. Different processing tasks are used to perform different types of processing on the text data. The target data includes data in at least two modalities: text, image, video, and sound. The second interface includes a second parameter, the value of which includes the target data.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0132] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0134] According to an embodiment of this application, a data processing system is also provided. Figure 7 This is a schematic diagram of a data processing system according to an embodiment of this application. Figure 7 As shown, the data processing system 72 includes: a client 74 and a server 76, wherein, as Figure 7 As shown, client 74 communicates with server 76, and server 76 includes: a first intelligent agent 76-1 and at least one second intelligent agent 76-2 (in... Figure 7 Only one is shown in the image.

[0135] The client is used to send text data; the server is used to perform intent analysis on the text data using a first agent, generate at least one processing task, assign the at least one processing task to at least one second agent, and use the at least one second agent to execute the corresponding processing task to obtain the processing result corresponding to the at least one processing task. The server then uses the first agent to integrate the processing results corresponding to the at least one processing task to generate target data. Different processing tasks are used to perform different types of processing on the text data, and at least one processing task corresponds one-to-one with at least one second agent. The client is also used to output the target data.

[0136] According to an embodiment of this application, a data processing system is also provided. Figure 8 This is a schematic diagram of a data processing system according to an embodiment of this application. Figure 8 As shown, the data processing system 72 includes: a plurality of intelligent agents 82, wherein the plurality of intelligent agents 82 includes a first intelligent agent 76-1 and at least one second intelligent agent (in Figure 8 Only one is shown in the image) 76-2.

[0137] The system comprises multiple intelligent agents, including a first intelligent agent and at least one second intelligent agent. The first intelligent agent performs intent analysis on the text data, generates at least one processing task, and assigns the at least one processing task to at least one second intelligent agent. Different processing tasks are used to perform different types of processing on the text data. At least one second intelligent agent executes the corresponding processing task to obtain the processing result corresponding to the at least one processing task. The at least one processing task corresponds one-to-one with at least one second intelligent agent. The first intelligent agent also integrates the processing results corresponding to the at least one processing task to generate target data.

[0138] According to an embodiment of this application, a data processing apparatus for implementing the above-described data processing method is also provided. Figure 9 This is a schematic diagram of a data processing apparatus according to an embodiment of this application, such as... Figure 9 As shown, the device includes: an acquisition module 92, an analysis module 94, a processing module 96, and an integration module 98.

[0139] The system comprises the following modules: an acquisition module for acquiring text data; an analysis module for performing intent analysis on the text data using a first agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; a processing module for assigning at least one processing task to at least one second agent and using at least one second agent to execute the corresponding processing task to obtain the processing result corresponding to at least one processing task, wherein at least one processing task corresponds one-to-one with at least one second agent; and an integration module for integrating the processing results corresponding to at least one processing task using the first agent to generate target data.

[0140] It should be noted that the acquisition module 92, analysis module 94, processing module 96, and integration module 98 mentioned above correspond to steps S202 to S208 in the above embodiments. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. The above modules can also be part of the device and run in the server 10 provided in the above embodiments.

[0141] In the above embodiments of this application, the analysis module includes an analysis unit and a generation unit.

[0142] The analysis unit is used to perform intent analysis on the text data using the first intelligent agent to determine at least one target intent corresponding to the text data; the generation unit is used to generate a processing task that matches the target intent based on any target intent.

[0143] In the above embodiments of this application, the analysis unit includes at least one of the following: a first determining subunit, a second determining subunit, and a third determining subunit.

[0144] The first determining subunit is used to identify whether the text data contains a preset instruction using a first intelligent agent. If the text data contains a preset instruction, the intent corresponding to the preset instruction is determined to be the target intent. The second determining subunit is used to determine whether the text data matches a preset template using the first intelligent agent. If the text data matches the preset template, the intent corresponding to the preset template is determined to be the target intent. The third determining subunit is used to identify the text data and determine the target intent if the text data does not contain a preset instruction or does not match the preset template.

[0145] In the above embodiments of this application, the processing module includes: a first generation unit and a second generation unit.

[0146] The first generation unit is used to generate a target prompt word corresponding to any second agent based on text data; the second generation unit is used to input the target prompt word into the second processing model in the second agent and use the second processing model to generate the processing result corresponding to the processing task of the second agent.

[0147] In the above embodiments of this application, the first generation unit includes: a fourth determining subunit and a generation subunit.

[0148] The fourth determining subunit is used to determine the prompt word template corresponding to the second agent; the generating subunit is used to generate the target prompt word based on the text data and the prompt word template.

[0149] In the above embodiments of this application, the processing module further includes a determination unit.

[0150] The determining unit determines the first processing model from multiple data processing models based on the type of text data processed by at least one processing task, wherein different data processing models have different network structures.

[0151] In the above embodiments of this application, the integration module includes a processing unit.

[0152] The processing unit is used to input the processing result corresponding to at least one processing task into the first processing model in the first intelligent body, and use the first processing model to generate target data, wherein the target data includes data in at least two modalities: text, image, video and audio.

[0153] In the above embodiments of this application, the device further includes: a first output module and a second output module.

[0154] The first output module is used to output prompt information corresponding to at least one second agent in accordance with the first output method during the process of processing text data using at least one second agent; the second output module is used to determine the second output method based on the modality of the target data when the target data is generated, and output the target data in accordance with the second output method.

[0155] According to an embodiment of this application, a data processing apparatus for implementing the above-described data processing method is also provided. Figure 10 This is a schematic diagram of a data processing apparatus according to an embodiment of this application, such as... Figure 10 As shown, the device includes: a determination module 1002, an analysis module 1004, a processing module 1006, an integration module 1008, and an output module 1010.

[0156] The system comprises the following modules: a determination module, which responds to input commands applied to the user interface and determines the corresponding text data; an analysis module, which uses a first agent to perform intent analysis on the text data and generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; a processing module, which assigns at least one processing task to at least one second agent and uses at least one second agent to execute the corresponding processing task to obtain the processing result corresponding to at least one processing task, wherein at least one processing task corresponds one-to-one with at least one second agent; an integration module, which uses the first agent to integrate the processing results corresponding to at least one processing task to generate target data; and an output module, which outputs the target data.

[0157] It should be noted that the aforementioned determining module 1002, analysis module 1004, processing module 1006, integration module 1008, and output module 1010 correspond to steps S402 to S4010 in the above embodiments. The five modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the aforementioned modules or units can be hardware or software components stored in memory and processed by one or more processors. These modules can also run as part of a device within the server 10 provided in the above embodiments.

[0158] According to an embodiment of this application, a data processing apparatus for implementing the above-described data processing method is also provided. Figure 11 This is a schematic diagram of a data processing apparatus according to an embodiment of this application, such as... Figure 11 As shown, the device includes: an acquisition module 1102, an analysis module 1104, a processing module 1106, an integration module 1108, and an output module 11010.

[0159] The system comprises the following modules: an acquisition module for acquiring text data by calling a first interface, wherein the first interface includes a first parameter whose value includes the text data; an analysis module for performing intent analysis on the text data using a first agent to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; a processing module for assigning at least one processing task to at least one second agent and using at least one second agent to execute the corresponding processing task to obtain the processing result corresponding to at least one processing task, wherein at least one processing task corresponds one-to-one with at least one second agent; an integration module for integrating the processing results corresponding to at least one processing task using the first agent to generate target data; and an output module for outputting the target data by calling a second interface, wherein the second interface includes a second parameter whose value includes the target data.

[0160] It should be noted that the acquisition module 1102, analysis module 1104, processing module 1106, integration module 1108, and output module 11010 mentioned above correspond to steps S602 to S6010 in the above embodiments. The five modules and their corresponding steps implement the same examples and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. The above modules can also run as part of the device in the server 10 provided in the above embodiments.

[0161] It should be noted that the preferred embodiments involved in the above embodiments of this application are the same as the solutions, application scenarios and implementation processes provided in the above embodiments, but are not limited to the solutions provided in the above embodiments.

[0162] Embodiments of this application may provide a computing device. Figure 12 This is a structural block diagram of a computing device according to an embodiment of this application. Figure 12 As shown, the computing device 1200 may include: one or more (only one is shown in the figure) processors 1202, memory 1204, memory controller, and peripheral interfaces.

[0163] The aforementioned computing device can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, PCs (Personal Computers), all-in-one model machines, mobile phones, tablets, or other portable smart terminals, and the computing device may have the model described in the above embodiments of this application pre-installed.

[0164] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing API calling capabilities, allowing models to be called into created applications through API interfaces, and providing application management tools for application management and monitoring.

[0165] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0166] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0167] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.

[0168] Embodiments of this application may provide an electronic device. Figure 13 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 13As shown, the electronic device may include: an input / output device 1302; a memory 1304; and a processor 1306, wherein the processor 1306 is connected to the input / output device 1302 and the memory 1304 via a bus 1308.

[0169] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0170] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.

[0171] It will be understood by those skilled in the art that the structure shown in the figure above is merely illustrative, and the computing device may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. This figure does not limit the structure of the aforementioned computing device. For example, the computing device may include more or fewer components (such as network interfaces, display devices, etc.) than shown in the figure above, or may have a different configuration than that shown in the figure.

[0172] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0173] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store program code executed by the method provided in the above embodiments.

[0174] Optionally, in this embodiment, the storage medium may be located in a computing device.

[0175] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, which, when the executable program is running, controls the device where the computer-readable storage medium is located to execute the method described in any of the above embodiments.

[0176] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.

[0177] Embodiments of this application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0178] Embodiments of this application also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the method provided in the above embodiments.

[0179] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0183] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0184] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, include: Get text data; The first intelligent agent performs intent analysis on the text data to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; The at least one processing task is assigned to at least one second intelligent agent, and the at least one second intelligent agent executes the corresponding processing task to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; The first intelligent agent integrates the processing results corresponding to the at least one processing task to generate target data.

2. The method according to claim 1, characterized in that, The step of using a first intelligent agent to perform intent analysis on the text data and generate at least one processing task includes: The first intelligent agent is used to perform intent analysis on the text data to determine at least one target intent corresponding to the text data. Generate a processing task that matches any given target intent.

3. The method according to claim 2, characterized in that, The step of using the first intelligent agent to perform intent analysis on the text data and determine the target intent corresponding to the text data includes one of the following: The first intelligent agent is used to identify whether the text data contains a preset instruction. If the text data contains the preset instruction, the intent corresponding to the preset instruction is determined to be the target intent. The first intelligent agent is used to determine whether the text data matches the preset template. If the text data matches the preset template, the intent corresponding to the preset template is determined to be the target intent. If the text data does not contain the preset instruction, or if the text data does not match the preset template, the text data is identified to determine the target intent.

4. The method according to any one of claims 1 to 3, characterized in that, The step of using the at least one second agent to execute corresponding processing tasks and obtain processing results corresponding to the at least one processing task includes: Based on the text data, generate target prompt words corresponding to any second agent; The target prompt is input into the second processing model within the second intelligent agent, and the second processing model is used to generate the processing result corresponding to the processing task of the second intelligent agent.

5. The method according to claim 4, characterized in that, The step of generating a target prompt word corresponding to any second agent based on the text data includes: Determine the prompt word template corresponding to the second agent; The target prompt word is generated based on the text data and the prompt word template.

6. The method according to claim 4, characterized in that, The method further includes: Based on the type of text data processed by the at least one processing task, the first processing model is determined from multiple data processing models, wherein the network structures of different data processing models are different.

7. The method according to any one of claims 1 to 3, characterized in that, The step of integrating the processing results corresponding to the at least one processing task using the first intelligent agent to generate the target data includes: The processing results corresponding to the at least one processing task are input into the first processing model within the first intelligent body, and the target data is generated using the first processing model. The target data includes data in at least two modalities: text, image, video, and audio.

8. The method according to any one of claims 1 to 3, characterized in that, The method further includes: During the process of processing the text data using the at least one second intelligent agent, the prompt information corresponding to the at least one second intelligent agent is output according to the first output method; When the target data is generated, a second output method is determined based on the modality of the target data, and the target data is output according to the second output method.

9. A data processing method, characterized in that, include: In response to an input command applied to the user interface, determine the text data corresponding to the input command; The first intelligent agent performs intent analysis on the text data to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; The at least one processing task is assigned to at least one second intelligent agent, and the at least one second intelligent agent executes the corresponding processing task to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; The first intelligent agent integrates the processing results corresponding to the at least one processing task to generate target data. Output the target data.

10. A data processing method, characterized in that, include: Text data is obtained by calling a first interface, wherein the first interface includes a first parameter, and the value of the first parameter includes the text data; The first intelligent agent performs intent analysis on the text data to generate at least one processing task, wherein different processing tasks are used to perform different types of processing on the text data; The at least one processing task is assigned to at least one second intelligent agent, and the at least one second intelligent agent executes the corresponding processing task to obtain the processing result corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; The first intelligent agent integrates the processing results corresponding to the at least one processing task to generate target data. The target data is output by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target data.

11. A data processing system, characterized in that, include: The client is used to send text data; The server, connected to the client, is configured to perform intent analysis on the text data using a first intelligent agent, generate at least one processing task, assign the at least one processing task to at least one second intelligent agent, and execute the corresponding processing task using the at least one second intelligent agent to obtain the processing result corresponding to the at least one processing task. The server then integrates the processing results corresponding to the at least one processing task using the first intelligent agent to generate target data. The different processing tasks are used to perform different types of processing on the text data, and the at least one processing task corresponds one-to-one with the at least one second intelligent agent. The client is also used to output the target data.

12. A data processing system, characterized in that, include: Multiple intelligent agents, including a first intelligent agent and at least one second intelligent agent; The first intelligent agent is used to perform intent analysis on the text data, generate at least one processing task, and assign the at least one processing task to the at least one second intelligent agent, wherein different processing tasks are used to perform different types of processing on the text data; The at least one second intelligent agent is used to execute corresponding processing tasks respectively to obtain processing results corresponding to the at least one processing task, wherein the at least one processing task corresponds one-to-one with the at least one second intelligent agent; The first intelligent agent is also used to integrate the processing results corresponding to the at least one processing task to generate target data.

13. A computing device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 10.

14. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor, connected to the memory via a bus, is used to run the program, wherein the program, when running, executes the method according to any one of claims 1 to 10.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 10.

16. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 10.

Images