Intelligent agent operation method and system and computer readable storage medium

Abstract

The invention discloses an agent operation method and system and a computer readable storage medium. Relates to the technical field of artificial intelligence, and the method comprises the steps that multi-modal request data input based on a target account is received, and the multi-modal request data comprises at least one of text data, image data, audio data, video data, point cloud data and sensor data; in the N standby agents, an agent combination is constructed based on the multi-modal request data, N is an integer larger than 0, the agent combination comprises one or M standby agents, and M is an integer larger than 0; and inputting the multi-modal request data into the agent combination to obtain a target operation result, the target operation result including a task instruction decision result and agent operation process information. Through the method and the device, the technical problem that efficient solving of cross-modal tasks is hindered due to the fact that an intelligent agent system structure lacks effective cooperation in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method, system, and computer-readable storage medium for operating an intelligent agent. Background Technology

[0002] In recent years, artificial intelligence technologies, especially deep learning, convolutional neural networks, and large language models, have significantly advanced the level of intelligence in fields such as image recognition, speech synthesis, and text generation. However, current intelligent agent systems have limitations in terms of understanding and generalization capabilities, social collaboration mechanisms, diversity of external interactions, and adaptive evolution. They are often statistical pattern matchers, lacking context transfer and causal reasoning, making it difficult to establish credible decisions in high-risk domains; they employ fixed connection structures, resulting in a lack of dynamic communication and exchange of ideas among agents, limiting the full potential of collective intelligence; their output forms are singular, lacking closed-loop verification mechanisms, and they cannot actively intervene in the physical world for experimental verification; furthermore, their structures are rigid, making it difficult for them to self-adjust and expand according to task requirements after deployment. These technological bottlenecks hinder the application of artificial intelligence in complex tasks and limit its deep integration with the real world.

[0003] There is currently no effective solution to the problem that the lack of effective collaboration in the structure of intelligent agent systems in related technologies hinders the efficient solution of cross-modal tasks. Summary of the Invention

[0004] The main objective of this application is to provide an agent operation method, system, and computer-readable storage medium to at least address the technical problem in the related art where the lack of effective cooperation in the structure of agent systems hinders the efficient solution of cross-modal tasks.

[0005] To achieve the above objectives, according to one aspect of this application, a method for operating an intelligent agent is provided. The method includes: receiving multimodal request data based on input from a target account, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data; constructing an agent combination based on the multimodal request data from N backup agents, wherein N is an integer greater than 0, and the agent combination includes one or M backup agents, where M is an integer greater than 0; inputting the multimodal request data into the agent combination to obtain a target operating result, wherein the target operating result includes task instruction decision results and agent operating process information.

[0006] Optionally, among the N backup agents, an agent combination is constructed based on the multimodal request data, including: determining the input length, number of modalities, and semantic ambiguity of the multimodal request data; calculating the complexity of the multimodal request data based on the input length, number of modalities, and semantic ambiguity; determining the combination mode of the agent combination based on the complexity, wherein the combination mode includes individual mode, group mode, and collective mode; and selecting from the N backup agents based on the combination mode to construct the agent combination.

[0007] Optionally, based on complexity, the combination mode of the agent combination is determined, including: if the complexity is less than a first preset complexity threshold, the combination mode is determined to be an individual mode, wherein the individual mode indicates that the agent combination includes a backup agent; or if the complexity is not less than the first preset complexity threshold and less than a second preset complexity threshold, the combination mode is determined to be a group mode, wherein the group mode indicates that the agent combination includes H backup agents, where H is an integer greater than 0; or if the complexity is not less than the second preset complexity threshold, the combination mode is determined to be a cluster mode, wherein the cluster mode indicates that the agent combination includes a backup agent as the operation center and K backup agents as the operation basis, where K is an integer greater than 0.

[0008] Optionally, the N backup agents include L primary agents, P intermediate agents, and Q advanced agents. The number of parameters of the primary agents is less than a first preset parameter threshold, the number of parameters of the intermediate agents is less than a second preset parameter threshold but not less than the first preset parameter threshold, and the number of parameters of the advanced agents is not less than the second preset parameter threshold. The attributes of the backup agents include identifiers, functional labels, influence coefficients, credibility scores, and value function weight vectors. L, P, and Q are integers greater than 0.

[0009] Optionally, the multimodal request data is input into the agent ensemble to obtain the target execution result, including: extracting the task context vector from the multimodal request data; determining J task parameter indicators based on the task context vector, wherein the J task parameter indicators include importance indicator, urgency indicator, innovation indicator, safety indicator, and resource consumption indicator, and J is an integer greater than 0; calculating the participation intensity of all standby agents in the agent ensemble based on the value function weight vector and the J task parameter indicators; and running all standby agents based on the participation intensity to obtain the target execution result.

[0010] Optionally, after inputting multimodal request data into the agent combination and obtaining the target running result, the method further includes at least one of the following: deleting agent networks in the agent combination whose gradient change amplitude is less than a preset change threshold for a preset number of consecutive runs; after each preset number of runs, replicating agents whose performance meets preset standards and subjecting them to random perturbation to form W new backup agents, where W is an integer greater than 0; generating new backup agents when the clustering distance between the N backup agents is greater than a preset distance threshold; and weighting and summing the accuracy loss, execution efficiency loss, and diversity loss based on preset weights to obtain the value of the loss function, where the loss function is used to guide the adjustment of the neuron parameters of the backup agents.

[0011] To achieve the above objectives, according to another aspect of this application, an intelligent agent operation device is provided. The device includes: a receiving module for receiving multimodal request data input based on a target account, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data; a construction module for constructing an intelligent agent combination from N backup intelligent agents based on the multimodal request data, wherein N is an integer greater than 0, and the intelligent agent combination includes one or M backup intelligent agents, where M is an integer greater than 0; and an input module for inputting the multimodal request data into the intelligent agent combination to obtain a target operation result, wherein the target operation result includes task instruction decision results and intelligent agent operation process information.

[0012] Optionally, the construction module further includes: a first determining unit, a calculation unit, a second determining unit, and a construction unit, wherein the first determining unit is used to determine the input length, number of modalities, and semantic ambiguity of the multimodal request data; the first calculation unit is used to calculate the complexity of the multimodal request data based on the input length, number of modalities, and semantic ambiguity; the second determining unit is used to determine the combination mode of the agent combination based on the complexity, wherein the combination mode includes individual mode, group mode, and collective mode; and the construction unit is used to select from N backup agents based on the combination mode to construct the agent combination.

[0013] Optionally, the second determining unit further includes: a third determining unit, a fourth determining unit, and a fifth determining unit, wherein the third determining unit is used to determine the combination mode as an individual mode when the complexity is less than a first preset complexity threshold, wherein the individual mode represents that the agent combination includes one backup agent; the fourth determining unit is used to determine the combination mode as a group mode, or when the complexity is not less than the first preset complexity threshold and less than the second preset complexity threshold, wherein the group mode represents that the agent combination includes H backup agents, where H is an integer greater than 0; the fifth determining unit is used to determine the combination mode as a cluster mode, or when the complexity is not less than the second preset complexity threshold, wherein the cluster mode represents that the agent combination includes one backup agent as the operation center and K backup agents as the operation basis, where K is an integer greater than 0.

[0014] Optionally, the N backup agents in the input module include L primary agents, P intermediate agents, and Q advanced agents. The number of parameters of the primary agents is less than a first preset parameter threshold, the number of parameters of the intermediate agents is less than a second preset parameter threshold but not less than the first preset parameter threshold, and the number of parameters of the advanced agents is not less than the second preset parameter threshold. The attributes of the backup agents include identifiers, function labels, influence coefficients, credibility scores, and value function weight vectors, where L, P, and Q are integers greater than 0.

[0015] Optionally, the input module further includes: an extraction unit, a sixth determination unit, a second calculation unit, and a running unit, wherein the extraction unit is used to extract the task context vector from the multimodal request data; the sixth determination unit is used to determine J task parameter indicators based on the task context vector, wherein the J task parameter indicators include importance indicator, urgency indicator, innovation indicator, safety indicator, and resource consumption indicator, and J is an integer greater than 0; the second calculation unit is used to calculate the participation intensity of all backup agents in the agent combination based on the value function weight vector and the J task parameter indicators; and the running unit is used to run all backup agents based on the participation intensity to obtain the target running result.

[0016] Optionally, the above device further includes: a deletion unit, a copying unit, a generation unit, and a weighting unit, wherein the deletion unit is used to delete agent networks in the agent combination whose gradient change amplitude is less than a preset change threshold for a preset number of consecutive runs; the copying unit is used to copy agents whose performance meets a preset standard after each preset number of runs of tasks are completed, and to randomly perturb them to form W new backup agents, where W is an integer greater than 0; the generation unit is used to generate new backup agents when the clustering distance between N backup agents is greater than a preset distance threshold; and the weighting unit is used to weight and sum the accuracy loss, execution efficiency loss, and diversity loss based on preset weights to obtain the value of the loss function, wherein the loss function is used to guide the adjustment of the neuron parameters of the backup agents.

[0017] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to execute any of the above-described intelligent agent operation methods.

[0018] To achieve the above objectives, according to another aspect of this application, an electronic device is also provided, including a memory storing an executable program; and a processor for running the program, wherein the program executes any of the above-described intelligent agent operation methods during runtime.

[0019] To achieve the above objectives, according to another aspect of this application, a computer program product is also provided, including computer instructions that, when executed by a processor, implement the steps of any of the above-described intelligent agent operation methods.

[0020] In this embodiment of the invention, an intelligent agent operation method is employed. This method receives multimodal request data input based on a target account. The multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data. Based on the multimodal request data, an intelligent agent combination is constructed from N backup intelligent agents, where N is a positive integer, and the intelligent agent combination includes one or M backup intelligent agents, where M is a positive integer. The multimodal request data is input into the intelligent agent combination to obtain the target operation result, which includes task instruction decision results and intelligent agent operation process information. This achieves the goal of dynamically constructing the most suitable intelligent agent combination, thereby improving the generalization ability, decision-making efficiency, and task execution reliability of the intelligent agent system. Furthermore, it solves the technical problem in related technologies where the intelligent agent system structure lacks effective collaboration, hindering the efficient resolution of cross-modal tasks. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0022] Figure 1 A hardware structure block diagram of a computer terminal for implementing an intelligent agent operation method is shown.

[0023] Figure 2 This is a flowchart of an intelligent agent operation method provided according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of an intelligent agent organizational morphology provided according to an optional embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the architecture of an intelligent agent operating system provided according to an optional embodiment of this application;

[0026] Figure 5 This is a schematic diagram of an intelligent agent operating device provided according to an embodiment of this application;

[0027] Figure 6 This is a structural block diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] It should be noted that the information collected in this application (including but not limited to user instruction information, account personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0031] Example 1

[0032] According to an embodiment of this application, an embodiment of an intelligent agent operation method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing an intelligent agent operation method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0034] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0035] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the intelligent agent operation method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned intelligent agent operation method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0037] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0038] Under the aforementioned operating environment, this application provides the following: Figure 2 The method of intelligent agent operation is shown. Figure 2 This is a flowchart of an intelligent agent operation method provided according to an embodiment of this application.

[0039] Step S201: Receive multimodal request data based on target account input, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data.

[0040] This step first identifies and parses diverse information input from specific users, covering a range of data types from traditional text, images, and audio to more advanced video, point clouds, and sensor readings. Users can express their needs or queries in any combination of methods, such as describing a complex business problem in text while uploading relevant images or videos as supplementary material, or requesting system analysis and prediction via sensor stream data in real-time monitoring scenarios. By establishing a unified semantic expression space and mapping multiple modal languages ​​into a unified system language, the "language barrier" between modalities can be resolved. This design breaks the limitations of a single modality, making intelligent services closer to natural human communication, and improving user experience and the flexibility and complexity of task handling.

[0041] Step S202: Among the N backup agents, construct an agent combination based on the multimodal request data, where N is an integer greater than 0, and the agent combination includes one or M backup agents, where M is an integer greater than 0.

[0042] In this step, when processing multimodal request data based on target account input, an agent pool is dynamically constructed from a pre-set pool of N backup agents (N being a positive integer representing the size of the agent pool) based on the specific requirements and complexity of the request content. This pool can include a single agent for simpler or more straightforward tasks, or M agents (M also a positive integer) to handle complex requests requiring cross-modal understanding and multi-domain knowledge fusion. The process of constructing the pool may involve matching agent capability labels, evaluating value functions, and prioritizing tasks, aiming to efficiently allocate resources and ensure that the agent group can best understand user intent and collaboratively solve problems. By simulating the division of labor and cooperation in human society, the overall response speed and task processing accuracy are improved, while knowledge transfer and experience accumulation among agents are promoted, achieving continuous optimization of system performance.

[0043] Step S203: Input the multimodal request data into the agent combination to obtain the target operation result, wherein the target operation result includes the task instruction decision result and the agent operation process information.

[0044] In this step, multimodal request data is passed to the constructed agent ensemble. This ensemble comprises agents best suited for the task, potentially possessing capabilities such as text understanding, image analysis, and speech recognition. The agent ensemble exchanges information and debates viewpoints through internal communication and consensus mechanisms, centralizing information in a shared memory pool. Agents with higher influence receive greater weight to drive the decision-making process. Ultimately, the agent ensemble reaches a consensus, generating a task instruction decision, while simultaneously recording complete process information, including the list of participating agents, key points of contention, logical reasoning chains, and sources of evidence. This process not only achieves a deep understanding and efficient processing of user requests but also ensures transparency and traceability of the decision-making process. The final target execution result guides subsequent actions and provides valuable data for the agent system's learning and self-optimization, strengthening collaboration among agents and enhancing the overall system efficiency.

[0045] Optionally, in the agent operation method provided in this application embodiment, an agent combination is constructed based on multimodal request data among N backup agents, including: determining the input length, number of modalities, and semantic ambiguity of the multimodal request data; calculating the complexity of the multimodal request data based on the input length, number of modalities, and semantic ambiguity; determining the combination mode of the agent combination based on the complexity, wherein the combination mode includes individual mode, group mode, and collective mode; and selecting from the N backup agents based on the combination mode to construct the agent combination.

[0046] First, the received multimodal request data based on the target account can be preliminarily analyzed to assess its input length, the number of modalities involved, and the complexity or ambiguity of its semantics, thereby quantifying the task's complexity. Then, based on the calculated complexity index, the agent combination mode is dynamically determined: for simple tasks, an individual mode is activated, where a single high-level agent can complete the task; for medium-complexity tasks, a team mode is activated, automatically forming a temporary team of 3 to 7 agents; and for extremely complex or cross-domain tasks, a group mode is required, where the agent network constructs a virtual company with functional departments, designating a "CEO agent" for overall management. After the mode is selected, the most suitable agent group for the current task requirements is intelligently selected and combined from N backup agents (N being the size of the agent pool, a positive integer), ensuring efficient resource utilization and successful task execution, while also promoting knowledge sharing among agents. This mechanism achieves precise matching between agent combination and task complexity, improving the overall efficiency and quality of processing multimodal request data.

[0047] Optionally, in the agent operation method provided in this application embodiment, determining the combination mode of the agent combination based on complexity includes: determining the combination mode as an individual mode when the complexity is less than a first preset complexity threshold, wherein the individual mode indicates that the agent combination includes one backup agent; or determining the combination mode as a group mode when the complexity is not less than the first preset complexity threshold and less than a second preset complexity threshold, wherein the group mode indicates that the agent combination includes H backup agents, where H is an integer greater than 0; or determining the combination mode as a cluster mode when the complexity is not less than the second preset complexity threshold, wherein the cluster mode indicates that the agent combination includes one backup agent as the operation center and K backup agents as the operation basis, where K is an integer greater than 0.

[0048] When the complexity of multimodal request data is below a first preset threshold (e.g., 3), the individual mode can be selected. This means that only one high-level agent is needed to complete the task independently, demonstrating rapid response and efficient processing capabilities for simple requests. If the complexity is between the first and second preset thresholds (e.g., 7), the team mode is activated, forming a small team of H agents (H is a positive integer, for example, 3 to 7) that collaborate based on their respective expertise. This is suitable for handling tasks that are challenging but not overly complex. When encountering particularly complex or multidimensional tasks, if the complexity reaches or exceeds the second preset threshold, the group mode is activated. In this case, the agent group can be led by a central agent, supplemented by K agents (K is a positive integer) that form a corporate-like organizational structure. Through hierarchical management and deep interaction between agents, the collective wisdom of the entire agent society is mobilized to address the challenges, ensuring that optimal strategies are adopted and solutions are generated efficiently and accurately regardless of the task's complexity.

[0049] The group model can be further subdivided. Figure 3 This is a schematic diagram of an intelligent agent organizational morphology provided according to an optional embodiment of this application, such as... Figure 3 As shown, in addition to the individual model of autonomous decision-making and the group model of majority voting, the group model can also be divided into the functional department model and the group company model. Among them, the functional department model adopts hierarchical command, while the group company model adopts a decision-making form similar to "central leadership + board of directors". The coexistence of these four organizational forms enables the intelligent agent society to flexibly respond to various task challenges, thereby improving the overall intelligence level and task processing capabilities.

[0050] Optionally, in the agent operation method provided in the embodiments of this application, the N backup agents include L primary agents, P intermediate agents, and Q advanced agents. The number of parameters of the primary agents is less than a first preset parameter threshold, the number of parameters of the intermediate agents is less than a second preset parameter threshold but not less than the first preset parameter threshold, and the number of parameters of the advanced agents is not less than the second preset parameter threshold. The attributes of the backup agents include identifiers, functional labels, influence coefficients, credibility scores, and value function weight vectors, where L, P, and Q are integers greater than 0.

[0051] N reserve agents form the foundation of the agent unit pool, which is further subdivided into L primary agents, P intermediate agents, and Q advanced agents, where L, P, and Q are all positive integers, ensuring rich diversity in the types and numbers of agents. Primary agents, due to their small parameter count (below a first preset parameter threshold, e.g., <1M), can quickly respond to and handle simple basic tasks, such as data preprocessing or feature extraction. Intermediate agents, with parameter counts between the first and second preset thresholds (e.g., 1M~50M), are suitable for performing more complex functions and can perform in-depth analysis and decision-making within specific domains. Advanced agents, with parameter counts at least reaching the second preset threshold (e.g., >50M), possess powerful cross-modal understanding and comprehensive reasoning capabilities, making them a key force in handling highly complex tasks. Each agent possesses unique attributes, including an identifier (for unique identification), a functional label (indicating its area of ​​expertise), an influence coefficient (affecting its status and voice in the agent society), a credibility score (dynamically adjusted based on the accuracy of historical tasks), and a value function weight vector (guiding its behavioral decisions). These attributes collectively determine the agent's role and performance in task processing, enabling the agent society to flexibly adapt to various scenarios and operate efficiently.

[0052] Optionally, in the agent operation method provided in this application embodiment, inputting multimodal request data into the agent combination to obtain the target operation result includes: extracting a task context vector from the multimodal request data; determining J task parameter indicators based on the task context vector, wherein the J task parameter indicators include importance indicator, urgency indicator, innovation indicator, safety indicator, and resource consumption indicator, and J is an integer greater than 0; calculating the participation intensity of all standby agents in the agent combination based on the value function weight vector and the J task parameter indicators; and running all standby agents based on the participation intensity to obtain the target operation result.

[0053] Upon receiving multimodal request data, advanced encoding techniques are first employed to extract a task context vector, which encapsulates the core information and intent of the request. Next, based on this vector, J key task parameters are automatically identified and quantified, including the task's importance, urgency, innovation requirements, security considerations, and expected resource consumption, comprehensively assessing the task's characteristics and challenges. Then, each agent's value function weight vector is combined with its respective task parameter to calculate its corresponding participation intensity, reflecting the agent's potential contribution and willingness to participate. Based on the participation intensity, all backup agents in the agent pool are scheduled to work in an orderly manner; low-intensity agents may be temporarily excluded to ensure efficient resource utilization. The agents operate independently, communicating, collaborating, and debating to ultimately integrate their outputs and insights, forming the target operational outcome, including refined task instruction decisions and detailed agent operational process information.

[0054] In addition to determining the scheduling of agents based on the intensity of participation, an attention mechanism can be used to simulate "competition for speaking rights". The attention score of each agent is calculated separately, and the top 30% of agents can obtain the qualification of "key speaker". Agents with the qualification of "key speaker" have higher priority in scheduling.

[0055] Optionally, in the agent operation method provided in this application embodiment, after inputting multimodal request data into the agent combination and obtaining the target operation result, it further includes at least one of the following: deleting agent networks in the agent combination whose gradient change amplitude is less than a preset change threshold for a preset number of consecutive runs; after each preset number of runs, copying agents whose performance meets preset standards and subjecting them to random perturbation to form W new backup agents, where W is an integer greater than 0; generating new backup agents when the clustering distance between N backup agents is greater than a preset distance threshold; and weighted summing the accuracy loss, execution efficiency loss, and diversity loss based on preset weights to obtain the value of the loss function, wherein the loss function is used to guide the adjustment of the neuron parameters of the backup agents.

[0056] After processing the multimodal request data and obtaining the target results, the agent combination can be further optimized to ensure its continuous evolution and efficient operation. This includes four mechanisms: First, structural pruning can be performed. For agent networks with small gradient changes within a preset number of consecutive runs, if their gradient changes are below a preset threshold, they are considered optimization targets and can be removed to reduce redundant computation and maintenance costs. For example, gradient magnitude < 1e -5First, the system freezes agents after 10 consecutive rounds and removes them after 100 rounds. Second, after each preset number of tasks, samples are selected from high-performing agents, replicated, and randomly perturbed to generate W new backup agents (W is a positive integer), expanding the agent pool and promoting model diversity and innovation. For example, the top 5% of agents are replicated every 1000 tasks, with parameter perturbation ±2%. Third, when N backup agents exhibit a large clustering distance exceeding a preset distance threshold, indicating an uncovered functional domain or knowledge blind spot, new backup agents can be automatically generated to fill the gap and enhance overall adaptability. For example, a new agent is created when the clustering distance > 0.95. Fourth, based on three core indicators—accuracy, execution efficiency, and diversity—a comprehensive loss function is generated through weighted summation using preset weights. This function guides the adjustment of neuron parameters in the agent network, promoting the optimization and self-evolution of agent combinations in various complex tasks, ensuring the long-term competitiveness and stability of the system. For example, the mathematical expression of the total loss function can be L. total =0.6L accuracy +0.3L efficiency +0.1L diversity .

[0057] The intelligent agent operation method provided in this application receives multimodal request data input based on a target account. The multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data. Based on the multimodal request data, an intelligent agent combination is constructed from N backup intelligent agents, where N is a positive integer, and the intelligent agent combination includes one or M backup intelligent agents, where M is a positive integer. The multimodal request data is input into the intelligent agent combination to obtain the target operation result, which includes task instruction decision results and intelligent agent operation process information. This achieves the goal of dynamically constructing the most suitable intelligent agent combination, thereby improving the generalization ability, decision-making efficiency, and task execution reliability of the intelligent agent system. Furthermore, it solves the technical problem in related technologies where the intelligent agent system structure lacks effective collaboration, hindering the efficient solution of cross-modal tasks.

[0058] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0059] Example 2

[0060] An optional embodiment of this application also provides an intelligent agent operation system. It should be noted that the intelligent agent operation system of this optional embodiment can apply the intelligent agent operation method provided in the embodiments of this application. The intelligent agent operation system provided in the optional embodiment of this application is described below.

[0061] According to an optional embodiment of this application, a system for implementing the above-described intelligent agent operation method is also provided. Figure 4 This is a schematic diagram of the architecture of an intelligent agent operating system provided according to an optional embodiment of this application. For example... Figure 4 As shown, the system includes:

[0062] The Task Parsing and Organization Module receives multimodal input signals from users, and its output connects to the next module via bidirectional arrows. It is responsible for understanding user needs, assessing task complexity, and assembling the intelligent agent team accordingly. The Intelligent Agent Social Operation Engine Module, the core processing unit, contains four sub-modules: an intelligent agent unit pool, a social organization management layer, a communication and consensus mechanism, and a value function module. Its output connects to the next module via unidirectional arrows, enabling efficient task execution through dynamic communication, collaboration, and decision-making among the internal intelligent agents. The Multimodal Unified Interface Module connects to the previous module at its input and to the next module at its output. It includes a "semantic latent space mapper" to ensure smooth conversion and integration of information from different modalities, eliminating communication barriers between intelligent agents. The Practice Feedback Execution Module connects to the external execution environment at its output and receives feedback data at its input, giving the system the ability to control external devices, design experiments, and obtain real-world feedback, breaking the theoretical limitations of traditional AI. The Self-Evolutionary Optimization Engine Module connects to itself at its output via a feedback channel. Through an innovative adaptive mechanism, it continuously optimizes the structure and parameters of the intelligent agents, driving the intelligent agent swarm towards social optimization. The interpretable output module, with its input connected to the previous module and its output facing the user terminal, provides a transparent view of the task processing, enhancing the credibility and usability of system decisions. All data transmission between modules uses an adapted communication protocol and undergoes serialization format conversion, resulting in a communication latency of <10ms.

[0063] Optionally, the task parsing and organization module, as the first point of contact for the system to receive multimodal requests from users, functions to deeply analyze the intent, urgency, and complexity of the task, and automatically assemble a suitable intelligent agent social organization based on the parsing results. Input sources include raw data such as text, images, audio, video, point clouds, and sensor streams. The module first uses a pre-trained multimodal encoder to extract joint embedding vectors, which is the first step in unifying the representation of multimodal data. Subsequently, the vector data is fed into a classification head for task recognition. The output layer dimension of this classification head is set to 128, and an activation function is used to ensure the interpretability of the output and the rationality of the probability distribution. The complexity score is calculated using a comprehensive function f, which considers the input length, the number of modalities involved, and semantic ambiguity. The score range is set between 0 and 10, retaining one decimal place to reflect accuracy. The urgency level is determined based on the detection of specific keywords (such as "immediate," "urgent," and "dangerous") and the assessment of sentiment polarity. If a keyword is detected and the sentiment polarity is not higher than -0.6, the task is marked as "high" urgency. If the data is received outside of working hours (i.e., outside of 9:00 to 17:00 on weekdays), the urgency level is marked as "medium." In other cases, the urgency level defaults to "low." Based on complexity scores and urgency levels, the organization follows these strategies: When the task complexity is below 3, the "individual agent mode" is activated, where a high-level agent with comprehensive processing capabilities completes the task independently. For tasks with complexity between 3 and 7, the "project team mode" is triggered, where the agent system automatically forms a temporary team of 3 to 7 agents to handle the task collaboratively. When the task complexity reaches or exceeds 7, the system adopts the "corporation company mode," establishing functional departments and appointing a CEO agent to be responsible for the overall management of the task, using a hierarchical organizational structure to address the most complex task challenges.

[0064] Optionally, an intelligent agent social operation engine drives the efficient operation of the intelligent agent unit pool. This unit pool includes at least 100 heterogeneous small intelligent agents, which are finely divided into three levels according to complexity and function, forming a dynamic and hierarchical ecosystem. The lowest level of intelligent agents corresponds to a single-layer neuron cluster or a small multilayer perceptron, characterized by fewer than 1M parameters, rapid response, and latency of less than 5 milliseconds. They undertake basic data processing and preliminary analysis tasks. Intermediate intelligent agents consist of complete sub-models, with parameters ranging from 1M to 50M. They specialize in deep analysis and task execution in specific domains, such as image recognition or text understanding. Their response time is slightly longer, but still within the efficient range. The highest level of intelligent agents has more than 50M parameters and adopts a sliced ​​version similar to a large language model. They possess powerful comprehensive reasoning and decision-making capabilities. They play a key role in handling complex and ever-changing tasks. Although their response time is long, they ensure processing quality and intelligent output.

[0065] Each agent is equipped with a unique set of attributes, including a unique identifier `agent_id` in UUID format to ensure individual uniqueness; a set of capability tags that list the agent's core skills, such as "vision" for visual processing, "reasoning" for reasoning ability, and "emotion" for emotional understanding, clearly defining the agent's functional scope; an influence coefficient `influence_score`, initially set to 1.0 and dynamically adjustable from 0.1 to 10.0, measuring the agent's weight in collective decision-making; a trustworthiness score dynamically updated based on historical performance, updated every 100 tasks completed, reflecting the agent's reliability and accuracy; and finally, a value function weight vector `ovalue_function_weights`, with a fixed dimension of 5, corresponding to the importance, urgency, potential innovation, security considerations, and cost-effectiveness of task execution, guiding the agent's value orientation and decision-making tendencies when performing tasks.

[0066] Optionally, the social organization management layer submodule plays a crucial role in regulating the social forms of intelligent agents within the system. Its core function is to support and manage four different organizational forms, ensuring that the intelligent agent society can flexibly adjust and operate efficiently according to task characteristics. By dynamically adjusting and optimizing these organizational forms, the social organization management layer ensures that the intelligent agent society can cope with various task challenges in the best possible state, achieving efficient management and intelligent emergence of the intelligent agent society. This design approach draws on principles from nature and sociology, providing a solid theoretical foundation and practical guide for the operation of the intelligent agent society, enabling it to demonstrate excellent collaborative capabilities and problem-solving wisdom in complex and ever-changing task environments.

[0067] Optionally, the sub-module communication and consensus mechanism is a crucial component of the intelligent agent social operation engine, aiming to achieve effective communication and consensus building among intelligent agents by simulating the decision-making process of human society. This mechanism employs a meticulously designed three-stage communication protocol. First, in the report submission phase, each participating agent is required to generate a detailed analysis brief. The brief not only covers the basic facts observed by the agent but also includes its reasoning process, suggested conclusions based on the reasoning, and a confidence assessment of the conclusions. This process ensures the transparency and traceability of the agent's decision-making. Next, all agents' briefs are uploaded to a shared memory pool with a maximum capacity of 1GB, providing a foundation for large-scale information exchange. The agents' briefs compete for "speaking rights" through an attention mechanism. Based on attention scores, the top 30% of agents are granted the qualification of "key speakers," playing a more crucial role in subsequent discussions.

[0068] In the debate phase, agents designated as "key speakers" engage in in-depth debate based on their presentations, while other agents can participate in voting or provide feedback. The consensus-building mechanism follows clear decision-making rules: for any given proposition, if the ratio of support to opposition votes exceeds 2:1, the proposition is accepted, guiding the next collective action; if the ratio is between 1:1 and 2:1, the proposition is marked as a "point of contention," indicating significant disagreement. In this case, the system automatically triggers further experimental verification to confirm the proposition's truthfulness or validity. This mechanism ensures transparency and scientific rigor in the agent society's decision-making process. Simultaneously, by quantifying confidence and influence, it optimizes resource allocation and decision-making efficiency, promoting the emergence of collective intelligence and the effective resolution of complex tasks.

[0069] Optionally, the value function module is a key part of the agent social operation engine responsible for agent participation decision-making. Its core function is to quantify the agent's willingness to participate in a specific task, i.e., participation intensity (s), limited to the range [0,1]. This value directly reflects the agent's subjective value judgment and motivation for participation. The module's input is a 256-dimensional task context vector (t), which integrates multi-dimensional information about the task, such as its nature, background, and requirements, providing the agent with a comprehensive understanding of the task.

[0070] The value function module uses a weighted summation formula. , where σ represents the Sigmoid function, used to map the calculation result to the [0,1] interval, ensuring that the output participation strength meets the expected range. In the formula... This represents five trainable weights, which correspond to the agent's emphasis on different attributes of the task, including the importance and urgency of the task, the potential benefits of innovation, the security risks involved, and the expected cost of performing the task. to These are the quantitative indicators corresponding to the five dimensions mentioned above for the task, which together constitute the input vector for the agent to evaluate the value of the task. By adjusting... Intelligent agents can flexibly assign different weights to different task attributes based on their own value preferences and strategies, thereby achieving personalized and intelligent task selection.

[0071] Based on the calculated participation strength *s*, the agent will decide whether to participate in the current task. If *s* is less than 0.3, it indicates that the agent believes the task is inconsistent with its value preferences and resource allocation strategy, or that the risks and costs of the task exceed the expected benefits. In this case, the agent will choose to refuse to participate in the task, thereby avoiding unnecessary resource consumption and risk-taking. This mechanism ensures the optimal allocation of resources in the agent society and allows agents to make autonomous choices based on the comprehensive value of tasks, thus improving the overall operational efficiency of the society and the quality of agent decision-making. Through the above sub-modules, the agent society can allocate tasks and schedule resources in a more rational and efficient manner, achieving a dynamic match between social structure and task value, and providing solid decision support for handling complex tasks.

[0072] Optionally, the multimodal unified interface module is a crucial bridge in this system, dedicated to eliminating the "language barrier" between different modal data and achieving seamless integration and unified understanding of cross-modal information. The main function of this module is to establish a shared semantic expression space, ensuring that regardless of whether the input is text, image, audio, point cloud, or action commands, it can be understood and processed consistently by the intelligent agent society, thereby greatly improving the efficiency and accuracy of multimodal collaboration. Through a carefully designed mapping method, the module successfully constructs a high-precision cross-modal retrieval system with an accuracy of @5 (i.e., the accuracy of the first 5 results) of no less than 85%. This means that no matter what type of modal data the intelligent agent society needs to integrate, it can find relevant information with almost lossless accuracy, thus greatly enhancing the system's multimodal processing capabilities and information retrieval efficiency.

[0073] Optionally, the practice feedback execution module aims to transform the decisions and action plans generated by the agent into real-world effects and feedback, thereby driving the system's learning and evolution. This module is designed with three execution paths, covering a full range of application scenarios from virtual to physical, and from operation to experimental verification. First, for digital world operations, module four enables interaction between the agent and the digital environment by calling communication protocols. The frequency of this process is strictly controlled to no more than 10 times per second, ensuring the stability and controllability of the system's operation, while also providing the agent with opportunities to adjust and improve its strategies. Second, the module extends to the realm of physical world intervention. Through close connection with the robot platform, the agent can command the robot to perform specific physical operations, transforming abstract instructions into actual actions, further expanding the system's scope and problem-solving capabilities. Finally, the addition of experimental design and verification functions enables the agent not only to generate theoretical hypotheses and plans, but also to automatically design and execute experimental schemes, verifying the effectiveness of theories through practice. This closed-loop mechanism not only deepens the agent's understanding of the task but also strengthens its self-correction and optimization capabilities. The results of each execution are returned by the module in a standardized format. The results include a number of key information such as execution status, specific data, operation time, and energy consumption. These details are not only an important basis for the agent to evaluate the efficiency and effectiveness of its own behavior, but also provide rich empirical references for subsequent decision-making.

[0074] Optionally, the self-evolutionary optimization engine, through a series of carefully designed mechanisms, ensures that individual intelligent agents and the entire society they constitute can continuously self-optimize and self-expand to adapt to more complex and changing task environments. This engine includes four key mechanisms, each designed to promote the evolution of intelligent agents and the optimization and upgrading of the intelligent agent society from different perspectives.

[0075] First, the "structural pruning" mechanism takes action against redundant and inefficient parts of the agent's internal model. This is achieved by monitoring the gradient magnitude during training; if the gradient magnitude is found to be below 1 for 10 consecutive rounds... This part of the model structure will be temporarily frozen to prevent it from continuing to consume computing resources. If the situation continues to deteriorate, after 100 rounds of freezing, the structure will be completely removed, thus achieving dynamic slimming of the model structure and efficient use of resources.

[0076] Next, the "genetic variation" mechanism draws on the principles of biological evolution, periodically (after every 1000 tasks) replicating the top 5% of agents in the system, creating new versions of agents through subtle parameter perturbations (±2%), increasing the diversity of the agent society, and also providing a possible gene pool for optimization and innovation.

[0077] Subsequently, the "newborn generator" is responsible for expanding the intelligent agent society. When the cluster distance between intelligent agents is greater than 0.95, it indicates that the modal coverage or functional distribution of the current intelligent agent society has reached a certain degree of differentiation and insufficiency. At this time, the system will automatically start the process of generating new intelligent agents to fill cognitive gaps or enhance existing capabilities, ensuring the comprehensive development and self-improvement of the intelligent agent society.

[0078] Finally, the "backpropagation implementation" mechanism provides a closed-loop guarantee for the optimization of the agent society. Through a comprehensive loss function Ltotal, composed of three sub-objectives—accuracy, efficiency, and diversity—with weights of 0.6, 0.3, and 0.1 respectively, it ensures that while pursuing task-solving accuracy, agents also consider improving efficiency and maintaining diversity, achieving a dynamic balance in multi-objective optimization. This mechanism, through the backpropagation algorithm, progressively transmits the optimization pressure of the three sub-objectives to the neuron parameter level of each agent, prompting agents to continuously adjust and improve themselves during learning and evolution to adapt to increasingly diverse and complex task requirements.

[0079] The self-evolutionary optimization engine constructs a dynamic intelligent agent ecosystem capable of self-adjustment, self-reproduction, and self-learning through four major mechanisms: structural pruning, genetic mutation, newborn generation, and backpropagation implementation. This ensures the adaptability and competitiveness of the intelligent agent society in handling complex tasks, while also promoting the continuous evolution and optimization of the system as a whole.

[0080] Optionally, an interpretable output module is designed to provide users and maintainers with in-depth insights into the system's workflow and decision-making processes, thereby enhancing trust and control over the system. This module's output encompasses a variety of rich and detailed information types, comprehensively showcasing the decision-making logic and participation of the intelligent agent society in task processing.

[0081] Specifically, the output includes a list of all agents participating in the task, which not only shows which agents participated in the task processing but also highlights their roles and contributions within the social structure; a key debate point graph, recording the disagreements and debate focal points that arose during the discussion and decision-making process of the agent society, demonstrating the dynamic process of the collision of collective wisdom and the formation of consensus; a logic chain tree, which clearly depicts the reasoning steps and thought paths of agents from accepting the task to generating the final solution, making the logical links behind the decisions readily apparent; evidence source annotations, providing supporting data and references behind each decision or conclusion, enhancing the credibility and reliability of the output results; and a self-reflection report, which is a critical analysis of the agents' own behavior and decisions, including summaries of experiences, lessons learned, and directions for future improvement during the task processing, reflecting the self-learning and growth mechanism of the agent society.

[0082] To adapt to the needs and usage scenarios of different users, the module also supports multiple output formats, ensuring the readability and accessibility of the output information. This allows users to easily and quickly obtain and understand the operational details of the intelligent agent society, whether browsing web pages, printing and reviewing, or editing code.

[0083] Example 3

[0084] This application also provides an intelligent agent operation device. It should be noted that the intelligent agent operation device of this application can be used to execute the intelligent agent operation method provided in this application. The intelligent agent operation device provided in this application is described below.

[0085] According to an embodiment of this application, an apparatus for implementing the above-described intelligent agent operation method is also provided. Figure 5 This is a schematic diagram of an intelligent agent operating device provided according to an embodiment of this application, such as... Figure 5 As shown, the device includes:

[0086] The receiving module 51 is used to receive multimodal request data based on the target account input, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data.

[0087] The construction module 52, connected to the receiving module 51, is used to construct an agent combination from N backup agents based on multimodal request data, where N is an integer greater than 0, and the agent combination includes one or M backup agents, where M is an integer greater than 0.

[0088] The input module 53, connected to the construction module 52, is used to input multimodal request data into the intelligent agent combination to obtain the target running result, wherein the target running result includes task instruction decision results and intelligent agent running process information.

[0089] The intelligent agent operation device provided in this application includes a receiving module for receiving multimodal request data input based on a target account, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data; a construction module for constructing an intelligent agent combination based on the multimodal request data from N backup intelligent agents, wherein N is an integer greater than 0, and the intelligent agent combination includes one or M backup intelligent agents, where M is an integer greater than 0; and an input module for inputting the multimodal request data into the intelligent agent combination to obtain the target operation result, wherein the target operation result includes task instruction decision results and intelligent agent operation process information, thereby achieving the purpose of dynamically constructing the most suitable intelligent agent combination, thus realizing the technical effect of improving the generalization ability, decision-making efficiency, and task execution reliability of the intelligent agent system, and further solving the technical problem in related technologies where the intelligent agent system structure lacks effective collaboration, hindering the efficient solution of cross-modal tasks.

[0090] Optionally, in the intelligent agent operation device provided in the embodiments of this application, the construction module further includes: a first determining unit, a calculation unit, a second determining unit, and a construction unit, wherein the first determining unit is used to determine the input length, number of modalities, and semantic ambiguity of the multimodal request data; the first calculation unit is used to calculate the complexity of the multimodal request data based on the input length, number of modalities, and semantic ambiguity; the second determining unit is used to determine the combination mode of the intelligent agent combination based on the complexity, wherein the combination mode includes individual mode, group mode, and collective mode; and the construction unit is used to select from N backup intelligent agents based on the combination mode to construct the intelligent agent combination.

[0091] Optionally, in the agent operation device provided in the embodiments of this application, the second determining unit further includes: a third determining unit, a fourth determining unit, and a fifth determining unit, wherein the third determining unit is used to determine the combination mode as an individual mode when the complexity is less than a first preset complexity threshold, wherein the individual mode represents that the agent combination includes a backup agent; the fourth determining unit is used to determine the combination mode as a group mode, or when the complexity is not less than the first preset complexity threshold and less than the second preset complexity threshold, wherein the group mode represents that the agent combination includes H backup agents, where H is an integer greater than 0; the fifth determining unit is used to determine the combination mode as a cluster mode, or when the complexity is not less than the second preset complexity threshold, wherein the cluster mode represents that the agent combination includes a backup agent as the operation center and K backup agents as the operation basis, where K is an integer greater than 0.

[0092] Optionally, in the intelligent agent operation device provided in the embodiments of this application, the N backup intelligent agents in the input module include L primary intelligent agents, P intermediate intelligent agents, and Q advanced intelligent agents. The number of parameters of the primary intelligent agents is less than a first preset parameter threshold, the number of parameters of the intermediate intelligent agents is less than a second preset parameter threshold but not less than the first preset parameter threshold, and the number of parameters of the advanced intelligent agents is not less than the second preset parameter threshold. The attributes of the backup intelligent agents include identifiers, functional labels, influence coefficients, credibility scores, and value function weight vectors, where L, P, and Q are integers greater than 0.

[0093] Optionally, in the intelligent agent operation device provided in this application embodiment, the input module further includes: an extraction unit, a sixth determining unit, a second calculation unit, and an operation unit, wherein the extraction unit is used to extract a task context vector from the multimodal request data; the sixth determining unit is used to determine J task parameter indicators based on the task context vector, wherein the J task parameter indicators include an importance indicator, an urgency indicator, an innovation indicator, a safety indicator, and a resource consumption indicator, and J is an integer greater than 0; the second calculation unit is used to calculate the participation intensity of all backup intelligent agents in the intelligent agent combination based on the value function weight vector and the J task parameter indicators; and the operation unit is used to run all backup intelligent agents based on the participation intensity to obtain the target operation result.

[0094] Optionally, in the agent operation device provided in the embodiments of this application, the device further includes: a deletion unit, a copying unit, a generation unit, and a weighting unit, wherein the deletion unit is used to delete agent networks in the agent combination whose gradient change amplitude is less than a preset change threshold for a certain number of consecutive runs; the copying unit is used to copy agents whose performance meets a preset standard after each preset number of runs are completed, and to randomly perturb them to form W new backup agents, wherein W is an integer greater than 0; the generation unit is used to generate new backup agents when the clustering distance between N backup agents is greater than a preset distance threshold; the weighting unit is used to weight and sum the accuracy loss, execution efficiency loss, and diversity loss based on preset weights to obtain the value of the loss function, wherein the loss function is used to guide the adjustment of the neuron parameters of the backup agents.

[0095] It should be noted that the receiving module 51, the building module 52, and the input module 53 mentioned above correspond to steps S201 to S203 in Embodiment 1. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of the device and run in the computer terminal 10 provided in Embodiment 1.

[0096] Example 4

[0097] Embodiments of this application may provide an electronic device. Figure 6 This is a structural block diagram of an electronic device provided according to an embodiment of this application. For example... Figure 6 As shown, the electronic device may include: one or more ( Figure 6 (Only one is shown) Processor 602, memory 604, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0098] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0099] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: receiving multimodal request data based on target account input, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data; constructing an agent combination based on the multimodal request data among N backup agents, wherein N is an integer greater than 0, and the agent combination includes one or M backup agents, where M is an integer greater than 0; inputting the multimodal request data into the agent combination to obtain the target running result, wherein the target running result includes task instruction decision results and agent running process information.

[0100] The processor can also invoke information and applications stored in the memory via a transmission device to perform the following steps: Constructing an agent ensemble based on multimodal request data from N backup agents, including: determining the input length, number of modalities, and semantic ambiguity of the multimodal request data; calculating the complexity of the multimodal request data based on the input length, number of modalities, and semantic ambiguity; determining the combination mode of the agent ensemble based on the complexity, wherein the combination mode includes individual mode, group mode, and syndicate mode; and selecting from the N backup agents based on the combination mode to construct the agent ensemble.

[0101] The processor can also invoke information and applications stored in the memory via a transmission device to execute the following steps: determining the combination mode of the agent combination based on complexity, including: determining the combination mode as an individual mode when the complexity is less than a first preset complexity threshold, wherein the individual mode indicates that the agent combination includes a backup agent; or determining the combination mode as a group mode when the complexity is not less than the first preset complexity threshold and less than a second preset complexity threshold, wherein the group mode indicates that the agent combination includes H backup agents, where H is an integer greater than 0; or determining the combination mode as a cluster mode when the complexity is not less than the second preset complexity threshold, wherein the cluster mode indicates that the agent combination includes a backup agent as the operation center and K backup agents as the operation basis, where K is an integer greater than 0.

[0102] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: N backup agents include L primary agents, P intermediate agents and Q advanced agents, wherein the number of parameters of the primary agents is less than the first preset parameter threshold, the number of parameters of the intermediate agents is less than the second preset parameter threshold and not less than the first preset parameter threshold, and the number of parameters of the advanced agents is not less than the second preset parameter threshold. The attributes of the backup agents include identifiers, function labels, influence coefficients, credibility scores and value function weight vectors, and L, P and Q are integers greater than 0.

[0103] The processor can also access information and applications stored in the memory via a transmission device to perform the following steps: inputting multimodal request data into the agent ensemble to obtain the target execution result, including: extracting the task context vector from the multimodal request data; determining J task parameter indicators based on the task context vector, wherein the J task parameter indicators include importance indicator, urgency indicator, innovation indicator, safety indicator, and resource consumption indicator, and J is an integer greater than 0; calculating the participation intensity of all backup agents in the agent ensemble based on the value function weight vector and the J task parameter indicators; and running all backup agents based on the participation intensity to obtain the target execution result.

[0104] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: after inputting multimodal request data into the agent combination and obtaining the target running result, it also includes at least one of the following: deleting the agent network in the agent combination whose gradient change amplitude is less than a preset change threshold for a preset number of consecutive runs; after each preset number of runs, copying agents whose performance meets the preset standard and subjecting them to random perturbation to form W new backup agents, where W is an integer greater than 0; generating new backup agents when the clustering distance between the N backup agents is greater than a preset distance threshold; and weighted summing the accuracy loss, execution efficiency loss, and diversity loss based on preset weights to obtain the value of the loss function, where the loss function is used to guide the adjustment of the neuron parameters of the backup agents.

[0105] This application provides a method for operating an intelligent agent. It receives multimodal request data input based on a target account, where the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data. Based on the multimodal request data, an intelligent agent combination is constructed from N backup intelligent agents, where N is a positive integer, and the intelligent agent combination includes one or M backup intelligent agents, where M is a positive integer. The multimodal request data is input into the intelligent agent combination to obtain a target operating result, where the target operating result includes task instruction decision results and intelligent agent operating process information. This achieves the goal of dynamically constructing the most suitable intelligent agent combination, thereby improving the generalization ability, decision-making efficiency, and task execution reliability of the intelligent agent system. Furthermore, it solves the technical problem in related technologies where the intelligent agent system structure lacks effective collaboration, hindering the efficient solution of cross-modal tasks.

[0106] Those skilled in the art will understand that Figure 6 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 6 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 6 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 6 The different configurations shown.

[0107] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0108] Example 5

[0109] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the intelligent agent operation method provided in Embodiment 1.

[0110] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0111] This application also provides a computer program product that, when executed on a data processing device, is suitable for performing steps of an intelligent agent operation method.

[0112] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0113] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0118] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for operating an intelligent agent, characterized in that, include: Receive multimodal request data based on target account input, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data; Among the N backup agents, an agent combination is constructed based on the multimodal request data, where N is an integer greater than 0, and the agent combination includes one or M backup agents, where M is an integer greater than 0. The multimodal request data is input into the agent combination to obtain the target operation result, wherein the target operation result includes task instruction decision results and agent operation process information.

2. The method according to claim 1, characterized in that, The step of constructing an agent ensemble from N backup agents based on the multimodal request data includes: Determine the input length, number of modalities, and semantic ambiguity of the multimodal request data; The complexity of the multimodal request data is calculated based on the input length, the number of modalities, and the semantic ambiguity. Based on the complexity, the combination mode of the agent combination is determined, wherein the combination mode includes individual mode, group mode and collective mode; Based on the aforementioned combination pattern, the agent combination is constructed by selecting from the N backup agents.

3. The method according to claim 2, characterized in that, The process of determining the combination pattern of the agent combination based on the complexity includes: If the complexity is less than a first preset complexity threshold, the combination mode is determined to be the individual mode, wherein the individual mode indicates that the agent combination includes a backup agent; Alternatively, if the complexity is not less than the first preset complexity threshold and less than the second preset complexity threshold, the combination mode is determined to be the group mode, wherein the group mode represents that the agent combination includes H backup agents, where H is an integer greater than 0; Alternatively, if the complexity is not less than the second preset complexity threshold, the combination mode is determined to be the group mode, wherein the group mode represents that the agent combination includes one backup agent as the operation center and K backup agents as the operation basis, where K is an integer greater than 0.

4. The method according to claim 1, characterized in that, The N backup agents include L primary agents, P intermediate agents, and Q advanced agents. The number of parameters of the primary agents is less than a first preset parameter threshold, the number of parameters of the intermediate agents is less than a second preset parameter threshold but not less than the first preset parameter threshold, and the number of parameters of the advanced agents is not less than the second preset parameter threshold. The attributes of the backup agents include identifiers, functional labels, influence coefficients, credibility scores, and value function weight vectors, where L, P, and Q are integers greater than 0.

5. The method according to claim 4, characterized in that, The step of inputting the multimodal request data into the agent combination to obtain the target running result includes: Extract the task context vector from the multimodal request data; Based on the task context vector, J task parameter indicators are determined, wherein the J task parameter indicators include importance indicators, urgency indicators, innovation indicators, safety indicators and resource consumption indicators, and J is an integer greater than 0; Based on the value function weight vector and the J task parameter indicators, calculate the participation intensity of all the backup agents in the agent combination; Based on the participation intensity, all the backup agents are run respectively to obtain the target running result.

6. The method according to any one of claims 1 to 5, characterized in that, After inputting the multimodal request data into the agent assembly to obtain the target running result, the process further includes at least one of the following: Delete the agent network in the agent combination whose gradient change amplitude is less than the preset change threshold for a preset number of consecutive runs; After each preset number of runs, the agent whose performance meets the preset standard is copied and randomly perturbed to form W new backup agents, where W is an integer greater than 0. If the clustering distance among the N backup agents is greater than a preset distance threshold, a new backup agent is generated. Based on preset weights, the accuracy loss, execution efficiency loss, and diversity loss are weighted and summed to obtain the value of the loss function, which is used to guide the adjustment of the neuron parameters of the backup agent.

7. An intelligent agent operating device, characterized in that, include: The receiving module is used to receive multimodal request data based on the target account input, wherein the multimodal request data includes at least one of the following: text data, image data, audio data, video data, point cloud data, and sensor data; A construction module is used to construct an agent combination from N backup agents based on the multimodal request data, where N is an integer greater than 0, and the agent combination includes one or M backup agents, where M is an integer greater than 0. The input module is used to input the multimodal request data into the agent combination to obtain the target running result, wherein the target running result includes the task instruction decision result and the agent running process information.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device containing the computer-readable storage medium to perform the intelligent agent operation method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program executes the agent operation method according to any one of claims 1 to 6 when it runs.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the intelligent agent operation method according to any one of claims 1 to 6.

Images