Multi-subject task hierarchical decision-making method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, provides a multi-subject task hierarchical decision-making method, device and equipment and a medium, is applied to financial and medical health care service scenes, can pre-process multi-modal data according to subjects, realizes effective extraction and distinguishing of multi-subject multi-modal data, and improves the efficiency of the multi-subject task hierarchical decision-making. It is ensured that the features of each subject can be independently and accurately captured; according to the multi-modal feature vector of each main body, constructing a dynamic main body interaction diagram to accurately reflect the dynamic relationship among multiple main bodies, and capturing the interaction change among the main bodies in real time; node feature fusion is carried out on a dynamic subject interaction graph based on a multi-head attention mechanism, and the interaction relationship between subjects can be deeply mined from different angles, so that the fused node features can better reflect the importance degree and association condition of the subjects in interaction; and inputting the dynamic main body interaction graph and the fusion node feature of each node into a hierarchical decision network for processing, thereby realizing reasonable and accurate hierarchical decision on the multi-main body task.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multi-agent task hierarchical decision-making method, apparatus, device, and medium. Background Technology

[0002] Under the current VLA (Vision-Language-Action) technology system, there are many challenges in handling tasks in multi-agent interaction scenarios.

[0003] First, most existing models treat multiple subjects as independent individuals, lacking effective modeling of the dynamic interactions between them. For example, in tasks involving collaborative robots in financial service halls or analyzing online medical interaction scenarios in the healthcare field, traditional models cannot accurately capture the coordination of actions, verbal communication, and mutual influence between subjects, leading to misunderstandings of complex interactive behaviors.

[0004] Secondly, the fusion of multimodal information in multi-agent scenarios is relatively simple. Some models simply stitch together the visual, linguistic, and behavioral features of all agents, ignoring the differences and correlations in multimodal information between different agents during interaction. This approach makes it difficult for models to uncover complex interaction logic between agents and to make collaborative decisions based on the characteristics of different agents.

[0005] Furthermore, existing models lack adaptive adjustment mechanisms when facing dynamically changing interactive scenarios, failing to respond promptly to changes in subject behavior and updates to scenario information, leading to delayed or erroneous decisions. Summary of the Invention

[0006] In view of the above, it is necessary to provide a multi-agent task hierarchical decision-making method, apparatus, equipment and medium, which aims to solve the problem of inaccurate decision-making for multi-agent tasks.

[0007] A multi-agent task hierarchical decision-making method, the multi-agent task hierarchical decision-making method comprising:

[0008] In response to hierarchical decision-making instructions triggered by multi-agent tasks based on objectives, multimodal data is collected;

[0009] The multimodal data is preprocessed for each subject to obtain a multimodal feature vector for each subject;

[0010] A dynamic subject interaction graph is constructed based on the multimodal feature vectors of each subject;

[0011] The node features of the dynamic subject interaction graph are fused based on the multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph.

[0012] Obtain a pre-built hierarchical decision network;

[0013] The dynamic subject interaction graph and the fused node features of each node are input into the hierarchical decision network for processing to obtain the target hierarchical decision result of the target multi-subject task.

[0014] A multi-agent task hierarchical decision-making device, the multi-agent task hierarchical decision-making device comprising:

[0015] The acquisition unit is used to acquire multimodal data in response to hierarchical decision-making instructions triggered by multi-agent tasks based on objectives.

[0016] The preprocessing unit is used to preprocess the multimodal data by subject to obtain the multimodal feature vector of each subject;

[0017] The building unit is used to construct a dynamic subject interaction graph based on the multimodal feature vector of each subject;

[0018] The fusion unit is used to perform node feature fusion on the dynamic subject interaction graph based on a multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph.

[0019] Acquisition unit, used to acquire a pre-built hierarchical decision network;

[0020] The processing unit is used to input the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, so as to obtain the target hierarchical decision result of the target multi-subject task.

[0021] A computer device, the computer device comprising:

[0022] Memory, storing at least one instruction; and

[0023] The processor executes instructions stored in the memory to implement the multi-agent task hierarchical decision-making method.

[0024] A computer-readable storage medium storing at least one instruction, which is executed by a processor in a computer device to implement the multi-agent task hierarchical decision-making method.

[0025] As can be seen from the above technical solutions, this invention can preprocess multimodal data by subject to obtain the multimodal feature vector of each subject, realizing the effective extraction and differentiation of multimodal data of multiple subjects, ensuring that the features of each subject can be captured individually and accurately; constructing a dynamic subject interaction graph based on the multimodal feature vector of each subject to accurately reflect the dynamic relationship between multiple subjects and capture the changes in the interaction between subjects in real time; fusing node features of the dynamic subject interaction graph based on the multi-head attention mechanism to obtain the fused node features of each node, which can deeply mine the interaction relationship between subjects from different angles, highlight important interaction information, and make the fused node features better reflect the importance and correlation of the subjects in the interaction; inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, realizing reasonable and accurate hierarchical decision-making for multi-subject tasks. Attached Figure Description

[0026] Figure 1 This is a flowchart of a preferred embodiment of the multi-agent task hierarchical decision-making method of the present invention.

[0027] Figure 2 This is a functional block diagram of a preferred embodiment of the multi-subject task hierarchical decision-making device of the present invention.

[0028] Figure 3 This is a schematic diagram of the structure of a computer device that implements a preferred embodiment of the multi-subject task hierarchical decision-making method of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the multi-agent task hierarchical decision-making method of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0031] The multi-agent task hierarchical decision-making method is applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0032] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0033] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0034] The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0035] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0036] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0037] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0038] S10, in response to hierarchical decision-making instructions triggered by multi-agent tasks based on objectives, collects multimodal data.

[0039] For example, in the financial field, the interaction between account managers and each client can be analyzed to determine the service quality of the account managers. The corresponding multi-subject task is the service quality analysis task.

[0040] For example, in a medical setting, the interactions between doctors, nurses, and emergency personnel can be analyzed during emergency resuscitation to optimize the resuscitation process. In this case, the corresponding multi-agent task is to generate a resuscitation strategy.

[0041] In this embodiment, the hierarchical decision-making instructions can be triggered by relevant staff members according to actual needs.

[0042] In this embodiment, the multimodal data may include visual data, language data, and motion data.

[0043] S11, preprocess the multimodal data for each subject to obtain the multimodal feature vector of each subject.

[0044] In this embodiment, in order to clarify the interaction relationship between the subjects, it is necessary to extract the effective data of each subject separately.

[0045] Specifically, the sub-subject preprocesses the multimodal data to obtain the multimodal feature vector for each subject, including:

[0046] For the visual data in the multimodal data, an improved multi-object tracking algorithm is used to continuously track each subject in the visual data to obtain the visual region to which each subject belongs; a convolutional neural network (CNN) is used to extract features from the visual region to which each subject belongs to obtain the position, pose and appearance features of each subject in each frame, and these features are used as the visual feature vector of each subject.

[0047] For the language data in the multimodal data, the language data of each subject is encoded using the Bidirectional Encoder Representations from Transformers (BERT) method to obtain the language feature vector of each subject;

[0048] For the action data in the multimodal data, a Long Short-Term Memory (LSTM) network is used to extract features from the action data of each subject to obtain the action feature vector of each subject;

[0049] By integrating the visual feature vector, language feature vector, and action feature vector of each subject, a multimodal feature vector for each subject is obtained.

[0050] The improved multi-target tracking algorithm can be the DeepSORT (Deep Simple Online and Realtime Tracking) model. The DeepSORT model can incorporate depth features to improve tracking accuracy; it can employ motion models to adapt to different motion modes; it can optimize data association strategies to improve tracking stability; and it also has high real-time performance, making it suitable for various application scenarios.

[0051] Before encoding the language data of each subject using the bidirectional encoder representation method, the language data can be divided according to different subjects using speech recognition technology and text segmentation technology.

[0052] Before using the Long Short-Term Memory network to extract features from the action data of each subject, the action data can be normalized to unify the format of different data and facilitate subsequent processing.

[0053] For example, in the monitoring of financial trading halls, the visual characteristics of traders (such as focus and emotion-related facial expressions), verbal communication (such as the content of communication with clients), and actions (such as the actions of operating a computer) can be extracted separately for each subject, providing data support for subsequent analysis of trader behavior.

[0054] For example, in a medical operating room, visual characteristics (such as surgical postures), verbal instructions (such as surgical instructions given by the doctor), and actions (such as the operation of surgical instruments) of different subjects such as doctors and nurses can be preprocessed separately for each subject, which helps to analyze and evaluate the surgical process later.

[0055] Through the above embodiments, the effective extraction and differentiation of multi-modal data of multiple subjects (visual, linguistic, and behavioral) are achieved, providing accurate basic data for subsequent processing and ensuring that the features of each subject can be captured individually and accurately.

[0056] S12, construct a dynamic subject interaction graph based on the multimodal feature vectors of each subject.

[0057] In this embodiment, constructing a dynamic subject interaction graph based on the multimodal feature vector of each subject includes:

[0058] Each subject is identified as a node to construct an initial structure graph; where each node includes the multimodal feature vector of the corresponding subject;

[0059] Obtain the relationships between the subjects, and construct the edges of the initial structure graph based on the relationships between the subjects to obtain an intermediate structure graph; wherein, the relationships between the subjects include spatial positional relationships, language communication objects, and action collaboration relationships between the subjects;

[0060] Obtain the time series information from the multimodal data;

[0061] By utilizing the message passing mechanism of a graph neural network and combining it with the time series information, the intermediate structure graph is dynamically updated to obtain the dynamic main interaction graph.

[0062] For example, if each subject is a node, and two subjects are within a certain distance and are having a dialogue, then an edge containing the interaction strength weight is established between the corresponding nodes; if there is a coordinated action between the subjects, an edge reflecting the degree of coordination is also established.

[0063] Among them, as the subject's behavior and the scene change, the node features and edge weights can be adjusted in real time.

[0064] Through the above embodiments, the constructed dynamic subject interaction graph can accurately reflect the dynamic relationships between multiple subjects; through dynamic updates, changes in the interaction between subjects can be captured in real time, providing structured interaction information for subsequent attention calculation and decision-making.

[0065] S13, perform node feature fusion on the dynamic subject interaction graph based on the multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph.

[0066] In this embodiment, in order to highlight the differences and correlations of multimodal information among different subjects during the interaction process, node feature fusion is performed on the dynamic subject interaction graph.

[0067] Specifically, the node feature fusion of the dynamic subject interaction graph based on the multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph includes:

[0068] For each attention head under the multi-head attention mechanism, the attention coefficient of each node relative to other nodes is calculated based on the multimodal feature vector of each node in the dynamic subject interaction graph;

[0069] The attention coefficient of each node relative to other nodes is normalized to obtain the attention weight of each node relative to other nodes.

[0070] The features of each node are concatenated according to the attention weight of each node relative to other nodes under each attention head, so as to obtain the concatenated features of each node.

[0071] A linear transformation is performed on the splicing features of each node to obtain the fused node features of each node.

[0072] Among them, by splicing and linearly transforming the attention results of each attention head, it is possible to achieve in-depth mining of the interaction information between subjects.

[0073] Through the above embodiments, the interaction relationships between subjects can be explored in depth from different perspectives, highlighting important interaction information, and making the fused node features better reflect the importance and relevance of the subjects in the interaction.

[0074] S14, Obtain the pre-built hierarchical decision network.

[0075] The hierarchical decision-making network may include a bottom-level decision-making network, a middle-level coordination network, and a high-level planning network, each independently constructed for each subject.

[0076] Decisions are made collaboratively by various decision-making networks, thereby improving the accuracy of the decisions.

[0077] S15, the dynamic subject interaction graph and the fusion node features of each node are input into the hierarchical decision network for processing to obtain the target hierarchical decision result of the target multi-subject task.

[0078] In this embodiment, the step of inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing includes:

[0079] The fusion node features of each node are input into the underlying decision network of the corresponding subject to obtain the initial decision data of each underlying decision network;

[0080] The underlying decision network adopts a multilayer perceptron (MLP) structure.

[0081] The underlying decision network generates corresponding initial decision data based on the state of the corresponding subject and local interaction information.

[0082] Through the above embodiments, preliminary action decisions or language response candidate sets can be generated by the underlying decision-making network independently constructed for each subject.

[0083] Furthermore, the step of inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing also includes:

[0084] Obtain global information of the dynamic subject interaction graph;

[0085] Using the aforementioned mid-level coordination network, the initial decision data of each bottom-level decision network is evaluated and coordinated based on the global information to obtain mid-level coordination data.

[0086] For example, when a conflict in decision-making between subjects is detected, the mid-level coordination network adjusts the decision priorities of each subject according to the relationship weights in the dynamic subject interaction graph.

[0087] Furthermore, the step of inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing also includes:

[0088] The target multi-agent task is analyzed to obtain task objectives and long-term planning information;

[0089] By utilizing the high-level planning network and combining the task objectives and long-term planning information, the mid-level coordination data is optimized and adjusted to obtain the target-level decision-making results.

[0090] For example, the high-level planning network can be built on the Transformer architecture. The high-level planning network models global information through a self-attention mechanism and outputs the final action command or language response for each subject.

[0091] Through the above embodiments, a hierarchical decision-making network can be used to realize the decision-making process from local to global. The bottom layer decision ensures the individualized initial decisions of each subject, the middle layer coordination resolves decision conflicts and ensures synergy, and the top layer planning realizes the final global optimization decision, thereby significantly improving the rationality and effectiveness of the decision.

[0092] For example, in financial risk control tasks, the bottom-level decision network makes a preliminary risk assessment of the trading behavior of each trading account (based on the multimodal features extracted in the early stage); the middle-level coordination network evaluates the risk decisions of multiple accounts and coordinates conflicting judgments (such as the risk assessment of related transactions of different accounts); the top-level planning network combines the overall financial market risk objectives and long-term risk control strategies to make the final risk control decision, such as whether to freeze accounts or issue risk warnings.

[0093] For example, in multidisciplinary joint testing in a medical setting, the bottom-level decision network provides doctors in each department with preliminary testing suggestions based on their own expertise and patient-related information; the middle-level coordination network evaluates and coordinates the suggestions from each department to resolve conflicts between different departmental suggestions; and the top-level planning network combines the patient's overall condition and treatment goals to formulate the final auxiliary testing plan.

[0094] In this embodiment, the target hierarchical decision results can also be converted into actual executable action instructions (such as robot joint control signals) or language text, and output to the corresponding execution subject for execution.

[0095] As can be seen from the above technical solutions, this invention can preprocess multimodal data by subject to obtain the multimodal feature vector of each subject, realizing the effective extraction and differentiation of multimodal data of multiple subjects, ensuring that the features of each subject can be captured individually and accurately; constructing a dynamic subject interaction graph based on the multimodal feature vector of each subject to accurately reflect the dynamic relationship between multiple subjects and capture the changes in the interaction between subjects in real time; fusing node features of the dynamic subject interaction graph based on the multi-head attention mechanism to obtain the fused node features of each node, which can deeply mine the interaction relationship between subjects from different angles, highlight important interaction information, and make the fused node features better reflect the importance and correlation of the subjects in the interaction; inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, realizing reasonable and accurate hierarchical decision-making for multi-subject tasks.

[0096] like Figure 2 The diagram shown is a functional block diagram of a preferred embodiment of the multi-agent task hierarchical decision-making device of the present invention. The multi-agent task hierarchical decision-making device 11 includes a data acquisition unit 110, a preprocessing unit 111, a construction unit 112, a fusion unit 113, an acquisition unit 114, and a processing unit 115. The module / unit referred to in this invention is a series of computer program segments that can be executed by a processor and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0097] The acquisition unit 110 is used to acquire multimodal data in response to hierarchical decision-making instructions triggered by multi-subject task targeting.

[0098] For example, in the financial field, the interaction between account managers and each client can be analyzed to determine the service quality of the account managers. The corresponding multi-subject task is the service quality analysis task.

[0099] For example, in a medical setting, the interactions between doctors, nurses, and emergency personnel can be analyzed during emergency resuscitation to optimize the resuscitation process. In this case, the corresponding multi-agent task is to generate a resuscitation strategy.

[0100] In this embodiment, the hierarchical decision-making instructions can be triggered by relevant staff members according to actual needs.

[0101] In this embodiment, the multimodal data may include visual data, language data, and motion data.

[0102] The preprocessing unit 111 is used to preprocess the multimodal data by subject to obtain the multimodal feature vector of each subject.

[0103] In this embodiment, in order to clarify the interaction relationship between the subjects, it is necessary to extract the effective data of each subject separately.

[0104] Specifically, the preprocessing unit 111 preprocesses the multimodal data by subject to obtain the multimodal feature vector for each subject, including:

[0105] For the visual data in the multimodal data, an improved multi-object tracking algorithm is used to continuously track each subject in the visual data to obtain the visual region to which each subject belongs; a convolutional neural network (CNN) is used to extract features from the visual region to which each subject belongs to obtain the position, pose and appearance features of each subject in each frame, and these features are used as the visual feature vector of each subject.

[0106] For the language data in the multimodal data, the language data of each subject is encoded using the Bidirectional Encoder Representations from Transformers (BERT) method to obtain the language feature vector of each subject;

[0107] For the action data in the multimodal data, a Long Short-Term Memory (LSTM) network is used to extract features from the action data of each subject to obtain the action feature vector of each subject;

[0108] By integrating the visual feature vector, language feature vector, and action feature vector of each subject, a multimodal feature vector for each subject is obtained.

[0109] The improved multi-target tracking algorithm can be the DeepSORT (Deep Simple Online and Realtime Tracking) model. The DeepSORT model can incorporate depth features to improve tracking accuracy; it can employ motion models to adapt to different motion modes; it can optimize data association strategies to improve tracking stability; and it also has high real-time performance, making it suitable for various application scenarios.

[0110] Before encoding the language data of each subject using the bidirectional encoder representation method, the language data can be divided according to different subjects using speech recognition technology and text segmentation technology.

[0111] Before using the Long Short-Term Memory network to extract features from the action data of each subject, the action data can be normalized to unify the format of different data and facilitate subsequent processing.

[0112] For example, in the monitoring of financial trading halls, the visual characteristics of traders (such as focus and emotion-related facial expressions), verbal communication (such as the content of communication with clients), and actions (such as the actions of operating a computer) can be extracted separately for each subject, providing data support for subsequent analysis of trader behavior.

[0113] For example, in a medical operating room, visual characteristics (such as surgical postures), verbal instructions (such as surgical instructions given by the doctor), and actions (such as the operation of surgical instruments) of different subjects such as doctors and nurses can be preprocessed separately for each subject, which helps to analyze and evaluate the surgical process later.

[0114] Through the above embodiments, the effective extraction and differentiation of multi-modal data of multiple subjects (visual, linguistic, and behavioral) are achieved, providing accurate basic data for subsequent processing and ensuring that the features of each subject can be captured individually and accurately.

[0115] The construction unit 112 is used to construct a dynamic subject interaction graph based on the multimodal feature vector of each subject.

[0116] In this embodiment, the construction unit 112 constructs a dynamic subject interaction graph based on the multimodal feature vector of each subject, including:

[0117] Each subject is identified as a node to construct an initial structure graph; where each node includes the multimodal feature vector of the corresponding subject;

[0118] Obtain the relationships between the subjects, and construct the edges of the initial structure graph based on the relationships between the subjects to obtain an intermediate structure graph; wherein, the relationships between the subjects include spatial positional relationships, language communication objects, and action collaboration relationships between the subjects;

[0119] Obtain the time series information from the multimodal data;

[0120] By utilizing the message passing mechanism of a graph neural network and combining it with the time series information, the intermediate structure graph is dynamically updated to obtain the dynamic main interaction graph.

[0121] For example, if each subject is a node, and two subjects are within a certain distance and are having a dialogue, then an edge containing the interaction strength weight is established between the corresponding nodes; if there is a coordinated action between the subjects, an edge reflecting the degree of coordination is also established.

[0122] Among them, as the subject's behavior and the scene change, the node features and edge weights can be adjusted in real time.

[0123] Through the above embodiments, the constructed dynamic subject interaction graph can accurately reflect the dynamic relationships between multiple subjects; through dynamic updates, changes in the interaction between subjects can be captured in real time, providing structured interaction information for subsequent attention calculation and decision-making.

[0124] The fusion unit 113 is used to perform node feature fusion on the dynamic subject interaction graph based on a multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph.

[0125] In this embodiment, in order to highlight the differences and correlations of multimodal information among different subjects during the interaction process, node feature fusion is performed on the dynamic subject interaction graph.

[0126] Specifically, the fusion unit 113 performs node feature fusion on the dynamic subject interaction graph based on a multi-head attention mechanism, obtaining the fused node features of each node in the dynamic subject interaction graph, including:

[0127] For each attention head under the multi-head attention mechanism, the attention coefficient of each node relative to other nodes is calculated based on the multimodal feature vector of each node in the dynamic subject interaction graph;

[0128] The attention coefficient of each node relative to other nodes is normalized to obtain the attention weight of each node relative to other nodes.

[0129] The features of each node are concatenated according to the attention weight of each node relative to other nodes under each attention head, so as to obtain the concatenated features of each node.

[0130] A linear transformation is performed on the splicing features of each node to obtain the fused node features of each node.

[0131] Among them, by splicing and linearly transforming the attention results of each attention head, it is possible to achieve in-depth mining of the interaction information between subjects.

[0132] Through the above embodiments, the interaction relationships between subjects can be explored in depth from different perspectives, highlighting important interaction information, and making the fused node features better reflect the importance and relevance of the subjects in the interaction.

[0133] The acquisition unit 114 is used to acquire a pre-constructed hierarchical decision network.

[0134] The hierarchical decision-making network may include a bottom-level decision-making network, a middle-level coordination network, and a high-level planning network, each independently constructed for each subject.

[0135] Decisions are made collaboratively by various decision-making networks, thereby improving the accuracy of the decisions.

[0136] The processing unit 115 is used to input the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, so as to obtain the target hierarchical decision result of the target multi-subject task.

[0137] In this embodiment, the processing unit 115 inputs the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, including:

[0138] The fusion node features of each node are input into the underlying decision network of the corresponding subject to obtain the initial decision data of each underlying decision network;

[0139] The underlying decision network adopts a multilayer perceptron (MLP) structure.

[0140] The underlying decision network generates corresponding initial decision data based on the state of the corresponding subject and local interaction information.

[0141] Through the above embodiments, preliminary action decisions or language response candidate sets can be generated by the underlying decision-making network independently constructed for each subject.

[0142] Furthermore, the processing unit 115 further includes inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, including:

[0143] Obtain global information of the dynamic subject interaction graph;

[0144] Using the aforementioned mid-level coordination network, the initial decision data of each bottom-level decision network is evaluated and coordinated based on the global information to obtain mid-level coordination data.

[0145] For example, when a conflict in decision-making between subjects is detected, the mid-level coordination network adjusts the decision priorities of each subject according to the relationship weights in the dynamic subject interaction graph.

[0146] Furthermore, the processing unit 115 further includes inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, including:

[0147] The target multi-agent task is analyzed to obtain task objectives and long-term planning information;

[0148] By utilizing the high-level planning network and combining the task objectives and long-term planning information, the mid-level coordination data is optimized and adjusted to obtain the target-level decision-making results.

[0149] For example, the high-level planning network can be built on the Transformer architecture. The high-level planning network models global information through a self-attention mechanism and outputs the final action command or language response for each subject.

[0150] Through the above embodiments, a hierarchical decision-making network can be used to realize the decision-making process from local to global. The bottom layer decision ensures the individualized initial decisions of each subject, the middle layer coordination resolves decision conflicts and ensures synergy, and the top layer planning realizes the final global optimization decision, thereby significantly improving the rationality and effectiveness of the decision.

[0151] For example, in financial risk control tasks, the bottom-level decision network makes a preliminary risk assessment of the trading behavior of each trading account (based on the multimodal features extracted in the early stage); the middle-level coordination network evaluates the risk decisions of multiple accounts and coordinates conflicting judgments (such as the risk assessment of related transactions of different accounts); the top-level planning network combines the overall financial market risk objectives and long-term risk control strategies to make the final risk control decision, such as whether to freeze accounts or issue risk warnings.

[0152] For example, in multidisciplinary joint testing in a medical setting, the bottom-level decision network provides doctors in each department with preliminary testing suggestions based on their own expertise and patient-related information; the middle-level coordination network evaluates and coordinates the suggestions from each department to resolve conflicts between different departmental suggestions; and the top-level planning network combines the patient's overall condition and treatment goals to formulate the final auxiliary testing plan.

[0153] In this embodiment, the target hierarchical decision results can also be converted into actual executable action instructions (such as robot joint control signals) or language text, and output to the corresponding execution subject for execution.

[0154] As can be seen from the above technical solutions, this invention can preprocess multimodal data by subject to obtain the multimodal feature vector of each subject, realizing the effective extraction and differentiation of multimodal data of multiple subjects, ensuring that the features of each subject can be captured individually and accurately; constructing a dynamic subject interaction graph based on the multimodal feature vector of each subject to accurately reflect the dynamic relationship between multiple subjects and capture the changes in the interaction between subjects in real time; fusing node features of the dynamic subject interaction graph based on the multi-head attention mechanism to obtain the fused node features of each node, which can deeply mine the interaction relationship between subjects from different angles, highlight important interaction information, and make the fused node features better reflect the importance and correlation of the subjects in the interaction; inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, realizing reasonable and accurate hierarchical decision-making for multi-subject tasks.

[0155] like Figure 3The diagram shown is a schematic representation of the structure of a computer device that implements a preferred embodiment of the multi-agent task hierarchical decision-making method of the present invention.

[0156] The computer device 1 may include a memory 12, a processor 13, and a bus (the arrow in the figure represents the bus), and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a multi-agent task hierarchical decision program.

[0157] Those skilled in the art will understand that the schematic diagram is merely an example of computer device 1 and does not constitute a limitation on computer device 1. Computer device 1 can be either a bus topology or a star topology. Computer device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, computer device 1 may also include input / output devices, network access devices, etc.

[0158] It should be noted that the computer device 1 described is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0159] The memory 12 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a portable hard drive of the computer device 1. In other embodiments, the memory 12 can be an external storage device of the computer device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device 1. Furthermore, the memory 12 can include both internal and external storage units of the computer device 1. The memory 12 can be used not only to store application software and various types of data installed on the computer device 1, such as the code of a multi-agent task hierarchical decision-making program, but also to temporarily store data that has been output or will be output.

[0160] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the computer device 1, connecting various components of the computer device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing multi-subject task hierarchical decision-making programs) and calls data stored in the memory 12 to perform various functions of the computer device 1 and process data.

[0161] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the various embodiments of the multi-agent task hierarchical decision-making method described above, for example... Figure 1 The steps are shown.

[0162] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a data acquisition unit 110, a preprocessing unit 111, a construction unit 112, a fusion unit 113, an acquisition unit 114, and a processing unit 115.

[0163] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the multi-agent task hierarchical decision-making method described in the various embodiments of the present invention.

[0164] If the modules / units integrated in the computer device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0165] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0166] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0167] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0168] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The bus is represented by only one straight line, but this does not mean that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0169] Although not shown, the computer device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The computer device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0170] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish a communication connection between the computer device 1 and other computer devices.

[0171] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the computer device 1 and to display a visual user interface.

[0172] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0173] It will be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0174] Combination Figure 1 The memory 12 in the computer device 1 stores multiple instructions to implement a multi-agent task hierarchical decision-making method, and the processor 13 can execute the multiple instructions to achieve:

[0175] In response to hierarchical decision-making instructions triggered by multi-agent tasks based on objectives, multimodal data is collected;

[0176] The multimodal data is preprocessed for each subject to obtain a multimodal feature vector for each subject;

[0177] A dynamic subject interaction graph is constructed based on the multimodal feature vectors of each subject;

[0178] The node features of the dynamic subject interaction graph are fused based on the multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph.

[0179] Obtain a pre-built hierarchical decision network;

[0180] The dynamic subject interaction graph and the fused node features of each node are input into the hierarchical decision network for processing to obtain the target hierarchical decision result of the target multi-subject task.

[0181] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0182] It should be noted that all data involved in this case was legally obtained. Software tools or components not belonging to this company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0183] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0184] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0185] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0187] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0188] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0189] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this invention can also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multi-agent task-level decision-making method, characterized in that, The multi-agent task hierarchical decision-making method includes: In response to hierarchical decision-making instructions triggered by multi-agent tasks based on objectives, multimodal data is collected; The multimodal data is preprocessed for each subject to obtain a multimodal feature vector for each subject; A dynamic subject interaction graph is constructed based on the multimodal feature vectors of each subject; The node features of the dynamic subject interaction graph are fused based on the multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph. Obtain a pre-built hierarchical decision network; The dynamic subject interaction graph and the fused node features of each node are input into the hierarchical decision network for processing to obtain the target hierarchical decision result of the target multi-subject task.

2. The multi-agent task hierarchical decision-making method as described in claim 1, characterized in that, The sub-subject preprocesses the multimodal data to obtain the multimodal feature vector for each subject, including: For the visual data in the multimodal data, an improved multi-object tracking algorithm is used to continuously track each subject in the visual data to obtain the visual region to which each subject belongs; a convolutional neural network is used to extract features from the visual region to which each subject belongs to obtain the position, pose and appearance features of each subject in each frame, and these features are used as the visual feature vector of each subject. For the language data in the multimodal data, the language data of each subject is encoded using a bidirectional encoder representation method to obtain the language feature vector of each subject; For the action data in the multimodal data, a long short-term memory network is used to extract features from the action data of each subject to obtain the action feature vector of each subject; By integrating the visual feature vector, language feature vector, and action feature vector of each subject, a multimodal feature vector for each subject is obtained.

3. The multi-agent task hierarchical decision-making method as described in claim 1, characterized in that, The construction of the dynamic subject interaction graph based on the multimodal feature vector of each subject includes: Each subject is identified as a node to construct an initial structure graph; where each node includes the multimodal feature vector of the corresponding subject; Obtain the relationships between the subjects, and construct the edges of the initial structure graph based on the relationships between the subjects to obtain an intermediate structure graph; wherein, the relationships between the subjects include spatial positional relationships, language communication objects, and action collaboration relationships between the subjects; Obtain the time series information from the multimodal data; By utilizing the message passing mechanism of a graph neural network and combining it with the time series information, the intermediate structure graph is dynamically updated to obtain the dynamic main interaction graph.

4. The multi-agent task hierarchical decision-making method as described in claim 1, characterized in that, The multi-head attention mechanism is used to fuse node features of the dynamic subject interaction graph to obtain the fused node features of each node in the dynamic subject interaction graph, including: For each attention head under the multi-head attention mechanism, the attention coefficient of each node relative to other nodes is calculated based on the multimodal feature vector of each node in the dynamic subject interaction graph; The attention coefficient of each node relative to other nodes is normalized to obtain the attention weight of each node relative to other nodes. The features of each node are concatenated according to the attention weight of each node relative to other nodes under each attention head, so as to obtain the concatenated features of each node. A linear transformation is performed on the splicing features of each node to obtain the fused node features of each node.

5. The multi-agent task hierarchical decision-making method as described in claim 1, characterized in that, The hierarchical decision network includes a bottom-level decision network independently constructed for each subject; The step of inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing includes: The fusion node features of each node are input into the underlying decision network of the corresponding subject to obtain the initial decision data of each underlying decision network; The underlying decision network adopts a multilayer perceptron structure; The underlying decision network generates corresponding initial decision data based on the state of the corresponding subject and local interaction information.

6. The multi-agent task hierarchical decision-making method as described in claim 5, characterized in that, The hierarchical decision network further includes a mid-level coordination network; the step of inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing also includes: Obtain global information of the dynamic subject interaction graph; Using the aforementioned mid-level coordination network, the initial decision data of each bottom-level decision network is evaluated and coordinated based on the global information to obtain mid-level coordination data.

7. The multi-agent task hierarchical decision-making method as described in claim 6, characterized in that, The hierarchical decision network further includes a high-level planning network; the step of inputting the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing also includes: The target multi-agent task is analyzed to obtain task objectives and long-term planning information; By utilizing the high-level planning network and combining the task objectives and long-term planning information, the mid-level coordination data is optimized and adjusted to obtain the target-level decision-making results.

8. A multi-agent task hierarchical decision-making device, characterized in that, The multi-agent task hierarchical decision-making device includes: The acquisition unit is used to acquire multimodal data in response to hierarchical decision-making instructions triggered by multi-agent tasks based on objectives. The preprocessing unit is used to preprocess the multimodal data by subject to obtain the multimodal feature vector of each subject; The building unit is used to construct a dynamic subject interaction graph based on the multimodal feature vector of each subject; The fusion unit is used to perform node feature fusion on the dynamic subject interaction graph based on a multi-head attention mechanism to obtain the fused node features of each node in the dynamic subject interaction graph. Acquisition unit, used to acquire a pre-built hierarchical decision network; The processing unit is used to input the dynamic subject interaction graph and the fused node features of each node into the hierarchical decision network for processing, so as to obtain the target hierarchical decision result of the target multi-subject task.

9. A computer device, characterized in that, The computer device includes: Memory, storing at least one instruction; and The processor executes instructions stored in the memory to implement the multi-agent task hierarchical decision-making method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in a computer device to implement the multi-agent task hierarchical decision-making method as described in any one of claims 1 to 7.

Images