Data and knowledge dual-driven group intelligent decision-making method and device

Abstract

The invention provides a data and knowledge dual-driven group intelligent decision-making method and device. The method comprises the following steps: constructing a swarm intelligence system; a knowledge processing controller and a neural network controller are configured independent of the swarm intelligence system; the knowledge processing layer generates a knowledge matching decision result based on the knowledge processing controller; the data processing layer obtains output of each corrected learning model based on a neural network controller, and the output is used as a correction result; generating a final decision result based on a controller cooperation mode, the knowledge matching decision result and the correction result; taking the final decision result as the output of the swarm intelligence system; and the unmanned cluster makes an individual decision by each individual in the unmanned cluster based on the final decision result, and the individual decision is used for guiding the behavior and action of the individual. According to the method, data driving and knowledge driving are combined, a data model is generated under the guidance of knowledge, and alternate iteration of knowledge and data is formed.

Description

Technical Field

[0001] The present invention relates to the field of group intelligence technology, and in particular to a group intelligence decision-making method and device driven by both data and knowledge. Background Art

[0002] Swarm intelligence (CI) originated from the observation and study of social organisms and human social behavior. Due to its advantages such as distribution, flexibility and robustness, it provides new solutions to many challenging and complex problems. It is one of the five major intelligent forms that the new generation of artificial intelligence focuses on. Furthermore, unmanned swarm systems composed of autonomous unmanned platforms such as drones and unmanned vehicles have made great progress, and have achieved a series of research and application results in the fields of intelligent traffic control, regional logistics scheduling, robot swarm control, and complex network synchronization. Especially in the field of military intelligence, swarm intelligence has been considered to be a new technology that may bring about disruptive changes. The current mainstream swarm intelligence decision-making methods mainly include two categories: knowledge-driven and data-driven.

[0003] Knowledge-driven methods can make full use of existing knowledge, including existing models and algorithm knowledge, rule experience knowledge, and specific domain knowledge. The broad connotation of knowledge facilitates the flexible integration of multidisciplinary knowledge; at the same time, many model-based knowledge-driven methods have a complete theoretical support system and have natural advantages in analyzing algorithm stability, optimality, convergence, etc.; in addition, knowledge-driven models have better interpretability; knowledge, as a highly condensed embodiment of data and information, often means more efficient algorithm execution efficiency.

[0004] "Knowledge" includes a series of algorithmic knowledge based on mathematical / physical models, rule-based empirical knowledge, and domain knowledge for specific applications, such as mathematical / physical model knowledge based on group dynamics models, various path planning algorithms based on model derivation, task allocation algorithms, and consistency control algorithms based on multi-order models. Algorithmic knowledge, empirical knowledge of simple cluster behavior rules built from people's cognition of basic cluster behaviors, and domain knowledge for specific application scenarios. Knowledge-driven is the main research path of many actual group intelligence systems, and has a broad application basis in unmanned cluster task planning, game decision-making, collaborative control, and other aspects.

[0005] Data-driven methods such as deep reinforcement learning, which have been widely used in recent years, have the characteristics of not requiring precise modeling, being able to achieve large-scale coverage and exploration of the solution space, continuously learning and evolving from data, and having strong algorithm versatility. They are also supported by tools such as massive open source models and algorithm libraries.

[0006] Data-driven methods include ant colony algorithm, particle swarm optimization algorithm, and biologically inspired evolutionary computing methods such as wolf pack algorithm and pigeon flock algorithm, which have important reference significance for the behavior of unmanned cluster systems. The "data-driven method" described in this article focuses on deep learning, reinforcement learning and other machine learning algorithms that have been widely used in recent years. Deep learning has the "perception" ability of high-dimensional data, and reinforcement learning has the "decision-making" ability in interacting with the environment. Therefore, the data-driven method has the advantage of being combined with the application of large-scale swarm intelligence systems.

[0007] In practical applications of knowledge-driven methods, especially in complex problems such as large-scale group collaboration, the knowledge mechanism of collective intelligence is not yet fully clear, and the cost of knowledge acquisition is high. At the same time, existing knowledge is difficult to fully cover the huge solution space of complex group behavior, and it is also difficult to support the continuous learning and evolution of cluster behavior.

[0008] Data-driven methods often have difficulties in analyzing theoretical characteristics, and their typical "black box" characteristics also bring about problems such as poor interpretability. At the same time, they are highly dependent on high-quality big data, and in group intelligence applications, such data are difficult to obtain. In addition, as the size of the group and the complexity of the problem increase, solving the problem of spatial dimensionality disaster poses severe challenges to learning efficiency. And their reliance on huge computing power also makes individuals or general institutions face serious bottlenecks when conducting research. Summary of the invention

[0009] In order to solve the above technical problems, the present invention proposes a data- and knowledge-driven group intelligent decision-making method to improve the accuracy, reliability and decision-making efficiency of group intelligent decision-making of each node in a distributed system.

[0010] The present invention proposes a data- and knowledge-driven group intelligent decision-making method, the method comprising:

[0011] Step S1: constructing a swarm intelligence system, which is a system used by the unmanned cluster to provide decision results to the unmanned cluster to assist the unmanned cluster in performing operations, including a knowledge processing layer and a data processing layer;

[0012] The unmanned cluster state data, the real-time sensed position data of the unmanned cluster, and the captured visual data are used as inputs of the swarm intelligence system; a knowledge processing controller and a neural network controller are configured independently of the swarm intelligence system, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data, and the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster;

[0013] The knowledge processing layer receives the first control instruction issued by the knowledge processing controller and the knowledge data generated based on the posture data and the visual data, compares the knowledge data with the prior knowledge in the knowledge base in the knowledge processing layer, and generates a knowledge matching decision result; the data processing layer integrates a variety of learning models, each learning model receives the second control instruction issued by the neural network controller and the unmanned cluster state data, modifies the parameters of each learning model, and obtains the output of each modified learning model as the correction result; based on the controller coordination mode, the knowledge matching decision result and the correction result, generates a final decision result; and uses the final decision result as the output of the swarm intelligence system;

[0014] Step S2: Based on the final decision result, each individual in the unmanned cluster makes an individual decision, and the individual decision is used to guide the individual's behavior and action.

[0015] Preferably, the controller coordination modes include a parallel combination mode, a knowledge priority series combination mode, and a data priority series combination mode.

[0016] Preferably, the controller coordination mode, the knowledge matching decision result and the correction result are used to generate the knowledge matching decision result calculated by the neural network model, wherein:

[0017] When the controller coordination mode is the parallel combination mode, the knowledge matching decision result is used as the output of the knowledge processing layer, and the correction result is used as the output of the data processing layer; the knowledge matching decision result and the correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

[0018] Preferably, the controller coordination mode, the knowledge matching decision result and the correction result are used to generate the knowledge matching decision result calculated by the neural network model, wherein:

[0019] When the controller coordination mode is the knowledge priority serial combination mode, the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster and the input of the swarm intelligence system to obtain a correction result; the correction result is split, and the corresponding unmanned cluster data and unmanned cluster individual data are split from the correction result; the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the split unmanned cluster data, and uses the knowledge matching decision result generated based on the first control instruction as the output of the knowledge layer; the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster and the split unmanned cluster individual data, and obtains a correction result again as a second correction result; the knowledge matching decision result and the second correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

[0020] Preferably, the controller coordination mode, the knowledge matching decision result and the correction result are used to generate the knowledge matching decision result calculated by the neural network model, wherein:

[0021] When the controller coordination mode is the data priority serial combination mode, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the input of the swarm intelligence system to obtain a knowledge matching decision result; the knowledge matching decision result is split, and the corresponding unmanned cluster data and unmanned cluster individual data are split from the knowledge matching decision result; the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the split unmanned cluster data, and uses the knowledge matching decision result generated again based on the first control instruction as the second knowledge matching decision result, and the second knowledge matching decision result is used as the output of the knowledge layer; the neural network controller generates a second control instruction based on the unmanned cluster real-time data and the split unmanned cluster individual data to obtain a correction result; the second knowledge matching decision result and the correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

[0022] The present invention provides a group intelligent decision-making device driven by both data and knowledge, the device comprising:

[0023] Swarm intelligence system building module: configured to build a swarm intelligence system, which is a system used by the unmanned cluster to provide decision results to the unmanned cluster to assist the unmanned cluster in performing operations, including a knowledge processing layer and a data processing layer;

[0024] The unmanned cluster state data, the real-time sensed position data of the unmanned cluster, and the captured visual data are used as inputs of the swarm intelligence system; a knowledge processing controller and a neural network controller are configured independently of the swarm intelligence system, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data, and the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster;

[0025] The knowledge processing layer receives the first control instruction issued by the knowledge processing controller and the knowledge data generated based on the posture data and the visual data, compares the knowledge data with the prior knowledge in the knowledge base in the knowledge processing layer, and generates a knowledge matching decision result; the data processing layer integrates a variety of learning models, each learning model receives the second control instruction issued by the neural network controller and the unmanned cluster state data, modifies the parameters of each learning model, and obtains the output of each modified learning model as the correction result; based on the controller coordination mode, the knowledge matching decision result and the correction result, generates a final decision result; and uses the final decision result as the output of the swarm intelligence system;

[0026] Output module: configured so that the unmanned cluster makes individual decisions based on the final decision result, and the individual decisions are used to guide the behaviors and actions of the individuals.

[0027] The present invention provides an electronic device, comprising:

[0028] at least one processor; and

[0029] a memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0031] The present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method as described above.

[0032] The present invention has the following technical effects:

[0033] The present invention combines data-driven and knowledge-driven approaches, uses knowledge to guide the generation of data models, and generates new knowledge from the data models, thus forming an alternating iteration of knowledge and data, injecting new impetus into group intelligent decision-making.

[0034] This application uses knowledge to guide the generation of data models, summarizes and generates new knowledge from data models, and forms an alternating iteration of knowledge and data. It is an important path to achieve the autonomous evolution of intelligent systems, and it is also an important paradigm for realizing artificial intelligence systems that can be understood by humans but transcend the human knowledge system. The methods from knowledge to data include imitation learning and various heuristic data-driven methods, and from data to knowledge include various rule learning, opponent modeling and other methods. For the intelligent decision-making behavior of swarm intelligent systems, combined with the actual application background, an iterative evolution paradigm of knowledge and data can be formed.

[0035] The data / knowledge dual-driven simulation computing technology proposed in this application plays an important role in clarifying the mechanism of group intelligence stimulation and convergence, improving the cognition of group intelligence systems, and supplementing high-quality training data. It is an important method to promote the research of group intelligence, especially group intelligence decision-making, and will also provide methodological support for the realization of a guideable, trustworthy, learnable, and evolvable group intelligence system.

[0036] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0038] Figure 1 A flow chart of the data- and knowledge-driven group intelligent decision-making method proposed in the present invention;

[0039] Figure 2 A schematic diagram of knowledge and data collaboration for the present invention;

[0040] Figure 3 This is a comparison chart between Markov decision process and semi-Markov decision process;

[0041] Figure 4 Schematic diagram of data-driven approach for knowledge enhancement;

[0042] Figure 5 Develop a conceptual model diagram for the networking of knowledge;

[0043] Figure 6 This is a schematic diagram of the control framework that complements knowledge-driven and neural network;

[0044] Figure 7This is a schematic diagram of the structure of the group intelligent decision-making device driven by both data and knowledge proposed in the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0046] Figure 1 The schematic diagram of the group intelligent decision-making method driven by data and knowledge is shown. It should be understood that in other embodiments, the order of some steps can be interchanged according to actual needs, or some steps can be omitted or deleted. Figure 1 As shown, the method includes:

[0047] Step S1: constructing a swarm intelligence system, which is a system used by the unmanned cluster to provide decision results to the unmanned cluster to assist the unmanned cluster in performing operations, including a knowledge processing layer and a data processing layer;

[0048] The unmanned cluster state data, the real-time sensed position data of the unmanned cluster, and the captured visual data are used as inputs of the swarm intelligence system; a knowledge processing controller and a neural network controller are configured independently of the swarm intelligence system, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data, and the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster;

[0049] The knowledge processing layer receives the first control instruction issued by the knowledge processing controller and the knowledge data generated based on the posture data and the visual data, compares the knowledge data with the prior knowledge in the knowledge base in the knowledge processing layer, and generates a knowledge matching decision result; the data processing layer integrates a variety of learning models, each learning model receives the second control instruction issued by the neural network controller and the unmanned cluster state data, modifies the parameters of each learning model, and obtains the output of each modified learning model as the correction result; based on the controller coordination mode, the knowledge matching decision result and the correction result, generates a final decision result; and uses the final decision result as the output of the swarm intelligence system;

[0050] Step S2: Based on the final decision result, each individual in the unmanned cluster makes an individual decision, and the individual decision is used to guide the individual's behavior and action.

[0051] In the present invention, the unmanned cluster is, for example, a drone cluster, and each individual is an intelligent agent.

[0052] Furthermore, the controller coordination mode includes a parallel combination mode, a knowledge priority series combination mode, and a data priority series combination mode.

[0053] The controller coordination mode, the knowledge matching decision result and the correction result are used to generate the knowledge matching decision result calculated by the neural network model.

[0054] When the controller coordination mode is the parallel combination mode, the knowledge matching decision result is used as the output of the knowledge processing layer, and the correction result is used as the output of the data processing layer; the knowledge matching decision result and the correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

[0055] The control law u1 of the parallel combination mode is:

[0056] u1=K(y,D,M,p)+N(y,w(P(y,u)))

[0057] Where K represents the knowledge processing controller, and the output is u k , N represents the learning model, and the output is u n , y=[y m ,y sp ], where y sp is the set value of the controlled quantity, y m is its measurement value, D, M represent the state model and output model in the prior knowledge respectively, p is the prior model parameter, w is the learning model weight, which is adjusted according to the performance indicator function P; at the same time, the parameters in the knowledge processing controller can also be adjusted according to P.

[0058] When the controller coordination mode is the knowledge priority serial combination mode, the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster and the input of the swarm intelligence system to obtain a correction result; the correction result is split, and the corresponding unmanned cluster data and unmanned cluster individual data are split from the correction result; the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the split unmanned cluster data, and uses the knowledge matching decision result generated based on the first control instruction as the output of the knowledge layer; the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster and the split unmanned cluster individual data, and obtains a correction result again as a second correction result; the knowledge matching decision result and the second correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

[0059] The control law u2 of the knowledge priority series combination mode is:

[0060] u2=N(K(y,D,M,p),y,w(P(y,u)))

[0061] Where K represents the knowledge processing controller, and the output is u k , N represents the learning model, and the output is u n , y=[y m ,y sp ], where y sp is the set value of the controlled quantity, y m is its measurement value, D, M represent the state model and output model in the prior knowledge respectively, p is the prior model parameter, w is the learning model weight, which is adjusted according to the performance indicator function P; at the same time, the parameters in the knowledge processing controller can also be adjusted according to P.

[0062] When the controller coordination mode is the data priority serial combination mode, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the input of the swarm intelligence system to obtain a knowledge matching decision result; the knowledge matching decision result is split, and the corresponding unmanned cluster data and unmanned cluster individual data are split from the knowledge matching decision result; the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the split unmanned cluster data, and uses the knowledge matching decision result generated again based on the first control instruction as the second knowledge matching decision result, and the second knowledge matching decision result is used as the output of the knowledge layer; the neural network controller generates a second control instruction based on the unmanned cluster real-time data and the split unmanned cluster individual data to obtain a correction result; the second knowledge matching decision result and the correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

[0063] The control law u3 of the data priority series combination mode is:

[0064] u3=K(y,D,M,N(I,w(P(y,u))))

[0065] Where K represents the knowledge processing controller, and the output is u k , N represents the learning model, and the output is u n , y=[y m ,y sp ], where y sp is the set value of the controlled quantity, y m is its measurement value, D, M represent the state model and output model in the prior knowledge respectively, p is the prior model parameter, w is the learning model weight, which is adjusted according to the performance indicator function P; at the same time, the parameters in the knowledge processing controller can also be adjusted according to P, and I is the relevant input of the neural network model.

[0066] I is the relevant input of the neural network model. These different combinations reflect different practical significances. In framework A, a data-driven model is often used to construct an uncertainty compensation model, thereby achieving algorithm optimization and robustness enhancement; in framework B, a neural network can be used to estimate the system inverse dynamics model, and then a knowledge-driven model is used to control it; in framework C, the role of the neural network is to estimate the parameter p in the knowledge-driven controller.

[0067] In the present invention, there is no obvious primary-secondary relationship between knowledge-driven and data-driven methods, and the two are tightly integrated in different forms. The combination of model-based knowledge-driven methods and learning model-based data-driven methods can be mainly divided into two categories from the perspective of architecture: parallel combination and series combination. In the parallel combination, the knowledge-driven and data-driven models use the same input, and the output results of the two are connected in parallel at the output end; in the series combination, the output of the knowledge-driven model can be used as the input of the data-driven model, or vice versa. The output of the data-driven model can be used as the input of the knowledge-driven model. Taking control system design as an example, the two combinations have derived three common system frameworks, such as Figure 6 shown.

[0068] In framework A, the control law u is:

[0069] u=K(y,D,M,p)+N(y,w(P(y,u)))

[0070] Where K represents the knowledge-driven controller, and the output is u k , N represents the neural network, and the output is u n , y=[y m ,y sp ], where y sp is the set value of the controlled quantity, y m is its measurement value, D and M represent the state model and output model in the prior knowledge respectively, p is the prior model parameter, and w is the neural network weight, which is adjusted according to the performance indicator function P; at the same time, the parameters in the knowledge-driven controller can also be adjusted according to P.

[0071] Similarly, the control law in framework B can be expressed as:

[0072] u=N(K(y,D,M,p),y,w(P(y,u)))

[0073] The control law in frame C can be expressed as:

[0074] u=K(y,D,M,N(I,w(P(y,u))))

[0075] Among them, I is the relevant input of the neural network model. These different combinations reflect different practical significances. In framework A, a data-driven model is often used to construct an uncertainty compensation model, thereby achieving algorithm optimization and robustness enhancement; in framework B, a neural network can be used to estimate the system inverse dynamics model, and then a knowledge-driven model is used to control it; in framework C, the role of the neural network is to estimate the parameter p in the knowledge-driven controller.

[0076] In order to optimize the group intelligent decision-making method, this application proposes a knowledge and data dual-driven algorithm model, the conceptual model of which is shown in Figure 2 In terms of data-driven, the mainstream approach is to use classic end-to-end machine learning algorithms, that is, input the initial state into the black box model and output the required results, such as object classification through the perception model and prediction of intelligent agent behavior through the decision model.

[0077] However, for complex systems and complex tasks, especially the complex tasks faced by unmanned swarms, end-to-end learning models are difficult to work, and a reasonable swarm intelligence system architecture is particularly important. The research on the swarm intelligence system architecture is based on at least two needs. One is to sort out the common mechanisms and logical processes in different complex tasks, which helps to explore the inherent invariance mechanism of the problem and perform standardized modeling; the other is to decompose complex problems into several sub-problems that are easier to solve, greatly reducing the complexity of problem processing.

[0078] Furthermore, each learning model integrated within the data processing layer is divided into multiple levels according to the type of data processed, and the learning models at different levels are then constructed as components.

[0079] The present invention uses knowledge to guide the generation of data models, and generates new knowledge from the data models, forming an alternating iteration of knowledge and data. It includes the architecture-level collaboration of knowledge and data, and the algorithmic collaboration of knowledge and data. The architecture-level collaboration of knowledge and data is used to describe the common mechanisms and logical processes in different complex tasks, and decompose complex problems into several sub-problems that are easier to solve. The system architecture acts as an algorithm container, allowing algorithms with different driving modes to form organic collaboration. The algorithmic collaboration of knowledge and data includes hierarchical collaborative algorithms and component-based collaborative algorithms.

[0080] According to the primary and secondary positions of knowledge-driven and data-driven methods, this application proposes a method of complementary combination of knowledge and data. In the complementary combination of knowledge and data, the primary and secondary relationship between the two types of methods is relatively unclear. The two will form an integrated algorithm in a complementary way to jointly complete group intelligent decision-making.

[0081] Data driven models such as Figure 4As shown in the figure, the model is composed of reinforcement learning and deep learning models, and prior knowledge needs to be input from outside the model to provide a basis for imitation learning. The search state space and strategy space in multi-agent reinforcement learning are huge, and due to problems such as sparse rewards and delayed returns, it is very difficult to learn decisions made many steps ago based on cumulative rewards. In real tasks, a group of expert decision-making process examples can be obtained, which allows the reinforcement learning model to directly imitate the expert's example trajectory to alleviate the aforementioned difficulties. This method is imitation learning.

[0082] According to the object of "imitation" in the reinforcement learning framework, imitation learning can be further divided into direct imitation learning and inverse reinforcement learning. In direct imitation learning, the expert's "state-action pair" example data is first obtained, and then supervised learning is used to learn a policy model that conforms to the expert's decision trajectory. The application scenarios of group intelligent decision clusters in actual physical environments often lack human experience or prior data, but knowledge-driven models can provide objects for imitation learning for data-driven models by providing prior knowledge.

[0083] In the knowledge-driven model, in order to facilitate the combination with the data-driven model, knowledge is expanded into a data representation, stored and operated in the form of a neural network, thus forming a special form of knowledge nested network. The structure and parameters of the network will reflect the characteristics of domain or expert knowledge. The network can be further embedded in a larger neural network for unified training and learning. The conceptual model of the neural network is as follows: Figure 5 shown.

[0084] Furthermore, the learning includes a deep learning model and a reinforcement learning model, and the deep learning model includes a neural network tree, a genetic fuzzy tree, and a hierarchical reinforcement learning model.

[0085] The neural network tree integrates multiple neural network tree models to obtain the optimal result, so as to improve the performance and stability of the overall model.

[0086] The genetic fuzzy tree uses a cross-validation approach to evaluate performance and ensure the generalization ability of the model.

[0087] The hierarchical reinforcement model achieves transfer learning by sharing part of the knowledge or strategy.

[0088] Furthermore, the reinforcement learning model includes a Markov decision model, a value function model, and a policy gradient model.

[0089] The Markov decision model guides decision making through the estimation of value functions, including state value functions and action value functions, and improves the accuracy and efficiency of decision making by optimizing the estimation of value functions.

[0090] The value function model stores the experience gained during the interaction between the intelligent agent and the environment in an experience pool, and then randomly samples from the experience pool for training, thereby improving the utilization efficiency of samples and the stability of the algorithm.

[0091] The policy gradient model adopts a truncated policy gradient method to reduce the variance of gradient updates and improve stability and convergence speed by limiting the size of the gradient.

[0092] In the present invention, in addition to neural networks, reinforcement learning is also used to form a complementary combination with the knowledge-driven method. Q-learning is used to form a compensation controller, which works together with a model-based benchmark controller to achieve stable control of a quadrotor drone; similarly, a type-two fuzzy method is used to form a benchmark controller, and a reinforcement learning method based on deep deterministic policy gradient (DDPG) is used to form a complementary controller to achieve control of the power grid regulation frequency. In the series combination mode, a shield is added to the strategy learning framework to supervise whether the learned action is safe and reasonable. There are two specific combination modes. One is to directly obtain a safe behavior from the shield when the intelligent agent makes a decision, and the other is to supervise the learning of the intelligent agent. Once an unsafe behavior occurs, the shield will correct the action; a similar idea is also adopted in MOBA games, using an action mask to prune the exploration process of reinforcement learning, and the design of the mask inherits the prior knowledge of experienced human players. Of course, whether it is the shield method or the action mask method, its knowledge-driven part is only a component of the data-driven part, and still reflects a certain degree of priority. It should be classified as the aforementioned knowledge-enhanced data-driven method. The introduction here mainly reflects its serial combination characteristics.

[0093] In the present invention, the neural network model represents data-driven, and the decision tree structure represents knowledge-driven. Its essence is to effectively organize several neural network models with the structure of a decision tree, so that it has the advantages of strong interpretability of the decision tree model, easy integration of expert knowledge, and autonomous learning of the neural network model. On the basis of the tree structure, fuzzy logic is used to establish a reasoning framework based on expert knowledge, and a genetic algorithm is used to optimize the parameters of the antecedent and antecedent rules in fuzzy reasoning, so that the tree structure further characterizes the hierarchical architecture in complex problems. The present invention adopts the idea of ​​strategic stratification and divide and conquer, introduces a large amount of prior or domain knowledge, and solves the dimensionality disaster caused by the exponential growth of the state space and action space. According to the input of prior knowledge, the present invention performs data-driven knowledge enhancement through imitation learning, heuristic reward function design, and deep learning model. The present invention uses the powerful optimization ability of the data-driven method to realize the optimization of the structure or parameters in the knowledge-driven method. There is no obvious primary and secondary relationship between the knowledge-driven and data-driven methods of the present invention. The two are tightly integrated in different forms, and are mainly divided into two categories from the perspective of architecture: parallel combination and series combination.

[0094] The present invention provides a specific embodiment of a group intelligent decision-making method driven by both data and knowledge.

[0095] To construct a drone swarm of common formations, we adopt a method of combining knowledge with complementary knowledge. First, we input information such as the attitude and environment of the drone into the knowledge-driven controller, and use the output of the knowledge-driven controller as the input of the data-driven neural network controller, and the output is the group decision of the drone swarm. At the same time, we input information such as the attitude and environment of the drone into the neural network controller, and use the output of the data-driven controller as the input of the knowledge-driven controller, and the output is also the group decision of the drone swarm. We combine performance indicators and action goals as training purposes, and strengthen and correct the above two training processes, so that the performance of the above training processes tends to be consistent and the degree of completion of the action goals tends to be complete.

[0096] Through complementary iterations of data-driven and knowledge-driven approaches, a group decision-making method for specific formations and classic environments is formed.

[0097] The following is a description of the device used to execute the present application. The specific implementation process and technical effects are described above and will not be repeated below.

[0098] Optionally, an embodiment of the present application provides a group intelligent decision-making device driven by both data and knowledge, the device comprising:

[0099] Swarm intelligence system building module: configured to build a swarm intelligence system, which is a system used by the unmanned cluster to provide decision results to the unmanned cluster to assist the unmanned cluster in performing operations, including a knowledge processing layer and a data processing layer;

[0100] The unmanned cluster state data, the real-time sensed position data of the unmanned cluster, and the captured visual data are used as inputs of the swarm intelligence system; a knowledge processing controller and a neural network controller are configured independently of the swarm intelligence system, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data, and the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster;

[0101] The knowledge processing layer receives the first control instruction issued by the knowledge processing controller and the knowledge data generated based on the posture data and the visual data, compares the knowledge data with the prior knowledge in the knowledge base in the knowledge processing layer, and generates a knowledge matching decision result; the data processing layer integrates a variety of learning models, each learning model receives the second control instruction issued by the neural network controller and the unmanned cluster state data, modifies the parameters of each learning model, and obtains the output of each modified learning model as the correction result; based on the controller coordination mode, the knowledge matching decision result and the correction result, generates a final decision result; and uses the final decision result as the output of the swarm intelligence system;

[0102] Output module: configured so that the unmanned cluster makes individual decisions based on the final decision result, and the individual decisions are used to guide the behaviors and actions of the individuals.

[0103] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0104] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0105] The above modules can be connected or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may include a connection in the form of a LAN, a WAN, Bluetooth, a ZigBee, or NFC, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units. Those skilled in the art will clearly understand that for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application.

[0106] It should be noted that the above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program codes. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0107] The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0108] The present application also provides a program product, such as a computer-readable storage medium, including a program, which is used to execute the above method embodiment when executed by a processor.

[0109] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0112] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

Claims

1. A data and knowledge-driven group intelligent decision-making method, characterized in that: The method comprises the following steps: Step S1: constructing a swarm intelligence system, which is a system used by the unmanned cluster to provide decision results to the unmanned cluster to assist the unmanned cluster in performing operations, including a knowledge processing layer and a data processing layer; The unmanned cluster state data, the real-time sensed position data of the unmanned cluster, and the captured visual data are used as inputs of the swarm intelligence system; a knowledge processing controller and a neural network controller are configured independently of the swarm intelligence system, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data, and the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster; The knowledge processing layer receives the first control instruction issued by the knowledge processing controller and the knowledge data generated based on the posture data and the visual data, compares the knowledge data with the prior knowledge in the knowledge base in the knowledge processing layer, and generates a knowledge matching decision result; the data processing layer integrates a plurality of learning models, each learning model receives the second control instruction issued by the neural network controller and the unmanned cluster state data, modifies the parameters of each learning model, and obtains the output of each modified learning model as the modification result; Generate a final decision result based on the controller coordination mode, the knowledge matching decision result and the correction result; Using the final decision result as the output of the swarm intelligence system; Step S2: Based on the final decision result, each individual in the unmanned cluster makes an individual decision, and the individual decision is used to guide the individual's behavior and action.

2. The method according to claim 1, characterized in that The controller coordination modes include a parallel combination mode, a knowledge priority series combination mode, and a data priority series combination mode.

3. The method according to claim 2, characterized in that The controller coordination mode, the knowledge matching decision result and the correction result are based on generating the knowledge matching decision result calculated by the neural network model, wherein: When the controller coordination mode is the parallel combination mode, the knowledge matching decision result is used as the output of the knowledge processing layer, and the correction result is used as the output of the data processing layer; the knowledge matching decision result and the correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

4. The method according to claim 2, characterized in that The controller coordination mode, the knowledge matching decision result and the correction result are based on generating the knowledge matching decision result calculated by the neural network model, wherein: When the controller coordination mode is the knowledge priority serial combination mode, the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster and the input of the swarm intelligence system to obtain a correction result; the correction result is split, and the corresponding unmanned cluster data and unmanned cluster individual data are split from the correction result; the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the split unmanned cluster data, and uses the knowledge matching decision result generated based on the first control instruction as the output of the knowledge layer; the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster and the split unmanned cluster individual data, and obtains a correction result again as a second correction result; the knowledge matching decision result and the second correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

5. The method according to claim 2, characterized in that The controller coordination mode, the knowledge matching decision result and the correction result are based on generating the knowledge matching decision result calculated by the neural network model, wherein: When the controller coordination mode is the data priority serial combination mode, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the input of the swarm intelligence system to obtain a knowledge matching decision result; the knowledge matching decision result is split, and the corresponding unmanned cluster data and unmanned cluster individual data are split from the knowledge matching decision result; the knowledge processing controller generates a first control instruction based on the unmanned cluster state data and the split unmanned cluster data, and uses the knowledge matching decision result generated again based on the first control instruction as the second knowledge matching decision result, and the second knowledge matching decision result is used as the output of the knowledge layer; the neural network controller generates a second control instruction based on the unmanned cluster real-time data and the split unmanned cluster individual data to obtain a correction result; the second knowledge matching decision result and the correction result are fused to obtain a fusion result, and the fusion result is used as the final decision result.

6. A data and knowledge-driven group intelligent decision-making device, characterized in that: The device comprises: Swarm intelligence system building module: configured to build a swarm intelligence system, which is a system used by the unmanned cluster to provide decision results to the unmanned cluster to assist the unmanned cluster in performing operations, including a knowledge processing layer and a data processing layer; The unmanned cluster state data, the real-time sensed position data of the unmanned cluster, and the captured visual data are used as inputs of the swarm intelligence system; a knowledge processing controller and a neural network controller are configured independently of the swarm intelligence system, the knowledge processing controller generates a first control instruction based on the unmanned cluster state data, and the neural network controller generates a second control instruction based on the real-time data of the unmanned cluster; The knowledge processing layer receives the first control instruction issued by the knowledge processing controller and the knowledge data generated based on the posture data and the visual data, compares the knowledge data with the prior knowledge in the knowledge base in the knowledge processing layer, and generates a knowledge matching decision result; the data processing layer integrates a variety of learning models, each learning model receives the second control instruction issued by the neural network controller and the unmanned cluster state data, modifies the parameters of each learning model, and obtains the output of each modified learning model as the correction result; based on the controller coordination mode, the knowledge matching decision result and the correction result, generates a final decision result; and uses the final decision result as the output of the swarm intelligence system; Output module: configured so that the unmanned cluster makes individual decisions based on the final decision result, and the individual decisions are used to guide the behaviors and actions of the individuals.

7. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images