Joint decision-making method and device based on man-machine mixing

Abstract

The invention provides a joint decision-making method and device based on man-machine mixing, and is applied to the technical field of artificial intelligence. The method comprises the following steps: acquiring visible track information and agent observable state information of an unseen teammate; determining a historical information representation vector according to the visible track information, and determining observable state representation according to the agent observable state information; determining non-teammate style information according to the historical information representation vector and the observable state representation; inputting the non-teammate-seeing style information into a joint decision network based on man-machine mixing to obtain teammate actions; inputting the teammate action into a pulse neural network to obtain a decision action; wherein the joint decision network based on man-machine mixing is constrained through a determinant point process so as to simulate self-learning integration of different styles. Through man-machine cooperation, the ability of teammates can be quickly received, so that a more robust and generalized decision result is obtained.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a joint decision-making method and device based on human-machine hybrid. Background Art

[0002] In recent years, a large number of real decision-making scenarios involve multiple entities with complex relationships between them, so the deployment of multi-agent reinforcement learning has received increasing attention.

[0003] A major challenge in multi-agent reinforcement learning algorithms is the problem of generalization. Generalization in multi-agents often involves generalization of opponents and teammates. The algorithm needs to quickly adapt to new opponents and teammates in order to efficiently complete tasks in real-world scenarios. However, existing reinforcement learning methods have difficulty achieving fast and effective generalization. Summary of the invention

[0004] The present invention provides a joint decision-making method and device based on human-machine hybrid, which are used to solve the defect that the existing reinforcement learning method is difficult to achieve fast and effective generalization.

[0005] The present invention provides a joint decision-making method based on human-machine hybrid, comprising: obtaining visible trajectory information of unseen teammates and observable state information of an intelligent agent; determining a historical information representation vector according to the visible trajectory information, and determining an observable state representation according to the observable state information of the intelligent agent; determining style information of unseen teammates according to the historical information representation vector and the observable state representation; inputting the unseen teammate style information into a joint decision-making network based on human-machine hybrid to obtain teammate actions; inputting the teammate actions into a pulse neural network to obtain decision actions; wherein the joint decision-making network based on human-machine hybrid is constrained by a determinant point process to simulate self-learning integration of different styles.

[0006] According to a joint decision-making method based on human-machine hybrid provided by the present invention, the historical information representation vector is determined according to the visible trajectory information, and the observable state representation is determined according to the observable state information of the agent, including: determining the observable state representation corresponding to the observable state information of the agent through a state encoder; determining the historical information representation vector corresponding to the visible trajectory information through a historical information encoder.

[0007] According to a joint decision-making method based on human-machine hybrid provided by the present invention, the determining of unseen teammate style information based on the historical information representation vector and the observable state representation includes: determining the unseen teammate style information corresponding to the historical information representation vector and the observable state representation through a multi-scale biologically reasonable mind model; wherein the multi-scale biologically reasonable mind model includes a multi-personality network composed of multiple base personality networks and a personality estimator.

[0008] According to a joint decision-making method based on human-machine hybrid provided by the present invention, the pulse neural network is a pulse actor network.

[0009] The present invention also provides a joint decision-making device based on human-machine hybrid, comprising the following modules: an acquisition module and a processing module; the acquisition module is used to acquire visible trajectory information of unseen teammates and observable state information of intelligent agents; the processing module is used to determine a historical information representation vector based on the visible trajectory information, and determine an observable state representation based on the observable state information of the intelligent agent; determine the style information of unseen teammates based on the historical information representation vector and the observable state representation; input the unseen teammate style information into a joint decision network based on human-machine hybrid to obtain teammate actions; input the teammate actions into a pulse neural network to obtain decision actions; wherein the joint decision network based on human-machine hybrid is constrained by a determinant point process to simulate self-learning integration of different styles.

[0010] According to a joint decision-making device based on human-machine hybrid provided by the present invention, the processing module is used to determine the observable state representation corresponding to the observable state information of the intelligent agent through a state encoder; and determine the historical information representation vector corresponding to the visible trajectory information through a historical information encoder.

[0011] According to a human-machine hybrid joint decision-making device provided by the present invention, the processing module is used to determine the historical information representation vector and the unseen teammate style information corresponding to the observable state representation through a multi-scale biologically reasonable mind model; wherein the multi-scale biologically reasonable mind model includes a multi-personality network composed of multiple base personality networks and a personality estimator.

[0012] According to a joint decision-making device based on human-machine hybrid provided by the present invention, the pulse neural network is a pulse actor network.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a human-machine hybrid joint decision-making method as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described human-machine hybrid joint decision-making methods.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned human-machine hybrid joint decision-making methods.

[0016] The joint decision-making method and device based on human-machine hybrid provided by the present invention can obtain the visible trajectory information of unseen teammates and the observable state information of the intelligent agent; determine the historical information representation vector according to the visible trajectory information, and determine the observable state representation according to the observable state information of the intelligent agent; determine the style information of unseen teammates according to the historical information representation vector and the observable state representation; input the unseen teammate style information into the joint decision-making network based on human-machine hybrid to obtain the teammate action; input the teammate action into the pulse neural network to obtain the decision action; wherein the joint decision-making network based on human-machine hybrid is constrained by the determinant point process to simulate the self-learning integration of different styles. Through this scheme, the joint decision-making network based on human-machine hybrid is constrained by the determinant point process, and its intrinsic reward is given by the determinant point process, which can ensure the difference between different base style strategies, so as to quickly receive the teammate ability and obtain more robust generalization results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the joint decision-making method based on human-machine hybrid provided by the present invention; Figure 2 This is the second flow chart of the joint decision-making method based on human-machine hybrid provided by the present invention; Figure 3 It is a structural schematic diagram of a joint decision-making device based on human-machine hybrid provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0021] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0022] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. are not limiting the quantity and execution order.

[0023] The embodiments of the present application describe some exemplary embodiments for the purpose of explanation. It should be understood that the present application can be implemented in other ways that are not specifically shown in the drawings.

[0024] This application designs a multi-agent learning framework Mixture of personality model (MoP) for generalization of unseen teammates that does not require additional expert datasets. The entire framework is divided into two stages: learning stage and generalization verification stage. In the learning stage, the reward return includes external environment rewards and intrinsic rewards. The intrinsic reward is given by the determinantal point process (DPP) to ensure the differences between different base personality strategies. In the generalization test stage, efficient cooperation with unseen teammates is achieved, and the ability to quickly receive new teammates is stronger, which can achieve more robust generalization results.

[0025] like Figure 1 As shown, the embodiment of the present application provides a joint decision-making method based on human-machine hybrid, and the joint decision-making method based on human-machine hybrid can be applied to a joint decision-making device based on human-machine hybrid. The joint decision-making method based on human-machine hybrid can include S101-S105: S101, a joint decision-making device based on human-machine hybrid obtains visible trajectory information of invisible teammates and observable state information of the intelligent agent.

[0026] S102, a joint decision-making device based on human-machine hybrid determines a historical information representation vector according to the visible trajectory information, and determines an observable state representation according to the observable state information of the intelligent agent.

[0027] Optionally, determining the historical information representation vector according to the visible trajectory information and determining the observable state representation according to the observable state information of the agent include: determining the observable state representation corresponding to the observable state information of the agent through a state encoder; and determining the historical information representation vector corresponding to the visible trajectory information through a historical information encoder.

[0028] Specifically, Figure 2 As shown in the figure, the input module includes a state encoder and a history information encoder. In the context encoder, the input is the partially visible trajectory information of the unseen teammates, and the output is the teammate's history information representation vector. In the observation encoder, the input is the partially observable state information of the current unseen teammate agent, and the output is the partially observable state representation of the unseen teammates.

[0029] S103: The human-machine hybrid joint decision-making device determines the unseen teammate style information according to the historical information representation vector and the observable state representation.

[0030] Optionally, determining the unseen teammate style information based on the historical information representation vector and the observable state representation includes: determining the unseen teammate style information corresponding to the historical information representation vector and the observable state representation through a multi-scale biologically reasonable mind model; wherein the multi-scale biologically reasonable mind model includes a multi-personality network composed of multiple base personality networks and a personality estimator.

[0031] Specifically, Figure 2 As shown, the multi-scale biologically reasonable mind model consists of a multi-personality network composed of multiple base personality networks and a personality estimator. Specifically, the multi-personality network in the multi-scale biologically reasonable mind model contains multiple base personality strategy networks, and the teammate's personality can be combined from these base personalities. The personality estimator is used to realize the personality estimation of the teammate, and then the multi-personality network predicts the teammate's action based on the estimated personality.

[0032] In the personality estimator, the input is the teammate's historical information representation vector, and the output is the personality representation of the unseen teammate, that is, the unseen teammate's style information. As shown in the following formula: ; in, Represents the historical information representation vector of teammates who have not met each other, and represents the learnable coefficient, express function.

[0033] The personality estimator completes the personality estimation of unseen teammates. In addition to the learnable coefficients In order to make the personality estimator robust enough for generalization, the personality estimator also includes learnable coefficients , the R function can make the noise term in a very small proportion.

[0034] S104. The human-machine hybrid joint decision-making device inputs the unseen teammate style information into the human-machine hybrid joint decision-making network to obtain teammate actions.

[0035] Among them, the above-mentioned joint decision-making network based on human-machine hybrid is constrained by a determinant point process to simulate the self-learning integration of different styles.

[0036] like Figure 2 As shown in the figure, accurate prediction of the actions of unseen teammates depends on accurate prediction of personality and accurate prediction of the actions that each base personality will take. In MoP, there are k base personality networks. The determination of k depends on the personality theory in cognitive psychology. Our goal is to build k base personality networks to cover the personality space, more effectively complete the rapid adaptation and generalization of unseen teammates, and ensure that the differences between these k base personality networks are large enough: ; in, Represents a multiscale biologically plausible mind model in MoP, Represents the teammate's joint decision-making based on human-machine hybrid, Represents observation information of the environment, Indicates that the historical information of teammates has not been seen. represents the action distribution of the base personality network i at time step t, Indicates the estimated personality of teammates.

[0037] In order to make the differences between these k base personality networks large enough or even orthogonal, we introduced the determinant point process DPP. The determinant point process DPP is a high-performance probability model that converts complex probability calculations into simple determinant calculations to measure the overall similarity of elements in a set. The determinant of the kernel matrix of a set corresponds to the similarity of the set. Through DPP, we can find the subsets and k subsets with the largest diversity. This method will serve as the source of intrinsic rewards for our model.

[0038] In our teammate modeling, we introduce the determinant point process DPP constraint to constrain the diversity of the modeling of unseen teammates. This is because people are often dynamic and changeable, and they will adopt different strategies in different situations or even in different moods. Through such diversity constraints, we ensure that our model framework can adapt to a variety of human teammates and can generalize quickly during testing. As shown below, each item in the kernel matrix represents the similarity between two items in the set. The diversity of the set can be measured by calculating the determinant of the kernel matrix.

[0039] The determinant point process can increase the diversity of elements in the set. The determinant point process is to maximize the kernel matrix of the set y. The determinant value is used to select the set y with the maximum diversity: ; The kernel matrix of the set y The calculation process is as follows: ; in, Represents the mapping matrices that constitute the kernel matrix.

[0040] For the matrix The calculation of each basic personality is completed through an additional neural network: ; in, represents the characterization mapping process, represents the action distribution of the base personality network k at time step t.

[0041] The determinant point process module introduces intrinsic rewards to the overall mental model framework as follows. Compared with environmental rewards, intrinsic rewards also have a relatively important weight: ; in, represents the intrinsic reward of the mental model framework, Indicates the parameters of the DPP module part.

[0042] S105. The human-machine hybrid joint decision-making device inputs the teammate action into the pulse neural network to obtain a decision action.

[0043] Optionally, the spiking neural network is a spiking actor network.

[0044] Specifically, in terms of the agent's strategy network, we selected the spiking actor network based on the spiking neural network as the strategy network. Unseen teammate generalization can be regarded as a noise perturbation test on teammates. First, we selected the spiking neural network as the strategy network structure. The formula of the spiking neural network is as follows: ; in, represents the membrane potential dynamic variable, Represents the postsynaptic integrated potential information.

[0045] Inspired by the important psychological foundation of the theory of mind, the multi-personality theory, the spiking neural network and the multi-personality theory can be applied to multi-agent cooperation scenarios to solve the difficulty of generalization when the teammate has not been seen. As shown in the following formula: ; in, represents the multi-agent cooperative joint strategy, Indicates joint action, represents the joint state estimation, represents the strategy of our agent, represents the actions of one's own agent, represents the state of one's own agent, represents the teammates’ actions predicted by the mental model, represents the teammate agent strategy, represents the actions of the teammate agent, Represents the observation of the teammate agent.

[0046] In this unseen teammate generalization task, the additional predictive information of the mental model is used to assist agent 1 in generalizing unseen teammate 2. The overall joint strategy of the agent team is as follows: and is the observation information of agent 1 and agent 2.

[0047] In the embodiments of the present application, we designed a multi-agent learning framework for generalization of unseen teammates that does not require additional expert data sets. For the first time, we applied biologically plausible spiking neural networks to the generalization task of multi-agent unseen teammates to reduce energy consumption. We constructed a multi-scale biologically plausible mind model and applied spiking neural networks (SNN) and multi-personality theory to multi-agent cooperation scenarios to solve the difficulty of generalization of unseen teammates. The model inspired by personality theory in the theory of mind is a two-stage model. In the generalization test stage, efficient cooperation with unseen teammates can be achieved, new teammates can be quickly accepted, and more robust generalization results can be obtained.

[0048] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0049] It should be noted that the device in the embodiment of the present application includes a virtual device and a physical device. The virtual device can be a joint decision-making device based on human-machine hybrid, and the physical device can include an electronic device, a computer storage medium, and a computer program product.

[0050] In the joint decision-making method based on human-machine hybrid provided in the embodiment of the present application, the execution subject may be a joint decision-making device based on human-machine hybrid, or a control module for joint decision-making based on human-machine hybrid in the joint decision-making device based on human-machine hybrid. In the embodiment of the present application, the joint decision-making method based on human-machine hybrid is executed by a joint decision-making device based on human-machine hybrid as an example to illustrate the joint decision-making device based on human-machine hybrid provided in the embodiment of the present application.

[0051] It should be noted that the embodiment of the present application can divide the functional modules of the joint decision-making device based on human-machine hybrid according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0052] like Figure 3 As shown, an embodiment of the present application provides a joint decision-making device 300 based on human-machine hybrid. The joint decision-making device 300 based on human-machine hybrid includes: an acquisition module 301 and a processing module 302. The acquisition module 301 is used to acquire the visible trajectory information of unseen teammates and the observable state information of the intelligent agent; the processing module 302 is used to determine the historical information representation vector according to the visible trajectory information, and determine the observable state representation according to the observable state information of the intelligent agent; determine the style information of unseen teammates according to the historical information representation vector and the observable state representation; input the style information of unseen teammates into the joint decision network based on human-machine hybrid to obtain the teammate action; input the teammate action into the pulse neural network to obtain the decision action; wherein, the joint decision network based on human-machine hybrid is constrained by the determinant point process to simulate the self-learning integration of different styles.

[0053] Optionally, the processing module 302 is used to determine the observable state representation corresponding to the observable state information of the agent through a state encoder; and determine the historical information representation vector corresponding to the visible trajectory information through a historical information encoder.

[0054] Optionally, the processing module 302 is used to determine the historical information representation vector and the unseen teammate style information corresponding to the observable state representation through a multi-scale biologically reasonable mind model; wherein the multi-scale biologically reasonable mind model includes a multi-personality network composed of multiple base personality networks and a personality estimator.

[0055] Optionally, the spiking neural network is a spiking actor network.

[0056] In the embodiment of the present application, the joint decision network based on human-machine hybrid is constrained by a determinant point process, and its intrinsic reward is given by the determinant point process, which can ensure the differences between different base style strategies, thereby quickly receiving the capabilities of teammates and achieving more robust generalized results.

[0057] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the joint decision-making method based on human-machine mixing, the method comprising: obtaining the visible trajectory information of the unseen teammates and the observable state information of the intelligent agent; determining the historical information representation vector according to the visible trajectory information, and determining the observable state representation according to the observable state information of the intelligent agent; determining the style information of the unseen teammates according to the historical information representation vector and the observable state representation; inputting the unseen teammate style information into the joint decision network based on human-machine mixing to obtain the teammate action; inputting the teammate action into the pulse neural network to obtain the decision action; wherein the joint decision network based on human-machine mixing is constrained by the determinant point process to simulate the self-learning integration of different styles.

[0058] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0059] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the joint decision-making method based on human-computer mixing provided by the above methods, the method including: obtaining visible trajectory information of unseen teammates and observable state information of the intelligent agent; determining a historical information representation vector based on the visible trajectory information, and determining an observable state representation based on the observable state information of the intelligent agent; determining style information of unseen teammates based on the historical information representation vector and the observable state representation; inputting the unseen teammate style information into a joint decision-making network based on human-computer mixing to obtain teammate actions; inputting the teammate actions into a pulse neural network to obtain decision actions; wherein, the joint decision-making network based on human-computer mixing is constrained by a determinant point process to simulate self-learning integration of different styles.

[0060] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the joint decision-making method based on human-machine hybrid provided by the above-mentioned methods, the method comprising: obtaining visible trajectory information of unseen teammates and observable state information of the intelligent agent; determining a historical information representation vector based on the visible trajectory information, and determining an observable state representation based on the observable state information of the intelligent agent; determining style information of unseen teammates based on the historical information representation vector and the observable state representation; inputting the unseen teammate style information into a joint decision-making network based on human-machine hybrid to obtain teammate actions; inputting the teammate actions into a pulse neural network to obtain decision actions; wherein the joint decision-making network based on human-machine hybrid is constrained by a determinant point process to simulate self-learning integration of different styles.

[0061] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A joint decision-making method based on human-machine hybrid, characterized in that: include: Obtain the visible trajectory information of unseen teammates and the observable state information of the agent; Determine a historical information representation vector according to the visible trajectory information, and determine an observable state representation according to the observable state information of the agent; Determining unseen teammate style information according to the historical information representation vector and the observable state representation; Inputting the unseen teammate style information into a joint decision-making network based on human-machine hybrid to obtain teammate actions; Inputting the teammate action into the spiking neural network to obtain a decision action; The human-machine hybrid joint decision network is constrained by a determinant point process to simulate different styles of self-learning integration.

2. The joint decision-making method based on human-machine hybrid according to claim 1 is characterized in that: The determining of the historical information representation vector according to the visible trajectory information and the determining of the observable state representation according to the observable state information of the intelligent agent include: Determining, by a state encoder, an observable state representation corresponding to the observable state information of the agent; A historical information representation vector corresponding to the visible trajectory information is determined by a historical information encoder.

3. The joint decision-making method based on human-machine hybrid according to claim 1 is characterized in that: The determining the unseen teammate style information according to the historical information representation vector and the observable state representation includes: Determining the historical information representation vector and the unseen teammate style information corresponding to the observable state representation through a multi-scale biologically plausible mental model; The multi-scale biologically plausible mind model includes a multi-personality network composed of multiple base personality networks and a personality estimator.

4. The joint decision-making method based on human-machine hybrid according to claim 1 is characterized in that: The spiking neural network is a spiking actor network.

5. A joint decision-making device based on human-machine hybrid, characterized in that: include: Acquisition module and processing module; The acquisition module is used to obtain the visible trajectory information of the unseen teammates and the observable state information of the agent; The processing module is used to determine the historical information representation vector according to the visible trajectory information, determine the observable state representation according to the observable state information of the agent; determine the unseen teammate style information according to the historical information representation vector and the observable state representation; input the unseen teammate style information into the joint decision network based on human-machine hybrid to obtain the teammate action; input the teammate action into the spiking neural network to obtain the decision action; The human-machine hybrid joint decision network is constrained by a determinant point process to simulate different styles of self-learning integration.

6. The joint decision-making device based on human-machine hybrid according to claim 5 is characterized in that: The processing module is used to determine the observable state representation corresponding to the observable state information of the intelligent agent through a state encoder; and determine the historical information representation vector corresponding to the visible trajectory information through a historical information encoder.

7. The joint decision-making device based on human-machine hybrid according to claim 5 is characterized in that: The processing module is used to determine the historical information representation vector and the unseen teammate style information corresponding to the observable state representation through a multi-scale biologically reasonable mind model; wherein the multi-scale biologically reasonable mind model includes a multi-personality network composed of multiple base personality networks and a personality estimator.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the human-machine hybrid joint decision-making method as described in any one of claims 1 to 4 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the human-machine hybrid joint decision-making method according to any one of claims 1 to 4 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the human-machine hybrid joint decision-making method according to any one of claims 1 to 4 is implemented.

Images