A generalizable state representation method for game-based intelligent algorithms

Abstract

The present disclosure provides a generalizable state representation method for game-playing intelligent algorithms, belonging to the field of artificial intelligence technology. The method includes: obtaining a task scenario; dividing each agent into an N×N discrete grid based on the spatial distribution information of each agent in the task scenario; determining an N×N feature matrix corresponding to each type of global attribute information based on the global attribute information of the agent included in each discrete grid; determining an N×N observation matrix corresponding to each local attribute information of the agent based on the discrete grid where the agent is located and the observation range of the agent; determining a first eigenvector based on each feature matrix, and determining a second eigenvector based on each observation matrix, and obtaining a state space corresponding to the task scenario based on the fusion of the first eigenvector and the second eigenvector. This method is used to improve the generalizability of the state space.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a generalizable state representation method for game-playing intelligent algorithms. Background Art

[0002] In recent years, with the increase in computing power and advancements in machine learning, research on general decision-making frameworks has made significant progress. Researchers have attempted to construct general decision-making frameworks capable of handling a wide range of tasks through various approaches, such as meta-learning, transfer learning, and multi-task learning. Although research on general decision-making frameworks has made some progress in the field of reinforcement learning, they still face some significant limitations in practical applications, particularly in terms of generalization across state spaces.

[0003] Existing general decision-making frameworks are significantly affected by changes in the number of agents when dealing with state spaces. In multi-agent environments, changes in the number of agents can lead to drastic changes in the dimensionality and structure of the state space, posing a significant challenge to the generalization ability of decision-making models. Summary of the Invention

[0004] The embodiments of the present disclosure provide a generalizable state representation method for game-playing adversarial intelligent algorithms.

[0005] A first aspect of the embodiments of the present disclosure provides a generalizable state representation method for a game-based intelligent algorithm, including:

[0006] Acquire a task scenario, wherein the task scenario includes at least two agents, and includes global attribute information of each of the agents and spatial distribution information of each of the agents;

[0007] Dividing each of the intelligent agents into N×N discrete grids according to the spatial distribution information of each of the intelligent agents, wherein one of the intelligent agents belongs to one of the discrete grids;

[0008] Determining, based on the global attribute information of the agents included in each discrete grid, an N×N feature matrix corresponding to each type of global attribute information; wherein, in the feature matrix corresponding to any of the global attribute information, the value of the element in the i-th row and j-th column is the number of agents with the global attribute information included in the discrete grid in the i-th row and j-th column;

[0009] The following processing is performed on each of the agents: an N×N observation matrix corresponding to each local attribute information of the agent is determined based on the discrete grid in which the agent is located and the observation range of the agent; in the observation matrix corresponding to any local attribute information, the value of the element in the i-th row and j-th column is the number of agents in the i-th row and j-th column of the discrete grid that are located within the observation range of the agent and have the local attribute information;

[0010] A first eigenvector is determined based on each of the feature matrices, and a second eigenvector is determined based on each of the observation matrices. A state space corresponding to the task scenario is obtained based on the fusion of the first eigenvector and the second eigenvector. The state space is used for the reinforcement learning process of the intelligent agent.

[0011] The embodiments of the present disclosure have the following advantages:

[0012] According to the spatial distribution information of each of the intelligent agents in the task scenario, discretization processing is performed to obtain the discrete grid to which each intelligent agent belongs. According to the intelligent agents distributed in the discrete grid, the N×N feature matrix corresponding to each of the global attribute information is determined, thereby obtaining the global common characteristics of the task scenario; according to the intelligent agents distributed in the discrete grid and the observation range of the intelligent agents, the N×N observation matrix corresponding to each local attribute information of the intelligent agent is determined, thereby obtaining the local heterogeneous characteristics of each intelligent agent in the task scenario; by fusing the global common characteristics and the local heterogeneous characteristics, the state space finally used for reinforcement learning is obtained, so that the reinforcement learning model framework relies on the state space, which can not only capture the common characteristics of the task scenario but also adapt to the individual differences of the intelligent agents, so that the reinforcement learning model framework can be widely used and migrated in various task scenarios, laying the foundation for improving the adaptability and efficiency of the reinforcement learning model in multiple tasks and multiple environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic flow chart of a generalizable state representation method for a game-based intelligent algorithm provided in an embodiment of the present disclosure;

[0014] Figure 2 A schematic diagram of a discretization of a task scenario provided in an embodiment of the present disclosure;

[0015] Figure 3 A schematic diagram of a global common state representation provided in an embodiment of the present disclosure;

[0016] Figure 4 A schematic diagram showing a local heterogeneous state provided in an embodiment of the present disclosure;

[0017] Figure 5A schematic diagram of a general feature vector extraction process for an intelligent agent provided in an embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0020] As used in this disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0021] The terms used in the present disclosure are only used to describe specific embodiments and are not intended to limit the present disclosure.As used in the present disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0022] When the terms “comprising” and / or “made of…” are used in the present disclosure, it specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not preclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0023] Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meanings as those commonly understood by those skilled in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined in this disclosure.

[0024] The embodiments of the present disclosure provide a generalizable state representation method for game-playing intelligent algorithms. The state space is used for reinforcement learning of human intelligence modules, and is particularly suitable for determining the state space used by artificial intelligence models in game-playing scenarios.

[0025] like Figure 1 FIG2 is a flow chart of a generalizable state representation method for a game-playing intelligent algorithm provided by an embodiment of the present disclosure. The method mainly includes the following steps:

[0026] Step 101: Acquire a task scenario, where the task scenario includes at least two agents, global attribute information of each agent, and spatial distribution information of each agent.

[0027] Step 102: Divide each of the agents into N×N discrete grids according to the spatial distribution information of each of the agents, wherein one of the agents belongs to one of the discrete grids.

[0028] Specifically, each discrete grid corresponds to a spatial area of the same size. According to the spatial position of the agent and the spatial area corresponding to each discrete grid, when the spatial position of the agent falls into the spatial area, the agent is assigned to the discrete grid corresponding to the spatial area.

[0029] In the disclosed embodiment, by dividing the space where all intelligent agents in the task scene are located into N×N discrete grids, a unified and standardized representation of the environmental state is achieved, which can effectively avoid the problem of changes in the representation dimension of the state space caused by changes in the number of intelligent agents or the task environment, and provide stable input features for the artificial intelligence model, thereby improving the generalization ability and adaptability of the artificial intelligence model.

[0030] Wherein, N is an integer greater than 1, and the specific value of N can be pre-configured by technical personnel according to needs.

[0031] According to the relative position information between the agent and the discrete grid, if it is determined that the agent falls into the area where the discrete grid is located, the agent belongs to the discrete grid.

[0032] A discrete grid may include one or more intelligent agents, or may not include any intelligent agent.

[0033] Among them, the spatial distribution information of each intelligent agent can be the relative position information of each intelligent agent or the absolute position information of each intelligent agent. It only needs to be able to determine the overall area including each intelligent agent and the position of each intelligent agent in the overall area through the spatial distribution information.

[0034] In an exemplary embodiment, Figure 2 The figure shows a schematic diagram of the discretization of the mission scene, which divides the space where each drone is located in the mission scene into a 3×3 discrete grid.

[0035] Step 103: Determine the N×N feature matrix corresponding to each type of global attribute information based on the global attribute information of the intelligent agent included in each discrete grid; in the feature matrix corresponding to any of the global attribute information, the value of the element in the i-th row and j-th column is the number of intelligent agents with the global attribute information contained in the i-th row and j-th column of the discrete grid.

[0036] In some embodiments, the global attribute information of the agent includes at least one of the following: the type of the agent; the team information to which the agent belongs; the ability of the agent; the attack method of the agent; the defense method of the agent; and the mounting information of the agent.

[0037] In an exemplary embodiment, in a game confrontation scenario, the team information to which the agent belongs includes the identifier of one of the teams participating in the game confrontation. For example, the red team and the blue team are two teams in the game confrontation. The attribute information of some agents in this task scenario includes the red team identifier, and the attribute information of some agents includes the blue team identifier.

[0038] In an exemplary embodiment, at least one type of intelligent agent, for example, the type of drone includes a reconnaissance drone, an attack drone, a defense drone, and the like.

[0039] The capabilities of an intelligent agent include at least one capability. For example, the capabilities of a drone include attack capability, reconnaissance capability, defense capability, etc.

[0040] The mounting information of the intelligent body includes the status of the mounted equipment of the intelligent body. For example, the mounting information of the drone includes the mounted machine gun, camera, radar, etc.

[0041] In an exemplary embodiment, the agents are grouped according to their global attribute information, and a corresponding N×N feature matrix is constructed for each group of agents, which is expressed as The value of the matrix element in row i, column j of the feature matrix is the number of agents belonging to that group in the discrete grid in row i, column j. This approach achieves a unified and standardized representation of environmental states, effectively avoiding the problem of state representation dimensionality changes caused by changes in the number of agents or changes in the task environment, and provides a foundation for improving the generalization and adaptability of AI models.

[0042] In an embodiment of the present invention, the global common state is represented by an N×N feature matrix corresponding to all global attribute information, which can avoid the impact of changes in task scenarios (such as changes in the number of intelligent agents) on the input feature dimensions of the artificial intelligence model, support diversified task training, have a unified representation and processing of multiple situations, and can cope with input features under various situations, thereby improving the wide adaptability of the artificial intelligence model.

[0043] In the exemplary embodiment, taking the drone attack and defense simulation scenario as an example, Figure 3The figure below shows a schematic diagram of a global common state representation. Assume that a drone has three types of global attributes: type information, capability information, and payload information. Type information represents different types of drones, such as attack drones, reconnaissance drones, and other N types. Each feature matrix corresponding to the type information represents a type, and each matrix element in the feature matrix represents the number of drones of that type in the corresponding discrete grid. It should be noted that the N types here do not necessarily correspond to the dimension N of the discrete grid; this is merely an example, and the number of types is not necessarily related to the dimension N of the discrete grid. Capability information represents the drone's capabilities, such as attack capability, reconnaissance capability, and defense capability. The matrix elements in the feature matrix corresponding to each capability represent the number of drones with that capability. Payload information represents the drone's payload, such as a cannon, camera, radar, etc. Similarly, the matrix elements in the feature matrix corresponding to each payload represent the number of drones with that payload.

[0044] It should be noted that Figure 3 The element values in each feature matrix in are only examples, and the element values in the feature matrices corresponding to different global attribute information may be the same or different.

[0045] Step 104, perform the following processing on each of the intelligent agents respectively: determine the N×N observation matrix corresponding to each local attribute information of the intelligent agent based on the discrete grid where the intelligent agent is located and the observation range of the intelligent agent; in the observation matrix corresponding to any of the local attribute information, the value of the element in the i-th row and j-th column is the number of intelligent agents in the i-th row and j-th column of the discrete grid that are located within the observation range of the intelligent agent and have the local attribute information.

[0046] In some embodiments, the local attribute information of the agent includes at least one of the following: attribute information of a friendly agent of the agent; attribute information of an opposing agent of the agent; and attribute information of the agent itself.

[0047] In an exemplary embodiment, the local attribute information of an agent may be a part or all of the global attribute information.

[0048] In an exemplary embodiment, the attribute information of the intelligent agent itself includes location information and health status information. These two attribute information can accurately describe the state of the intelligent agent in a specific environment and provide support for the artificial intelligence model to formulate more personalized and effective decision-making strategies.

[0049] In an embodiment of the present invention, the observation matrices corresponding to the multiple local attribute information of the intelligent agent are used as the local heterogeneous state representation of the intelligent agent, and the original observation features of the intelligent agent (the original observed local attribute information) are converted into a multi-layer observation matrix, which can capture the local environment and heterogeneity of the intelligent agent in detail.

[0050] In the exemplary embodiment, a multi-layer observation matrix is constructed based on the discretization of the task scene. Describe the local heterogeneous state of the agent. Assume that the original observation characteristics of the agent are expressed as: ,in, Represents the attribute information of the other agent within the observation range of the agent, Represents the attribute information of friendly agents within the observation range of the agent, Represents the attribute information of the agent itself. In different adversarial tasks, and The length of often varies with task characteristics (e.g., the number of agents), but will not change, therefore, the heterogeneous characteristics of the agent are reflected in In. Convert the original observation features into a multi-layer observation matrix.

[0051] Specifically, assuming Represents the set of index numbers of all discrete grids, Represents a collection of agents in a discrete grid. If an agent On a discrete grid Inside, there is .use represents the attribute set of each agent, Represents an agent Attributes For any selected attribute Construct the corresponding observation matrix , the value of each matrix element in the observation matrix is determined by the formula: For example, let In this way, the observation matrix of position features and the observation matrix of health status features are constructed.

[0052] In the exemplary embodiment, taking the drone attack and defense simulation scenario as an example, Figure 4 The figure shows a schematic diagram of the local heterogeneous state representation. Assume that the mission scenario includes three red drones and three blue drones. Assume that the space where these drones are located is divided into 9 discrete grids, and the visible range of each drone is only the adjacent discrete grids. It only includes two attributes: location and health status. Figure 4The numbers 1-9 represent the numbers of the discrete grids. The red drones are located in the discrete grids 2 and 3, and the blue drones are located in the discrete grids 5, 6, and 8. The red drone located in the discrete grid 5 is drone 1. Each element in the position vector represents the number of the location of the agent observed by drone 1. The blue drone 4 located in the discrete grid 3 is not within the observation range of drone 1. The position vector of the friendly agent observed by drone 1 is , the observed position vector of the other agent is , the position vector of UAV No. 1 itself is Similarly, each element of the health state vector represents the life value of the agent observed by UAV No. 1. Figure 4 In this example, the health value of each drone is 2. The position vector and health state vector are converted into corresponding observation matrices, namely the position feature matrix and the health state feature matrix.

[0053] Step 105: Determine a first eigenvector based on each of the feature matrices, and determine a second eigenvector based on each of the observation matrices, and obtain a state space corresponding to the task scenario based on the fusion of the first eigenvector and the second eigenvector. The state space is used for reinforcement learning of the artificial intelligence model.

[0054] In some embodiments, determining the first eigenvector based on each of the eigenmatrices comprises:

[0055] Each of the feature matrices is input into a first convolution filter group, and the first feature vector is determined according to an output vector of the first convolution filter group.

[0056] In some embodiments, the first convolution filter group includes P convolution filters, where P is equal to the number of global attribute information of the agent, and N is an integer greater than or equal to 1;

[0057] The step of inputting each of the feature matrices into the first convolution filter group and determining the first feature vector according to the output vector of the first convolution filter group includes: for each of the feature matrices, selecting a target convolution filter from P convolution filters of the first convolution filter group according to the global attribute information corresponding to the feature matrix, inputting the feature matrix into the target convolution filter, and obtaining a sub-feature vector corresponding to the feature matrix; and summing the sub-feature vectors corresponding to each of the feature matrices to obtain the first feature vector.

[0058] In some embodiments, determining the second eigenvector based on each of the observation matrices comprises:

[0059] Each of the observation matrices is input into a second convolution filter group, and the second eigenvector is determined according to an output vector of the second convolution filter group.

[0060] In some embodiments, the second convolution filter group includes Q convolution filters, where Q is equal to the number of local attribute information of the agent, and Q is an integer greater than or equal to 1;

[0061] The step of inputting each observation matrix into the second convolution filter group and determining the second eigenvector according to the output vector of the second convolution filter group includes: for each observation matrix, selecting a target convolution filter from the Q convolution filters of the second convolution filter group according to the local attribute information corresponding to the observation matrix, inputting the observation matrix into the target convolution filter, and obtaining a sub-eigenvector corresponding to the observation matrix; and summing the sub-eigenvectors corresponding to each observation matrix to obtain the second eigenvector.

[0062] In the embodiment of the present disclosure, a dual-channel convolution is formed by a first convolution filter group and a second convolution filter group, the feature matrix and the observation matrix are converted into corresponding feature vectors, and the converted feature vectors are fused to obtain a general state space of the intelligent agent.

[0063] Specifically, combined Figure 5 The general feature vector extraction process of the intelligent agent is shown in the figure, and the dual-channel convolution process is explained as follows:

[0064] The two-channel convolution is composed of the first convolution filter group and the second convolution filter group; the first convolution filter group is expressed as , the second convolution filter bank is expressed as .

[0065] The first convolution filter bank consists of convolution filters Composition, that is ,in is the number of global attribute information, and the calculation formula of the convolution filter is expressed as: , responsible for transforming the feature matrix Convert to the first eigenvector .

[0066] The second convolution filter bank consists of convolution filters, namely , the calculation formula of the convolution filter is expressed as: , responsible for transforming the agent's observation matrix Convert to the second eigenvector .

[0067] Finally, the first eigenvector and the second eigenvector are connected to obtain the fused general eigenvector of the agent, that is, the state space ,Right now .

[0068] It should be noted that Figure 5 The element values in the observation matrix corresponding to the local heterogeneous state and the feature matrix corresponding to the global common state are only examples and do not mean that the two matrices must be the same. The corresponding element values in the observation matrix and the feature matrix may be the same or different, which is determined according to the actual statistical situation.

[0069] In the embodiments of the present disclosure, by constructing a universal feature vector that integrates global common state representation and local heterogeneous state representation as a state space, the artificial intelligence model can capture the commonalities between tasks and adapt to individual differences and environmental changes when performing reinforcement learning. It can be effectively migrated and applied in different tasks and environments to promote the development of artificial intelligence models in a wider and more diverse application field, overcome the limitations of state space generalization when migrating different task scenarios in existing related technologies, and provide a new solution for reinforcement learning in complex environments.

[0070] The method provided by the embodiments of the present disclosure can not only adapt to changes in the number of intelligent agents, but also effectively deal with the generalization problem of the state space caused by changes in different task types and intelligent agent types, thereby realizing efficient formulation and decision-making of the state space of complex environments, and significantly improving the adaptability and efficiency of artificial intelligence models in multiple tasks and multiple environments.

[0071] The steps of the various methods above are divided only for clarity of description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this disclosure. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this disclosure.

[0072] The present disclosure provides a generalizable state representation device for game-playing intelligent algorithms. The specific implementation of the device can be found in the relevant description of the method embodiment and will not be repeated here. The device mainly includes:

[0073] An acquisition module is used to acquire a task scenario, wherein the task scenario includes at least two agents, and includes global attribute information of each of the agents and spatial distribution information of each of the agents;

[0074] a discretization module, configured to divide each of the intelligent agents into an N×N discrete grid according to spatial distribution information of each of the intelligent agents, wherein one of the intelligent agents belongs to one of the discrete grids;

[0075] a global processing module configured to determine, based on the global attribute information of the agents included in each discrete grid, an N×N feature matrix corresponding to each type of global attribute information; wherein the value of the element in the i-th row and j-th column of the feature matrix corresponding to any of the global attribute information is the number of agents with the global attribute information included in the i-th row and j-th column of the discrete grid;

[0076] A local processing module is configured to perform the following processing on each of the agents: determining an N×N observation matrix corresponding to each local attribute information of the agent based on the discrete grid in which the agent is located and the observation range of the agent; wherein the value of the element in the i-th row and j-th column of the observation matrix corresponding to any local attribute information is the number of agents in the i-th row and j-th column of the discrete grid that are within the observation range of the agent and have the local attribute information;

[0077] A fusion module is used to determine a first eigenvector based on each of the feature matrices and a second eigenvector based on each of the observation matrices, and to obtain a state space corresponding to the task scenario based on the fusion of the first eigenvector and the second eigenvector, wherein the state space is used for the reinforcement learning process of the intelligent agent.

[0078] The functions or modules included in the apparatus provided in the embodiments of the present disclosure can be used to execute the method described in the method embodiments. The specific implementation and technical effects thereof can be referred to the description of the above method embodiments, and will not be repeated here for the sake of brevity.

[0079] It should be noted that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovations of this disclosure, this embodiment does not include units that are not closely related to solving the technical problems proposed by this disclosure. However, this does not mean that other units do not exist in this embodiment.

[0080] Reference Figure 6 , an embodiment of the present disclosure provides an electronic device, comprising:

[0081] at least one processor 1001;

[0082] a memory 1002 storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the above method;

[0083] At least one I / O interface 1003 is connected between the processor and the memory and is configured to implement information exchange between the processor and the memory.

[0084] Among them, the processor 1001 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 1002 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) 1003 is connected between the processor 1001 and the memory 1002, and can realize information interaction between the processor 1001 and the memory 1002, including but not limited to a data bus (Bus), etc.

[0085] In some embodiments, the processor 1001 , the memory 1002 , and the I / O interface 1003 are connected to each other via a bus, and further connected to other components of the computing device.

[0086] This embodiment further provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the method provided in this embodiment is implemented. To avoid repeated description, the specific steps of the method are not repeated here.

[0087] Those skilled in the art will appreciate that all or some of the steps, systems, and functional modules / units in the methods, systems, and devices described above may be implemented as software, firmware, hardware, or any combination thereof. In hardware implementations, the division between functional modules / units described above does not necessarily correspond to the division between physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is well known to those skilled in the art, the term computer storage media encompasses volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0088] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0089] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present embodiment and to form different embodiments.

[0090] It is understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present disclosure, and the present disclosure is not limited thereto. Those skilled in the art may make various modifications and improvements without departing from the spirit and substance of the present disclosure, and such modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A generalizable state representation method for game-based intelligent algorithms, characterized by: include: Acquire a task scenario, wherein the task scenario includes at least two agents, and includes global attribute information of each of the agents and spatial distribution information of each of the agents; The intelligent agent is a drone; Dividing each of the intelligent agents into N×N discrete grids according to the spatial distribution information of each of the intelligent agents, wherein one of the intelligent agents belongs to one of the discrete grids; Based on the global attribute information of the agents included in each discrete grid, an N×N feature matrix corresponding to each type of the global attribute information is determined; in the feature matrix corresponding to any of the global attribute information, the value of the element in the i-th row and j-th column is the number of agents with the global attribute information included in the discrete grid in the i-th row and j-th column; the global attribute information includes at least one of the following: the type of the agent; information about the team to which the agent belongs; the ability of the agent; the attack mode of the agent; the defense mode of the agent; and the mount information of the agent. The following processing is performed on each of the agents: an N×N observation matrix corresponding to each local attribute information of the agent is determined based on the discrete grid in which the agent is located and the observation range of the agent; the value of the element in the i-th row and j-th column of the observation matrix corresponding to any local attribute information is the number of agents in the i-th row and j-th column of the discrete grid that are located within the observation range of the agent and have the local attribute information; the local attribute information includes at least one of the following: attribute information of a friendly agent of the agent; attribute information of an opponent agent of the agent; and attribute information of the agent itself; Each of the feature matrices is input into the first convolution filter group, and a first eigenvector is determined based on the output vector of the first convolution filter group. Each of the observation matrices is input into the second convolution filter group, and a second eigenvector is determined based on the output vector of the second convolution filter group. The state space corresponding to the task scenario is obtained based on the fusion of the first eigenvector and the second eigenvector, and the state space is used for reinforcement learning of the artificial intelligence model.

2. The method according to claim 1, characterized in that The first convolution filter group includes P convolution filters, where P is equal to the number of global attribute information of the agent, and P is an integer greater than or equal to 1; Inputting each of the feature matrices into a first convolution filter group, and determining the first feature vector according to an output vector of the first convolution filter group, comprises: For each of the feature matrices, selecting a target convolution filter from the P convolution filters of the first convolution filter group according to the global attribute information corresponding to the feature matrix, inputting the feature matrix into the target convolution filter, and obtaining a sub-feature vector corresponding to the feature matrix; The sub-eigenvectors corresponding to each of the characteristic matrices are summed to obtain the first eigenvector.

3. The method according to claim 1, characterized in that The second convolution filter group includes Q convolution filters, where Q is equal to the number of local attribute information of the agent, and Q is an integer greater than or equal to 1; Inputting each of the observation matrices into a second convolution filter group, and determining the second eigenvector according to an output vector of the second convolution filter group, comprises: For each observation matrix, selecting a target convolution filter from the Q convolution filters of the second convolution filter group according to the local attribute information corresponding to the observation matrix, inputting the observation matrix into the target convolution filter, and obtaining a sub-eigenvector corresponding to the observation matrix; The second eigenvector is obtained by summing the sub-eigenvectors corresponding to each of the observation matrices.

Images