Emission recognition method and device, computer equipment, storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and discloses an emergence recognition method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring task information of a multi-agent system; wherein the task information comprises a feature vector of each node and a feature vector of an edge where each node is located; based on the target node, determining a first message of an adjacent node of the target node and a second message of an edge where the target node is located; wherein the target node is any one of the nodes; performing aggregation processing on the first message and the second message, and determining a target feature vector of the target node; inputting the target feature vector of the target node into a pre-trained emergence recognition model to obtain a reconstruction error value; wherein if the reconstruction error value exceeds a preset reconstruction error value, occurrence of emergence identification is determined.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an emergence recognition method, device, computer equipment, storage medium and program product. Background Art

[0002] Multi-agent systems consist of multiple autonomous or semi-autonomous agents that can complete complex tasks through interaction and collaboration. Emergence is particularly prominent in such multi-agent systems, because the overall behavior of the multi-agent system often cannot be simply inferred from the behavior of a single agent.

[0003] Currently, in multi-agent systems, each agent can be equipped with sensors to monitor the surrounding environment and the status of other agents. By analyzing and comparing the changes in sensor data from different agents, emergent behaviors can be identified.

[0004] However, the type and precision of sensors may affect the accuracy and reliability of data, and thus affect the recognition effect of emergent behavior. Therefore, how to accurately identify emergent behavior becomes a problem that needs to be solved. Summary of the invention

[0005] In view of this, the present invention provides an emergence identification method, apparatus, computer device, storage medium and program product.

[0006] In the first aspect, the present invention provides an emergence identification method, which includes: obtaining task information of a multi-agent system; wherein the task information includes: a feature vector of each node and a feature vector of an edge where each node is located; based on a target node, determining a first message of an adjacent node of the target node and a second message of an edge where the target node is located; wherein the target node is any one of the nodes; aggregating the first message and the second message to determine a target feature vector of the target node; inputting the target feature vector of the target node into a pre-trained emergence identification model to obtain a reconstruction error value; wherein, if the reconstruction error value exceeds a preset reconstruction error value, it is determined that emergence identification has occurred.

[0007] The method for emerging identification provided in this embodiment obtains the feature vectors of each node and edge, and determines the messages of the adjacent nodes and edges based on the target node. This provides a specific and accurate data basis for the identification of emergent behavior, and determines the target feature vector of the target node by aggregating the first message and the second message, which helps to overcome the influence of sensor type and accuracy on the identification of emergent behavior. And by inputting the target feature vector of the target node into a pre-trained emergence identification model and calculating the reconstruction error value to evaluate the occurrence of emergent behavior, a means of quantitatively evaluating emergent behavior is provided, which improves the accuracy and reliability of identification.

[0008] In one possible implementation, the first message and the second message are aggregated to determine a target feature vector of a target node, including: using a self-attention mechanism to determine a weight of an edge type corresponding to the target message; wherein the target message includes the first message and the second message; performing weighted averaging on the target messages according to the weight of the edge type corresponding to the target message to determine a feature vector of an edge where the target node is located; and fusing the feature vectors of each node on the edge where the target node is located and the feature vector of the edge where the target node is located to determine the target feature vector of the target node.

[0009] In the emergence recognition method provided in this embodiment, the self-attention mechanism allows the model to directly refer to the information of all other nodes or edges in the graph when calculating the representation of a certain node, so as to capture long-distance dependencies. In addition, by fusing the feature vectors of each node on the edge where the target node is located and the target feature vector, it is possible to fully utilize multi-source information and improve the representation ability of the target node feature vector.

[0010] In a possible implementation, the method also includes: inputting the target feature vector of the target node into a pre-trained emergence recognition model to determine the first feature vector of the target node and the second feature vector of the edge where the target node is located; obtaining the third feature vector of the target node at the target time and the fourth feature vector of the edge where the target node is located at the target time; determining the reconstruction error score based on the first feature vector, the second feature vector, the third feature vector and the fourth feature vector; determining the type of emergence based on the reconstruction error score; wherein the types of emergence include: forward emergence and reverse emergence.

[0011] The emergence identification method provided in this embodiment can more comprehensively reflect the state and behavior of the target node and its edge by comprehensively considering multiple feature vectors of the target node (including the first feature vector, the second feature vector, the third feature vector and the fourth feature vector), improve the accuracy of emergence identification, and reduce the possibility of misjudgment and missed judgment. In addition, the type of emergence (such as positive emergence, reverse emergence, etc.) can be accurately determined according to the reconstruction error score.

[0012] In one possible implementation, the type of emergence is determined based on the reconstruction error score, including: determining the environmental reward at the target moment; fusing the environmental reward at the target moment and the reconstruction error score to determine the quantitative score of the emergence behavior; when the quantitative score of the emergence behavior is greater than a preset threshold, the type of emergence is determined to be positive emergence; when the quantitative score of the emergence behavior is not greater than the preset threshold, the type of emergence is determined to be reverse emergence.

[0013] The emergence identification method provided in this embodiment not only considers the reconstruction error score, which reflects the change of the internal state of the system, but also introduces the external factor of environmental reward. Environmental reward usually reflects the performance or benefit of the agent or system in a specific environment, so the integration of the two can more comprehensively evaluate the nature and impact of emergent behavior.

[0014] In a possible implementation, the method also includes: obtaining a preset stage-by-stage reward function; when the type of emergence is positive emergence, incorporating the quantitative score of the emergence behavior into the preset stage-by-stage reward function to achieve repeated positive emergence; when the type of emergence is reverse emergence, incorporating the quantitative score of the emergence behavior into the preset stage-by-stage reward function to achieve suppression of reverse emergence.

[0015] The emergence identification method provided in this embodiment, when the type of emergence is positive emergence, incorporates the quantitative score of the emergence behavior into the stage reward function, which can strengthen this positive behavior. By increasing the reward of positive emergence, the system or intelligent body will be more inclined to repeat this beneficial emergence behavior, thereby promoting the positive development and optimization of the system. In addition, when the type of emergence is reverse emergence, the quantitative score of the emergence behavior is incorporated into the stage reward function and set as a negative reward, which can suppress this reverse behavior. By reducing the reward of reverse emergence, the system or intelligent body will try to avoid executing this harmful emergence behavior, thereby reducing system risk.

[0016] In one possible implementation, a pre-trained emergence recognition model is constructed, including: obtaining a historical target feature vector; obtaining a low-dimensional feature from the historical target feature through an encoder network, and then decoding it through a decoder network to obtain a reconstructed feature vector; determining a loss value corresponding to a reconstruction error based on the reconstructed feature vector; minimizing the loss value corresponding to the reconstruction error through an optimization algorithm, and determining the gradient of the reconstruction error with respect to the network parameters; updating the parameters of the encoder network and the decoder network based on the gradient; repeatedly executing the steps of obtaining a low-dimensional feature from the historical target feature through an encoder network, and then decoding it through a decoder network to obtain a reconstructed feature vector until the number of convergence or repetitions reaches a preset number, thereby obtaining a pre-trained emergence recognition model.

[0017] In the emergence recognition method provided in this embodiment, the encoder network can compress high-dimensional historical target features into low-dimensional feature representations. This process removes redundant information, retains key features, and provides an efficient data basis for subsequent classification or recognition tasks. At the same time, the low-dimensional features are decoded by the decoder network to obtain a reconstructed feature vector, which is compared with the original feature vector to calculate the reconstruction error. This process can intuitively reflect the model's ability to restore the original data.

[0018] Furthermore, by minimizing the reconstruction error through the optimization algorithm, the parameters of the encoder and decoder can be continuously adjusted to improve the accuracy and robustness of the model. The encoding, decoding, and parameter update steps are repeated until the model converges or the preset number of iterations is reached. Since each iteration is optimized based on the previous result, the model can quickly converge to the optimal solution.

[0019] In the second aspect, the present invention provides an emergence identification device, which includes: an acquisition module for acquiring task information of a multi-agent system; wherein the task information includes: a feature vector of each node and a feature vector of an edge where each node is located; a first determination module for determining, based on a target node, a first message of an adjacent node of the target node and a second message of an edge where the target node is located; wherein the target node is any one of the nodes; a second determination module for aggregating the first message and the second message to determine a target feature vector of the target node; an identification module for inputting the target feature vector of the target node into a pre-trained emergence identification model to obtain a reconstruction error value; wherein, if the reconstruction error value exceeds a preset reconstruction error value, it is determined that emergence identification has occurred.

[0020] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the emergence identification method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the emergence identification method of the first aspect or any corresponding embodiment thereof.

[0022] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the emergence identification method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a schematic diagram of an emergence identification system provided according to an embodiment of the present invention;

[0025] Figure 2 is a flow chart of an emergence identification method provided according to an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of an emergence identification method provided according to an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of the architecture of an emergence identification method provided according to an embodiment of the present invention;

[0028] Figure 5 is a schematic diagram of another emergence identification method according to an embodiment of the present invention;

[0029] Figure 6 is a structural block diagram of an emergence identification device according to an embodiment of the present invention;

[0030] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0032] Please refer to Figure 1 , Figure 1 is a schematic diagram of an emergence identification system provided according to an embodiment of the present invention.

[0033] The emergence identification system includes: component relationship representation module, emergence identification module, and emergence traction module. Among them, the component relationship representation module can construct the component relationship of the multi-intelligent component system based on the graph neural network, which is used to represent the graph structure, the mission tasks, component capabilities and organizational relationships between components of the multi-intelligent component system, so as to form the overall situation characteristics. At the same time, a dynamic side information update mechanism is introduced to adjust the connection relationship between components according to environmental changes or human changes.

[0034] The component relationship representation module is used to represent the relationship between components based on the graph neural network and generate overall situation characteristics to support intelligent emergence recognition and analysis. The interactive relationship between components is constructed based on the graph structure, taking into account the different types of components of the red and blue sides and the diverse connection methods between them, effectively establishing the behavior patterns of various components, accurately capturing the information flow and interaction within the red / blue system and between the red and blue sides, and laying a solid foundation for further situation analysis and recognition of emergent behavior.

[0035] The emergence recognition module includes an encoder and a decoder. The encoder is connected to the component relationship representation module to encode the node feature vector into a low-dimensional representation and the edge feature vector into a low-dimensional representation. The node feature vector generated by the encoder analyzes the complex structure between the multi-intelligent component system and recognizes the emergence phenomenon through reconstruction error calculation. The decoder decodes the encoded low-dimensional representation into a reconstructed feature vector of the original feature space.

[0036] The emergence traction module is designed with dense phased rewards, which integrates the emergence of intelligence into the phased calculation of the reward function, and realizes the active traction of the emergence of the positive system or the suppression of the emergence of the reverse system through the feedback mechanism.

[0037] The emergence traction module is used to complete the identification of emergence. In order to integrate the identified emergence phenomena into the staged calculation of the reward function, two reward functions are designed:

[0038] Final reward function (total reward): used to evaluate the effect of the entire game process.

[0039] Phased reward function (instant reward): Instant rewards are calculated at each step in the process, which are used to refine the behavior evaluation during the game.

[0040] In order to effectively drive the emergence of the system, the identified emergence phenomenon is quantified into a score, and the calculated quantitative score of the emergent behavior is incorporated into the staged reward function to drive the components to repeat positive emergent behavior or inhibit reverse emergent behavior during the training process. The complex structure between the multi-intelligent component system is analyzed, and the intelligent emergence is integrated into the staged calculation of the reward function, and the active driving of positive system emergence or the inhibition of reverse system emergence is achieved through the feedback mechanism.

[0041] According to an embodiment of the present invention, an embodiment of an emergence identification method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] In this embodiment, an emergence identification method is provided, which can be used in the above-mentioned emergence identification system. Figure 2 is a flow chart of an emergence identification method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0043] Step S201, obtaining task information of the multi-agent system; wherein the task information includes: a feature vector of each node and a feature vector of an edge where each node is located.

[0044] This embodiment can use a heterogeneous graph neural network model (Heterogeneous Graph Neural Network, HGNN) to perform feature representation learning on the constructed heterogeneous graph. First, a heterogeneous graph convolution layer is constructed. In the message passing mechanism, a corresponding message passing method is designed according to different types of edges, and the updated feature vector of the target node is calculated by aggregating the feature vectors of adjacent nodes. For the target node, the feature vectors of the adjacent nodes of the target node transmitted through different types of edges corresponding to the target node are weighted averaged. Then, an activation function (such as ReLU) is used to perform a nonlinear transformation on the aggregated feature vector. Through multi-layer convolution operations, the feature vectors of more adjacent nodes are gradually fused, and the final updated feature vector is finally obtained.

[0045] The task information of the multi-agent system is used to indicate the characteristics between multi-intelligent components based on graph neural networks. Among them, the task information may include the feature vectors of each node and the feature vectors of the edges where each node is located. Among them, the feature vectors of the nodes include the state attributes of the component (position, movement speed, health value, etc.), component functional attributes (component type, component ability, etc.), and component historical behavior data. The feature vectors of the edges where each node is located represent the interaction relationship between components, which may specifically include the interaction frequency (number of interactions between components), interaction intensity (degree of intensity of confrontation), and interaction duration (duration of interaction).

[0046] As an example, the task information of a multi-agent system can be a game confrontation task; game confrontation refers to the generation and optimization of confrontation strategies to deal with possible risks when the system faces potential threats or confrontational environments. Using intelligent methods for simulation and strategy adjustment, game confrontation can help the system maintain resilience in confrontational situations and effectively counter threats.

[0047] As an example, the task information of a multi-agent system may be a situational awareness task; wherein situational awareness refers to real-time insight and deep understanding of the status between subsystems and between subsystems and the external environment.

[0048] As an example, the task information of a multi-agent system can be a collaborative reconstruction task; collaborative reconstruction refers to the flexible adjustment of resource allocation through collaborative operations between subsystems. In the face of sudden demands or environmental changes, collaborative reconstruction can quickly reallocate resources and optimize the collaboration between modules to ensure the stability and efficiency of the system in a dynamic environment.

[0049] It should be noted that the above example is only an example of this embodiment, and the task information of the multi-agent system may also include other tasks, which are not specifically limited here and can be implemented by technical personnel in this field.

[0050] Step S202, based on the target node, determine the first message of the neighboring node of the target node and the second message of the edge where the target node is located; wherein the target node is any one of the nodes.

[0051] The target node can be any one of the nodes. For each target node, a first message is generated from adjacent nodes. The second message of the edge where the target node is located can be determined by the target node and other nodes connected to the edge where the target node is located.

[0052] As an example, for each target node v, a first message m is generated from each neighboring node u u→v The specific formula can be as follows:

[0053] in, is the edge type φ(e u v) the corresponding linear transformation matrix; h u is the feature vector of neighbor node u, m u→v For the first message.

[0054] As an example, for each edge e uv , generate its feature vector h euv The specific formula can be as follows:

[0055] Among them, h u and h v It is edge uv The feature vector of the connected node u and the target node v; It is edge uv The initial eigenvector of φ is a learnable message generation function, For the second message.

[0056] Step S203: aggregate the first message and the second message to determine a target feature vector of the target node.

[0057] For the first message and the second message, firstly, a message is generated according to the relationship between the target node and the adjacent node (such as the type and weight of the edge) and the characteristics of the target node. The generated first message and the second message are aggregated to obtain a target feature vector. There are many ways of aggregation, such as summation, average, maximum value, attention weighting, etc., which are not specifically limited here and can be implemented by those skilled in the art.

[0058] Step S204, inputting the target feature vector of the target node into a pre-trained emergence recognition model to obtain a reconstruction error value; wherein, if the reconstruction error value exceeds a preset reconstruction error value, it is determined that emergence recognition occurs.

[0059] The target feature vector extracted from the target node is passed as input to a pre-trained emergence recognition model. The pre-trained emergence recognition model can be a neural network-based architecture, such as an autoencoder or a variational autoencoder (VAE), which can reconstruct similar outputs from the input feature vector.

[0060] Specifically, the pre-trained emergence recognition model will try to reconstruct a feature vector (i.e., output) from the input target feature vector. Then, the difference between the reconstructed feature vector and the original input feature vector is calculated. This difference can be measured by a loss function (such as mean square error MSE, cross entropy loss, etc.) to obtain a reconstruction error value. The calculated reconstruction error value is compared with a preset reconstruction error threshold. The reconstruction error threshold can be set based on the training data of the pre-trained emergence recognition model and the prior knowledge of the emergence phenomenon, and is used to distinguish between normal situations and emergence phenomena. If the reconstruction error value exceeds the preset reconstruction error threshold, it is considered that emergence recognition has occurred.

[0061] The method for emerging identification provided in this embodiment obtains the feature vectors of each node and edge, and determines the messages of the adjacent nodes and edges based on the target node. This provides a specific and accurate data basis for the identification of emergent behavior, and determines the target feature vector of the target node by aggregating the first message and the second message, which helps to overcome the influence of sensor type and accuracy on the identification of emergent behavior. And by inputting the target feature vector of the target node into a pre-trained emergence identification model and calculating the reconstruction error value to evaluate the occurrence of emergent behavior, a means of quantitatively evaluating emergent behavior is provided, which improves the accuracy and reliability of identification.

[0062] In a possible implementation, the above step S103 includes:

[0063] Step a1, using a self-attention mechanism to determine the weight of the edge type corresponding to the target message; wherein the target message includes a first message and a second message.

[0064] The self-attention mechanism is used to characterize a mechanism that can automatically learn the correlation between different parts of the data when processing sequence data or graph data. It calculates the attention scores (or weights) between different parts.

[0065] As an example, a multi-head self-attention mechanism can be used, where each head focuses on a different subset of features. The type of the edge can also be encoded as an embedding vector and fed into the attention mechanism along with the message features. Positional encoding or graph convolutional layers (GCN) can also be used to enhance the representation of edge type information.

[0066] Step a2: perform weighted averaging on the target message according to the weight of the edge type corresponding to the target message, and determine the feature vector of the edge where the target node is located.

[0067] The target message refers to information transmitted from the neighboring nodes of the target node, and the target information may be node features, edge features, or other related information. The target message may include a first message and a second message.

[0068] As an example, one can directly multiply the weights by the target message and then sum them to get the target feature vector, or one can use normalized weights to ensure that the sum of the contributions of all messages is 1, etc.

[0069] Step a3, merging the feature vectors of each node on the edge where the target node is located and the feature vector of the edge where the target node is located, to determine the target feature vector of the target node.

[0070] The feature vectors of each node on the edge where the target node is located can be directly concatenated with the target feature vector to form a longer vector, and the feature vectors of each node can also be added element by element to the target feature vector. Among them, the dimensions of all vectors are the same, etc., which are not specifically limited here and can be implemented by those skilled in the art.

[0071] As an example, the generated first message and the second message are transmitted to the target node v through the corresponding edge type, and the messages of each edge type are calculated separately. For the target node v, messages of different edge types can use different weights.

[0072] in, is the edge type φ(e uv ) corresponding to the weight, φ(e uv ) is the edge type, which can be calculated by the self-attention mechanism, m v The weights of the edge types corresponding to different nodes can be represented.

[0073] Aggregate the received target messages at the target node. Aggregate all received target messages at the target node v and perform weighted average:

[0074] Among them, σ is a nonlinear activation function. ReLU is used in this model, and h′ vIt is the feature vector of the edge where the target node is located after weighted average of the target node.

[0075] It should be noted that by stacking multiple layers of heterogeneous graph convolutional layers, the model gradually integrates more neighbor information and continuously updates the feature representation of nodes and edges.

[0076] Optionally, to adaptively adjust the weights of different edge types, an attention mechanism can be set:

[0077] Among them, a is a learnable attention vector; || represents the vector concatenation operation, e u′v For edge type φ(e uv ) is the edge eigenvector after weighted averaging.

[0078] For each edge e uv , merge the feature vector of the target node with the feature vector of the edge where the target node is located to generate the target feature vector of the target node:

[0079] Among them, W e is a learnable weight matrix; h u and h v It is edge uv The feature vectors of the connected nodes u and v; It is edge uv The initial features of ; || represents the connection operation of the vector; σ is a nonlinear activation function, and ReLU can be used in this model.

[0080] As an example, there is a social network graph where nodes represent users and edges represent relationships between users (such as friends, colleagues, etc.). The goal is to calculate a target feature vector for each user (target node) that reflects their social environment and relationship type. For each target node, calculate the weight of the edge type corresponding to the messages sent by its neighbor nodes (such as the user's interests, activities, etc.). This may involve encoding the type of edge (such as friends, colleagues) as an embedding vector and using a self-attention mechanism to calculate the weight. Using the calculated weights, the messages of the neighbor nodes are weighted averaged to obtain the feature vector of the edge where the target node is located. Finally, the feature vector of the target node (such as the user's profile, historical activities, etc.) is fused with the target feature vector to obtain the final target feature vector of the target node.

[0081] In the emergence recognition method provided in this embodiment, the self-attention mechanism allows the model to directly refer to the information of all other nodes or edges in the graph when calculating the representation of a certain node, so as to capture long-distance dependencies. In addition, by fusing the feature vectors of each node on the edge where the target node is located and the target feature vector, it is possible to fully utilize multi-source information and improve the representation ability of the target node feature vector.

[0082] In a possible implementation, the method further includes:

[0083] Step b1, inputting the target feature vector of the target node into a pre-trained emergence recognition model to determine the first feature vector of the target node and the second feature vector of the edge where the target node is located.

[0084] The pre-trained emergence recognition model can be a graph neural network (GNN) or a deep learning model containing a graph convolutional layer (GCN), which can extract features from graph structured data. After the target feature vector is input, the model processes the input through its internal mechanisms (such as convolution, pooling, fully connected layers, etc.) and outputs two feature vectors. The first feature vector reflects the position of the target node in the graph, neighbor information, and its own properties. The second feature vector may represent information such as the type, weight, or relationship of the edge where the target node is located with other edges.

[0085] Step b2, obtaining the third eigenvector of the target node at the target time and the fourth eigenvector of the edge where the target node is located at the target time.

[0086] The third feature vector may contain the real-time status, activity or interaction information with other nodes of the target node at the target time. The fourth feature vector may represent the dynamic changes of the edge where the target node is located at the target time, such as the weight change of the edge, the appearance of a new edge or the disappearance of an edge.

[0087] Step b3: determining a reconstruction error score according to the first eigenvector, the second eigenvector, the third eigenvector and the fourth eigenvector.

[0088] The reconstruction error score can be calculated by comparing the difference between the first and second eigenvectors (model output) and the third and fourth eigenvectors (actual data). The difference can be measured by some form of distance metric (such as Euclidean distance, cosine similarity, or a custom loss function).

[0089] As an example, since the autoencoder learns the data distribution under normal conditions, it usually has a higher reconstruction error for input data that does not conform to the normal pattern. Therefore, for each time step t, the reconstruction error score of the nodes and edges can be calculated:

[0090] in, and is the feature vector of the node and edge at the target time t (the feature vector of the target node at the target time is the first feature vector, and the feature vector of the edge where the target node is located at the target time is the second feature vector), and They are the feature vectors of the node and the edge reconstructed by the decoder respectively (the feature vector of the target node reconstructed by the decoder is the third feature vector and the feature vector of the edge where the target node is located reconstructed by the decoder is the fourth feature vector).

[0091] Step b4, determining the type of emergence according to the reconstruction error score; wherein the types of emergence include: positive emergence and reverse emergence.

[0092] Specifically, the above step b4 includes:

[0093] Step b41, determine the environmental reward at the target moment.

[0094] The environmental reward can be based on a variety of factors, such as the performance of the target node, the activity of the edge where the target node is located, the interaction effect with other nodes, etc. The environmental reward value can be positive (indicating a positive effect) or negative (indicating a negative effect), or it can be zero (indicating no significant effect).

[0095] Step b42, the environmental reward and reconstruction error score at the target moment are integrated to determine the quantitative score of the emergent behavior.

[0096] Specifically, the two scores may be fused using a weighted sum, product, or other mathematical operation.

[0097] Step b43: when the emergence behavior quantification score is greater than a preset threshold, the emergence type is determined to be positive emergence.

[0098] Step b44: when the emergence behavior quantification score is not greater than the preset threshold, the emergence type is determined to be reverse emergence.

[0099] If the emergent behavior quantification score is higher than the threshold, it indicates that the performance of the target node and its edges at the target time exceeds expectations and has a positive impact or effect. If the emergent behavior quantification score is lower than or equal to the threshold, it indicates that the performance of the target node and its edges at the target time does not meet expectations and may have a negative impact or effect.

[0100] As an example, for the behavior of a component in an environment, the impact of emergent behavior can be quantified in terms of the rewards obtained in a reinforcement learning task:

[0101] E reward (t) = r t ; Among them, r t is the environmental reward obtained by the component at time t.

[0102] Fusion reconstruction error score and reward score, calculate the final emergent behavior quantitative score E t :

[0103] E t=β·E recon (t)+(1-β)·E reward (t); where β is a hyperparameter used to adjust the weights of the reconstruction error score and the reward score. By adjusting β, the importance of the reconstruction error detected by the autoencoder and the reinforcement learning reward feedback can be balanced according to the specific application scenario.

[0104] The emergence identification method provided in this embodiment can more comprehensively reflect the state and behavior of the target node and its edge by comprehensively considering multiple feature vectors of the target node (including the first feature vector, the second feature vector, the third feature vector and the fourth feature vector), improve the accuracy of emergence identification, and reduce the possibility of misjudgment and missed judgment. In addition, the type of emergence (such as positive emergence, reverse emergence, etc.) can be accurately determined according to the reconstruction error score.

[0105] In addition, this embodiment not only considers the reconstruction error score, which reflects the change of the internal state of the system, but also introduces the external factor of environmental reward. Environmental rewards usually reflect the performance or benefits of the agent or system in a specific environment, so the fusion of the two can more comprehensively evaluate the nature and impact of emergent behavior.

[0106] In a possible implementation, the method further includes:

[0107] Step c1, obtaining a preset phased reward function.

[0108] The phased reward function can be a mathematical formula or algorithm that accepts a series of inputs (such as emergent behavior quantitative scores, timestamps, other system states, etc.) and outputs a numerical value as a reward.

[0109] Step c2, when the type of emergence is positive emergence, the quantitative score of the emergence behavior is incorporated into the preset stage reward function to achieve repeated positive emergence.

[0110] The quantitative score of positive emergence can be used as a positive factor in the stage return function, increasing its weight or coefficient so that when similar positive emergence occurs again, the system can obtain higher returns.

[0111] Step c3, when the type of emergence is reverse emergence, the quantitative score of the emergence behavior is incorporated into the preset stage-by-stage reward function to suppress reverse emergence.

[0112] The quantitative score of reverse emergence can be used as a negative factor in the stage-by-stage return function, increasing its negative weight or coefficient so that when a similar reverse emergence occurs again, the system will be punished or suffer losses.

[0113] Please refer to Figure 3 , Figure 3is a schematic diagram of an emergence identification method provided according to an embodiment of the present invention.

[0114] As an example, in order to integrate the identified emergent phenomena into the staged calculation of the reward function, two reward functions are designed:

[0115] Final reward function (total reward): used to evaluate the effect of the entire game process.

[0116] Phased reward function (instant reward): Instant rewards are calculated at each step in the process, which are used to refine the behavior evaluation during the game.

[0117] In order to effectively guide the emergence of the system, this module quantifies the identified emergence phenomenon into a score, and quantifies the calculated emergence behavior into a score E t Incorporate it into the stage-by-stage reward function to encourage components to repeat positive emergent behaviors or inhibit negative emergent behaviors during training.

[0118] r t =r env (s t ,a t )+α·E t ; Among them, r t is the periodic reward function; r env (s t , a t ) is the standard reward function of the component in the environment; α is a hyperparameter used to adjust the reward weight of emergent behavior.

[0119] The emergence identification method provided in this embodiment, when the type of emergence is positive emergence, incorporates the quantitative score of the emergence behavior into the stage reward function, which can strengthen this positive behavior. By increasing the reward of positive emergence, the system or intelligent body will be more inclined to repeat this beneficial emergence behavior, thereby promoting the positive development and optimization of the system. In addition, when the type of emergence is reverse emergence, the quantitative score of the emergence behavior is incorporated into the stage reward function and set as a negative reward, which can suppress this reverse behavior. By reducing the reward of reverse emergence, the system or intelligent body will try to avoid executing this harmful emergence behavior, thereby reducing system risk.

[0120] Please refer to Figure 3 , Figure 4 and Figure 5 , Figure 3 is a schematic diagram of an emergence identification method provided according to an embodiment of the present invention; Figure 4 is a schematic diagram of the architecture of another emergence identification method provided according to an embodiment of the present invention; Figure 5 is a schematic diagram of another emergence identification method provided according to an embodiment of the present invention.

[0121] In one possible implementation, a simulation environment is simulated and system component training is performed to construct a component relationship representation graph structure of the multi-intelligent component system based on a graph neural network, characterizing information such as the mission tasks, component capabilities, and organizational relationships between components of the multi-intelligent component system to form overall situation characteristics. At the same time, a dynamic side information update mechanism is introduced to adjust the connection relationship between components according to environmental changes or human changes; the complex structure between the multi-intelligent component systems is analyzed through the emergence recognition module, and the emergence phenomenon is identified through reconstruction error calculation; when the emergence phenomenon occurs, it is input into the staged reward function, and the system component training is re-performed until the end of the confrontation.

[0122] In this embodiment, the system component relationship construction module, the component relationship feature extraction module, the emergence identification module, and the emergence traction module construct a component relationship representation graph structure of the multi-intelligent component system, characterizing the mission tasks, component capabilities, and organizational relationships between components of the multi-intelligent component system to form overall situation characteristics. At the same time, a dynamic side information update mechanism is introduced to adjust the connection relationship between components according to environmental changes or human changes. After the system feature extraction is completed, the complex structure between components is analyzed by the autoencoder, and the emergence phenomenon is identified by reconstruction error. Finally, a dense staged reward is designed to integrate the intelligent emergence behavior or functional characteristics into the staged reward function to urge the components to repeat positive emergence behaviors or suppress reverse emergence behaviors during the training process. This patent promotes the emergence effect of each component system to form "the whole is far greater than the sum of its parts", which can effectively support the continuous emergence of better capabilities of the system.

[0123] In a possible implementation, the method further includes:

[0124] Step d1, obtaining the historical target feature vector.

[0125] Collect and organize a series of historical target feature vectors, which will be used to train the emergence recognition model. Specifically, historical feature data related to the target node can be extracted from databases, log files or other data sources.

[0126] In step d2, the historical target features are converted into low-dimensional features through the encoder network, and then decoded through the decoder network to obtain the reconstructed feature vector.

[0127] Step d3, determining the loss value corresponding to the reconstruction error based on the reconstructed feature vector.

[0128] The loss value indicates the difference between the reconstructed feature vector and the historical target feature vector, and is the goal to be minimized during model training. Specifically, a distance metric (such as mean square error (MSE) or cross entropy loss) can be used to calculate the reconstruction error.

[0129] Step d4, minimizing the loss value corresponding to the reconstruction error through an optimization algorithm, and determining the gradient of the reconstruction error with respect to the network parameters.

[0130] Use optimization algorithms (such as gradient descent, Adam, etc.) to minimize the loss value corresponding to the reconstruction error and determine the gradient of the reconstruction error with respect to the network parameters.

[0131] Step d5, update the parameters of the encoder network and the decoder network according to the gradient.

[0132] The calculated gradient is applied to the update formula of the network parameters. The updated parameters will be used for the encoding and decoding process in the next iteration. Specifically, during the neural network training process, the optimization algorithm (such as gradient descent, Adam, etc.) will use this gradient information to update the network parameters, thereby gradually reducing the reconstruction error (or more generally, the loss function value). This process will continue until a certain stopping condition is reached (such as the loss value no longer decreases significantly, the preset number of training times is reached, etc.).

[0133] Step d6, repeatedly execute the steps of obtaining low-dimensional features from historical target features through the encoder network, and then decoding through the decoder network to obtain reconstructed feature vectors to update the parameters of the encoder network and the decoder network according to the gradient, until convergence or the number of repetitions reaches a preset number, and a pre-trained emergence recognition model is obtained.

[0134] In each iteration, a new historical target feature vector is used for training. The change in loss value is monitored, and training is stopped when the loss value no longer decreases significantly or reaches the preset number of training times.

[0135] As an example, the input feature vector is passed through the encoder network to obtain a low-dimensional representation, and then passed through the decoder network to obtain a reconstructed feature vector.

[0136] z v =f encoder node (h v )

[0137]

[0138] The loss value is calculated based on the reconstruction error, which is used as a loss function to measure the difference between the input data and the decoder's reconstructed data.

[0139] Here, ||·|| represents the L2 norm and N is the number of samples.

[0140] The gradient is calculated by the back propagation algorithm and the network parameters are updated. The optimization algorithm (stochastic gradient descent) is used to minimize the reconstruction error, calculate the gradient of the loss function with respect to the network parameters, and update the parameters.

[0141] Among them, θ represents the network parameters, η represents the learning rate, represents the gradient of the loss function.

[0142] Repeat the above steps until the loss function converges or the number of repetitions reaches the preset number.

[0143] In the emergence recognition method provided in this embodiment, the encoder network can compress high-dimensional historical target features into low-dimensional feature representations. This process removes redundant information, retains key features, and provides an efficient data basis for subsequent classification or recognition tasks. At the same time, the low-dimensional features are decoded by the decoder network to obtain a reconstructed feature vector, which is compared with the original feature vector to calculate the reconstruction error. This process can intuitively reflect the model's ability to restore the original data.

[0144] In this embodiment, an emergence identification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0145] This embodiment provides an emergence recognition device, such as Figure 6 As shown, it includes: an acquisition module 601, which is used to obtain task information of the multi-agent system; wherein the task information includes: a feature vector of each node and a feature vector of the edge where each node is located; a first determination module 602, which is used to determine, based on the target node, a first message of the adjacent nodes of the target node and a second message of the edge where the target node is located; wherein the target node is any one of the nodes; a second determination module 603, which is used to aggregate the first message and the second message to determine the target feature vector of the target node; an identification module 604, which is used to input the target feature vector of the target node into a pre-trained emergence recognition model to obtain a reconstruction error value; wherein, if the reconstruction error value exceeds the preset reconstruction error value, it is determined that emergence recognition has occurred.

[0146] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0147] The emergence identification device in this embodiment is presented in the form of a functional unit, where the functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0148] The embodiment of the present invention also provides a computer device having the above Figure 6 The emergence identification device shown.

[0149] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional implementations, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0150] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0151] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0152] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0153] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0154] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0155] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0156] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0157] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An emergence identification method, characterized in that: The method comprises: Acquire task information of the multi-agent system; wherein the task information includes: a feature vector of each node and a feature vector of an edge where each node is located; Based on the target node, determine the first message of the neighboring node of the target node and the second message of the edge where the target node is located; wherein the target node is any one of the nodes; Aggregate the first message and the second message to determine a target feature vector of the target node; The target feature vector of the target node is input into a pre-trained emergence recognition model to obtain a reconstruction error value; wherein, if the reconstruction error value exceeds a preset reconstruction error value, it is determined that emergence recognition occurs.

2. The emergence identification method according to claim 1, characterized in that: The aggregating the first message and the second message to determine the target feature vector of the target node includes: Determine the weight of the type of edge corresponding to the target message using a self-attention mechanism; wherein the target message includes a first message and a second message; According to the weight of the edge type corresponding to the target message, the target message is weighted averaged to determine the feature vector of the edge where the target node is located; The feature vectors of each node on the edge where the target node is located and the feature vector of the edge where the target node is located are fused to determine the target feature vector of the target node.

3. The emergence identification method according to claim 1, characterized in that: The method further comprises: Inputting the target feature vector of the target node into a pre-trained emergence recognition model to determine a first feature vector of the target node and a second feature vector of the edge where the target node is located; Obtain the third eigenvector of the target node at the target time and the fourth eigenvector of the edge where the target node is located at the target time; Determining a reconstruction error score according to the first eigenvector, the second eigenvector, the third eigenvector, and the fourth eigenvector; According to the reconstruction error score, the type of emergence is determined; wherein the type of emergence includes: positive emergence and reverse emergence.

4. The emergence identification method according to claim 3, characterized in that: Determining the type of emergence according to the reconstruction error score includes: Determine the environmental rewards at the target moment; The environmental reward at the target moment and the reconstruction error score are combined to determine a quantitative score of the emergent behavior; When the quantitative score of the emergence behavior is greater than a preset threshold, the type of emergence is determined to be positive emergence; When the emergence behavior quantification score is not greater than a preset threshold, the emergence type is determined to be reverse emergence.

5. The emergence identification method according to claim 4, characterized in that: The method further comprises: Get the preset phased reward function; When the type of emergence is positive emergence, the quantitative score of the emergence behavior is incorporated into the preset stage reward function to achieve repeated positive emergence; When the type of emergence is reverse emergence, the quantitative score of the emergence behavior is incorporated into the preset stage-by-stage reward function to suppress reverse emergence.

6. The emergence identification method according to claim 1, characterized in that: Build a pre-trained emergence recognition model, including: Get the historical target feature vector; The historical target features are passed through the encoder network to obtain low-dimensional features, and then decoded through the decoder network to obtain the reconstructed feature vector; According to the reconstructed feature vector, determine the loss value corresponding to the reconstruction error; The loss value corresponding to the reconstruction error is minimized through the optimization algorithm to determine the gradient of the reconstruction error with respect to the network parameters; According to the gradient, update the parameters of the encoder network and the decoder network; Repeat the steps of obtaining low-dimensional features from historical target features through an encoder network, decoding them through a decoder network to obtain reconstructed feature vectors, and updating the parameters of the encoder network and the decoder network according to the gradient, until convergence or the number of repetitions reaches a preset number, thereby obtaining a pre-trained emergence recognition model.

7. An emergence recognition device, characterized in that: The device comprises: An acquisition module is used to acquire task information of the multi-agent system; wherein the task information includes: a feature vector of each node and a feature vector of an edge where each node is located; A first determination module is used to determine, based on a target node, a first message of an adjacent node of the target node and a second message of an edge where the target node is located; wherein the target node is any one of the nodes; A second determination module is used to perform aggregation processing on the first message and the second message to determine a target feature vector of a target node; The recognition module is used to input the target feature vector of the target node into a pre-trained emergence recognition model to obtain a reconstruction error value; wherein, if the reconstruction error value exceeds a preset reconstruction error value, it is determined that emergence recognition occurs.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the emergence identification method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the emergence identification method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the emergence identification method according to any one of claims 1 to 6.

Images