Strategy interaction scene auxiliary method and system of state transition probability model

Abstract

The invention discloses a strategy interaction scene auxiliary method and system of a state transition probability model, and relates to a strategy interaction and scene modeling technology, and the method comprises the steps: collecting a strategy interaction game track under a plurality of strategy combinations; based on the acquired game track, extracting a state representation vector so as to perform quantitative representation on each track point in the game track; dimensionality reduction is carried out on the extracted state representation vectors, and clustering is carried out on the state representation vectors after dimensionality reduction; constructing a directed graph according to a clustering result; and according to the obtained current game state, mapping the game state to the nodes of the directed graph, and reasoning the state of the next step based on the directed graph. According to the embodiment of the invention, the interpretability and decision support capability of the state transition diagram are remarkably improved by enhancing state characterization through contrastive learning, optimizing state division through dynamic clustering and introducing time sequence connection with weight.

Description

Technical Field

[0001] This application relates to the fields of strategy interaction and scene modeling technology, and in particular to a strategy interaction scene assistance method and system based on a state transition probability model. Background Technology

[0002] In traditional methods, end-to-end deep reinforcement learning (DRL) is the mainstream technique. It directly learns the mapping relationship from state to action through neural networks. For example, systems like AlphaGo achieve efficient decision-making by combining Monte Carlo Tree Search (MCTS) with deep networks. However, this type of method has significant limitations: Lack of black-box nature and interpretability: The decision-making process of end-to-end models lacks explicit modeling of state transition relationships, making it difficult for users to intuitively understand "why the current state leads to subsequent decisions." Insufficient global state awareness: The model focuses on locally optimal actions and struggles to provide an overview of the state evolution throughout the entire policy interaction process, potentially leading to long-term policy planning failure. Limited dynamic adaptability: When faced with changes in opponent's strategy or environmental disturbances, the end-to-end model needs to be retrained and cannot quickly adjust its state transition logic.

[0003] Existing deep reinforcement learning (DRL) directly learns policy functions (such as PPO and DQN) through neural networks and stores historical trajectories in an experience replay pool, but does not explicitly model state transitions.

[0004] In strategy-based interactive scenarios, existing technologies generally employ implicit modeling methods, primarily using deep learning models such as neural networks to learn and represent state transitions. While these methods possess powerful nonlinear fitting capabilities, the complexity of their model structures and high parameter dimensionality leads to a lack of interpretability in the resulting state transition models. Users cannot explicitly identify specific state nodes and their transition relationships, nor can they understand the logical path by which the model makes a prediction or decision. This results in the following specific technical problems: Lack of interpretability: As a "black box" structure, neural network models are difficult to output state nodes and their transition rules with semantic clarity, which limits their application in fields where decision controllability is required.

[0005] Difficulty in performing local state analysis and intervention: Without clearly defining the state nodes, users find it difficult to assess possible transition paths based on the current state, and thus cannot intervene in or adjust local strategies.

[0006] Model results are difficult to verify and track: Due to the opaque state representation and transition relationships, the system behavior lacks traceability, which increases the difficulty of testing, evaluation and iterative optimization. Summary of the Invention

[0007] This application provides a method and system for assisting in policy interaction scenarios using a state transition probability model, which significantly improves the interpretability and decision support capability of the state transition diagram.

[0008] This application provides an auxiliary method for policy interaction scenarios based on a state transition probability model, including: Collect the strategic interaction game trajectories under multiple strategy combinations, where one game trajectory is used to represent a complete game and is described in the form of a sequence; Save the trajectory feature vector corresponding to the output of the policy model after training in the game. The policy model is based on the MASK policy style constraint and is finally obtained after training through population game. Any MASK policy style constraint has a stage goal for network training, which is transformed into a vector with the same dimension as the output network to constrain the output of the policy function. The trajectory features of the collected game trajectories are extracted, and the trajectory feature vectors corresponding to the strategy model are filtered. The trajectory features and the filtered trajectory feature vectors are combined as state representation vectors to quantify each trajectory point in the game trajectory. The extracted state representation vectors are dimensionality reduced, and the dimensionality-reduced state representation vectors are then clustered. A directed graph is constructed based on the clustering results, wherein the directed graph includes nodes with interpretable text and weighted edges, the weights of which describe the transition probability from one state category to another. Based on the current game state, the game state is mapped to the nodes of the directed graph, and the next state and probability are inferred based on the directed graph.

[0009] This application provides a policy interaction scenario assistance system based on a state transition probability model, including: The data acquisition module is used to collect the strategic interaction game trajectories under various strategy combinations. One game trajectory is used to represent a complete game and is described in the form of a sequence. It also saves the trajectory feature vectors corresponding to the output of the trained strategy model in the game. The strategy model is based on the MASK strategy style constraint and is finally obtained after training through population game. Each MASK strategy style constraint has a stage goal designed for network training and is transformed into a vector with the same dimension as the output network to constrain the output of the strategy function. The preprocessing module is used to extract the trajectory features of the acquired game trajectory, filter the trajectory feature vectors corresponding to the strategy model, and combine the trajectory features and the filtered trajectory feature vectors as state representation vectors to quantify each trajectory point in the game trajectory. The representation learning module is used to reduce the dimensionality of the extracted state representation vectors; The situation clustering module is used to cluster the dimensionality-reduced state representation vectors. The graph modeling module is used to construct a directed graph based on clustering results, wherein the directed graph includes nodes with interpretable text and weighted edges, the weights of which describe the transition probability from one state category to another. The situation recognition module is used to obtain the current game state, map the game state to the nodes of the directed graph, and infer the next state and probability based on the directed graph.

[0010] This application's embodiments enhance state representation through contrastive learning, optimize state partitioning through dynamic clustering, and introduce weighted temporal connections, significantly improving the interpretability and decision support capabilities of the state transition diagram. Compared to existing technologies, the method of this application is more suitable for global state evolution analysis and real-time policy mapping in complex policy interaction scenarios.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0012] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a basic flowchart illustrating the strategy interaction scenario assistance method according to an embodiment of this application; Figure 2 This is a schematic diagram of the architecture of the strategy interaction scenario assistance system according to an embodiment of this application. Detailed Implementation

[0013] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0014] To address the technical problems mentioned in the background, this application aims to propose a method for constructing and optimizing state transition probability models in policy interaction scenarios, thereby improving the interpretability and structure of state modeling. This involves achieving interpretable representation of state nodes: extracting semantically clear state categories through trajectory collection, representation learning, and cluster analysis, thus enabling the identifiability and semantic annotation of state nodes.

[0015] Based on the temporal structure of the original policy interaction trajectory, a weighted directed graph is constructed to accurately represent the transition probabilities and directions between states. This enhances policy evaluation and decision support capabilities: users can map real-time states onto the graph structure to obtain the current state's position and possible transition paths, thereby enabling control over the policy interaction process and assisting in decision-making, and reconstructing state transition relationships with a causal structure.

[0016] By inductively learning multi-strategy trajectories, the state model achieves good adaptability to different strategy combinations and changing scenarios, enhancing the model's generalization and adaptability. Specifically, this application provides a method for assisting strategy interaction scenarios using a state transition probability model, such as... Figure 1 As shown, it includes the following steps: In step S101, the game trajectories of strategy interactions under various strategy combinations are collected. One game trajectory is used to represent a complete game and is described in the form of a sequence, such as the state sequence of the game trajectory: }in: This represents the state of the strategy interaction at time t; This represents the action to be executed. Simultaneously, it stores the trajectory feature vector corresponding to the output of the policy model after training during the game. The policy model is based on MASK policy style constraints and is the final policy model obtained after training through population game theory. Each MASK policy style constraint includes a stage objective for network training, which is transformed into a vector with the same dimension as the output network, used to constrain the output of the policy function. This algorithm further constrains the action space by setting stage objectives for network training, and finally transforms the constraints into a vector with the same dimension as the network output. This vector is used to shape the probability distribution of the network output, ultimately changing the policy action of the network output.

[0017] In step S102, trajectory features of the collected game trajectories are extracted, and trajectory feature vectors corresponding to the policy model are selected. The trajectory features and the selected trajectory feature vectors are combined as state representation vectors to quantify and represent each trajectory point in the game trajectory. Specifically, trajectory feature extraction can be performed using convolutional neural networks (CNNs), handcrafted features, etc. The policy model's ability to represent the downstream task state after interaction with the environment is considered. Low-dimensional vectors from the policy model are selected as representation vectors to obtain the policy model feature vectors.

[0018] In step S103, the extracted state representation vector is reduced in dimensionality, and the reduced state representation vector is clustered.

[0019] In step S104, a directed graph is constructed based on the clustering results, wherein the directed graph includes nodes with interpretable text and weighted edges, the weights of which are used to describe the frequency probability of transitions from one state category to another.

[0020] In step S105, based on the obtained current game state, the game state is mapped to the nodes of the directed graph, and the next state and probability are inferred based on the directed graph, thereby realizing assisted decision-making and control.

[0021] This application's embodiments enhance state representation through contrastive learning, optimize state partitioning through dynamic clustering, and introduce weighted temporal connections, significantly improving the interpretability and decision support capabilities of the state transition diagram. Compared to existing technologies, the method of this application is more suitable for global state evolution analysis and real-time policy mapping in complex policy interaction scenarios.

[0022] In some embodiments, after acquiring the game trajectory, the method further includes structuring the state information in the game trajectory, wherein structuring the state information includes denoising, standardization, and unifying dimensions, such as unifying the dimensions of the state feature vector.

[0023] In some embodiments, extracting trajectory features from the acquired game trajectories includes: firstly, extracting features from the original trajectory points, combining the extracted scalar feature values ​​to form a basic feature vector, further processing the representation vector using an interpretability module to generate interpretable text, and then using an embedding module to convert it into a semantic vector. This enhances semantic representation capabilities.

[0024] Trajectory feature vectors corresponding to the filtering strategy model This includes: the use of the policy model and the ability of the policy model to represent the state of downstream tasks after interaction with the environment; selecting low-dimensional vectors in the policy model as representation vectors, where the closer the low-dimensional vector is to the output layer, the stronger the ability related to downstream tasks it contains, and the greater the probability of it being selected.

[0025] Combined trajectory features and the selected trajectory feature vector As a state representation vector This includes: introducing an attention mechanism to assign dynamic weights to different trajectory features, thereby obtaining a state representation vector. Building an attention layer: ); Fusion characteristics: .

[0026] in, Represents attention weight, Represents the weight matrix. represents the bias term, and softmax represents the attention calculation function.

[0027] This application also proposes a density-adaptive dynamic clustering method. This method introduces the state similarity between trajectory points (calculated from the state representation vector trained by contrastive learning) as a metric, and combines local density and distance thresholds to dynamically determine the cluster centers and the number of clusters, thereby achieving flexible partitioning of the state space. Furthermore, addressing the state drift and staged clustering characteristics in multi-strategy interaction processes, this application further proposes an incremental clustering update mechanism. By updating only the local state cluster structure when new trajectory data arrives, it avoids full re-clustering operations, improves system response efficiency, and preserves the consistency of the original state clustering.

[0028] In some embodiments, clustering the dimensionality-reduced state representation vectors includes: First, state similarity calculation and feature extraction are performed. From the collected multi-round game trajectories, a contrastive learning model is used to generate a high-dimensional state representation vector for each trajectory point. Let the... The state vectors are Then the similarity is defined as:

[0029] This similarity is used in subsequent density estimation and clustering decisions to ensure that states are semantically separable.

[0030] Secondly, adaptive density estimation and cluster center selection are performed for each state point. Its local density is defined as:

[0031] in, It is an adjustable smoothing coefficient that reflects the contribution weight of neighboring states to the local density.

[0032] For each point, calculate its minimum distance (or minimum "similarity loss") to all points with higher density:

[0033] Finally, a two-dimensional "density-distance" decision graph is constructed to identify state points with high local density that are far from other high-density points as cluster centers. This method does not require a preset number of clusters K and can adaptively identify cluster structures.

[0034] Furthermore, dynamic cluster allocation and boundary optimization are performed. After the cluster centers are determined, the remaining samples are recursively assigned to the nearest cluster centers based on state similarity. For boundary state points, a fuzzy assignment strategy is adopted, i.e., their assignment weights are... Assigned to multiple candidate clusters:

[0035] Where γ is a temperature coefficient that controls the degree of fuzziness. Let j be the center of the j-th cluster. This mechanism can effectively improve the robustness of state boundary identification.

[0036] Considering the continuous growth of policy trajectories in real-world environments, in some embodiments, the method of this application designs an incremental local update process, including the following steps: When the new state point Upon arrival: Calculate its similarity to existing cluster centers; If the maximum similarity is greater than the threshold θ, then it is merged into the corresponding cluster; Otherwise, initialize new cluster centers and dynamically expand the cluster structure; Update the local density and similarity map; All update operations are limited to the affected cluster to avoid global recalculation.

[0037] This approach maintains the consistency of the original clustering while effectively addressing state drift and the introduction of new strategies, thereby improving the system's online learning capabilities and real-time responsiveness.

[0038] In some embodiments, dimensionality reduction of the extracted state representation vector includes: This application proposes a feature dimensionality reduction method that integrates contrastive learning and semantic information. The input includes high-dimensional feature vectors and their corresponding textual descriptions. Specifically, a multimodal embedding space composed of feature vectors and textual descriptions is first constructed. An encoder maps the textual information into semantic vectors, which are then fused with the original features. Next, a contrastive learning strategy is used to construct positive and negative sample pairs, maximizing the similarity between semantically consistent samples and minimizing the similarity between irrelevant samples. By introducing a semantically enhanced supervisory signal, the dimensionality reduction process is guided to retain semantic discriminative ability, thereby achieving more effective low-dimensional representation learning and improving the performance of downstream tasks. The output after dimensionality reduction is: .

[0039] or, This application proposes an alternative feature dimensionality reduction method that combines scenario-based prior knowledge with Principal Component Analysis (PCA) to extract high-information-content low-dimensional representations from high-dimensional state features. Specifically, a prior knowledge base is first constructed based on the specific application scenario, including environmental attributes, target behavior patterns, and domain experience rules. On this basis, an interpretable rule-based method is designed to preprocess the original data and construct features, significantly improving the structural and semantic relevance of feature representations. Subsequently, PCA is used to reduce the dimensionality of the processed high-dimensional features, extracting principal components along the direction of maximum variance and retaining the most informative low-dimensional vectors. Compared to traditional PCA, this method exhibits stronger scenario adaptability and feature compression robustness, significantly reducing dimensionality reduction errors and improving subsequent modeling performance. The output after dimensionality reduction is: .

[0040] In some embodiments, clustering the dimensionality-reduced state representation vectors includes: K-means or DBSCAN clustering algorithms are used to group all dimensionality-reduced state representation vectors to obtain typical state categories. Each category represents a state node, and each node is accompanied by interpretable semantic features. Descriptive statistical features include average state and typical action distribution. Mapping the implicit state space to a finite number of semantically informational "state categories" while retaining the original representative features is a key step that distinguishes it from the "black box state" of traditional neural networks, thus improving the interpretability of the overall model.

[0041] In some embodiments, constructing a directed graph based on clustering results includes: The nodes in the directed graph are used to represent state categories, the edges between nodes in the directed graph are used to represent state transitions, and transition probabilities are assigned to the edges. The weights of the corresponding edges form a directed graph G=(V,E,W). Represents the set of state nodes, Represents the set of directed edges. The edge weight matrix is ​​represented by this. The method in this application establishes probabilistic directed edges (directed graphs) between state nodes based on the temporal sequence in the trajectory. This directed graph structure not only preserves the dynamic evolution path of policy interactions, but also supports subsequent inference and prediction.

[0042] Finally, in step S105, based on the acquired current game state, the game state is mapped to the nodes of the directed graph, and the next state is inferred based on the directed graph, thereby achieving assisted decision-making and control. That is, in the specific example, the current real-time strategy interaction state is mapped to specific state nodes in the graph, supporting path prediction, transition probability deduction, and strategy interaction direction control.

[0043] The method in this application utilizes clustering results to construct "explicit state nodes," which are non-black-box and enable interpretable modeling. It extracts causal transition sequences from trajectories to express dynamic policy interaction logic, constructs a structured transition graph, and supports reasoning and control based on this graph. It supports real-time mapping, prediction, and intervention, realizing a closed-loop application logic from "perception → understanding → reasoning."

[0044] The method in this application combines contrastive learning, PCA, and clustering methods to enhance the representational power and usability of the model.

[0045] This application also proposes a strategy interaction scenario assistance system based on a state transition probability model, such as... Figure 2 As shown, it includes: The data acquisition module is used to collect the strategic interaction game trajectories under various strategy combinations. One game trajectory is used to represent a complete game and is described in the form of a sequence. It also saves the trajectory feature vectors corresponding to the output of the trained strategy model in the game. The strategy model is based on the MASK strategy style constraint and is finally obtained after training through population game. Each MASK strategy style constraint has a stage goal designed for network training and is transformed into a vector with the same dimension as the output network to constrain the output of the strategy function. The preprocessing module is used to extract the trajectory features of the acquired game trajectory, filter the trajectory feature vectors corresponding to the strategy model, and combine the trajectory features and the filtered trajectory feature vectors as state representation vectors to quantify each trajectory point in the game trajectory. The representation learning module is used to reduce the dimensionality of the extracted state representation vectors; The situation clustering module is used to cluster the dimensionality-reduced state representation vectors. The graph modeling module is used to construct a directed graph based on clustering results, wherein the directed graph includes nodes with interpretable text and weighted edges, the weights of which describe the transition probability from one state category to another. The situation recognition module is used to obtain the current game state, map the game state to the nodes of the directed graph, and infer the next state and probability based on the directed graph.

[0046] In some embodiments, the representation learning module is specifically used for: By employing contrastive learning, the distance between similar states is narrowed, while the distance between dissimilar states is widened, resulting in discriminative state representation vectors for dimensionality reduction; or, Principal component analysis is used to retain the low-dimensional vector representation with the most information to achieve dimensionality reduction.

[0047] In some embodiments, the situation clustering module is specifically used for: All dimensionality-reduced state representation vectors are grouped using K-means or DBSCAN clustering algorithms to obtain typical state categories, where each category represents a state node, and each node is accompanied by descriptive statistical features.

[0048] In some embodiments, the graph modeling module is specifically used for: The nodes in the directed graph are used to represent state categories, the edges between nodes in the directed graph are used to represent state transitions, and transition probabilities are assigned to the edges. The weights of the corresponding edges form a directed graph G=(V,E,W). Represents the set of state nodes, Represents the set of directed edges. This represents the edge weight matrix.

[0049] This application employs multi-strategy game analysis to generate diverse trajectories, ensuring broad data coverage and rich information, providing a solid foundation for modeling and avoiding modeling distortion caused by "data bias." It introduces contrastive learning or principal component analysis for state dimensionality reduction and combines clustering to extract semantically meaningful state nodes, making the model no longer a "black box" and significantly improving the interpretability and controllability of the results. This application constructs a state transition graph through temporal information in the trajectory, forming a structured, graphical state evolution model, enabling real-time states to accurately map, predict paths, and assist in strategic interaction decision-making.

[0050] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0051] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0052] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0053] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A method for assisting policy interaction scenarios using a state transition probability model, characterized in that, include: Collect the strategic interaction game trajectories under multiple strategy combinations, where one game trajectory is used to represent a complete game and is described in the form of a sequence; Save the trajectory feature vector corresponding to the output of the policy model after training in the game. The policy model is based on the MASK policy style constraint and is finally obtained after training through population game. Any MASK policy style constraint has a stage goal for network training, which is transformed into a vector with the same dimension as the output network to constrain the output of the policy function. The trajectory features of the collected game trajectories are extracted, and the trajectory feature vectors corresponding to the strategy model are filtered. The trajectory features and the filtered trajectory feature vectors are combined as state representation vectors to quantify each trajectory point in the game trajectory. The extracted state representation vectors are dimensionality reduced, and the dimensionality-reduced state representation vectors are then clustered. A directed graph is constructed based on the clustering results, wherein the directed graph includes nodes with interpretable text and weighted edges, the weights of which describe the transition probability from one state category to another. Based on the current game state, the game state is mapped to the nodes of the directed graph, and the next state and probability are inferred based on the directed graph.

2. The strategy interaction scenario assistance method for the state transition probability model as described in claim 1, characterized in that, After collecting the game trajectory, the process also includes structuring the state information in the game trajectory, which includes denoising, standardization, and unifying dimensions.

3. The strategy interaction scenario assistance method for the state transition probability model as described in claim 1, characterized in that, The trajectory features extracted from the collected game trajectories include: By extracting features from the original trajectory points, and combining the extracted scalar feature values ​​to form a basic feature vector; The representation vectors are processed to generate interpretable text, and then embedded to transform them into semantic vectors. ; Trajectory feature vectors corresponding to the filtering strategy model This includes selecting a low-dimensional vector from the output of the policy model as a representation vector. The closer the low-dimensional vector is to the output layer, the stronger its ability to be related to downstream tasks, and the greater its probability of being selected. Combined trajectory features and the selected trajectory feature vector As a state representation vector This includes: introducing an attention mechanism to assign dynamic weights to different trajectory features, thereby obtaining a state representation vector. Building an attention layer: ; Fusion characteristics: ; in, Represents attention weight, Represents the weight matrix. represents the bias term, and softmax represents the attention calculation function.

4. The strategy interaction scenario assistance method for the state transition probability model as described in claim 3, characterized in that, Clustering the dimensionality-reduced state representation vectors includes: State similarity calculation and feature extraction: Based on the dimensionality-reduced state representation vector, a contrastive learning model is used to generate a state vector for each trajectory point. Let the first... The state vectors are Then the similarity is defined as: Adaptive density estimation and cluster center selection: for each state point Its local density is defined as: in, It is an adjustable smoothing coefficient; For each point, calculate its minimum distance to all points with higher density: Construct a two-dimensional "density-distance" decision graph to identify state points with high local density that are far away from other high-density points as cluster centers; Perform dynamic cluster allocation and boundary optimization: After the cluster centers are determined, the remaining samples are recursively assigned to the nearest cluster centers based on state similarity; and, For boundary state points, a fuzzy attribution strategy is adopted, that is, their attribution weights are determined. Assigned to multiple candidate clusters: Where γ is a temperature coefficient that controls the degree of fuzziness. It is the center of the j-th cluster.

5. The strategy interaction scenario assistance method for the state transition probability model as described in claim 4, characterized in that, Clustering the dimensionality-reduced state representation vectors also includes the following incremental local update process: When the new state point Upon arrival, calculate its similarity to existing cluster centers; If the maximum similarity is greater than the threshold θ, then merge it into the corresponding cluster; otherwise, initialize new cluster centers to dynamically expand the cluster structure. Update the local density and similarity map.

6. The strategy interaction scenario assistance method for the state transition probability model as described in claim 3, characterized in that, Dimensionality reduction of the extracted state representation vector includes: This method employs a feature dimensionality reduction approach that integrates contrastive learning and semantic information. The input includes high-dimensional feature vectors and their corresponding feature text descriptions. A multimodal embedding space composed of feature vectors and text descriptions is constructed. The text information is mapped into semantic vectors by an encoder and fused with the original state representation vectors. By employing a contrastive learning strategy, positive and negative sample pairs are constructed to maximize the similarity between semantically consistent samples and minimize the similarity between irrelevant samples, thereby achieving dimensionality reduction.

7. The strategy interaction scenario assistance method for the state transition probability model as described in claim 3, characterized in that, Dimensionality reduction of the extracted state representation vector includes: A feature dimensionality reduction method combining scenario prior knowledge and principal component analysis (PCA) is employed to extract high-information low-dimensional representations from high-dimensional state features: A prior knowledge base is constructed based on the application scenario. The prior knowledge base includes environmental attributes, target behavior patterns, and domain experience rules. Design interpretable rule-based methods to preprocess and construct features from the original state representation vectors; PCA is used to reduce the dimensionality of the processed high-dimensional features, extract the principal components along the direction of maximum variance, and retain the low-dimensional vectors with the most information content.

8. The strategy interaction scenario assistance method for the state transition probability model as described in claim 1, characterized in that, Clustering the dimensionality-reduced state representation vectors includes: All dimensionality-reduced state representation vectors are grouped using K-means or DBSCAN clustering algorithms to obtain typical state categories, where each category represents a state node, and each node is accompanied by descriptive statistical features.

9. The strategy interaction scenario assistance method for the state transition probability model as described in claim 7, characterized in that, Constructing a directed graph based on clustering results includes: The nodes in the directed graph are used to represent state categories, the edges between nodes in the directed graph are used to represent state transitions, and transition probabilities are assigned to the edges. The weights of the corresponding edges form a directed graph G=(V,E,W). Represents the set of state nodes, Represents the set of directed edges. This represents the edge weight matrix.

10. A policy interaction scenario assistance system based on a state transition probability model, characterized in that, include: The data acquisition module is used to collect the strategic interaction game trajectories under various strategy combinations. One game trajectory is used to represent a complete game and is described in the form of a sequence. It also saves the trajectory feature vectors corresponding to the output of the trained strategy model in the game. The strategy model is based on the MASK strategy style constraint and is finally obtained after training through population game. Each MASK strategy style constraint has a stage goal designed for network training and is transformed into a vector with the same dimension as the output network to constrain the output of the strategy function. The preprocessing module is used to extract the trajectory features of the acquired game trajectory, filter the trajectory feature vectors corresponding to the strategy model, and combine the trajectory features and the filtered trajectory feature vectors as state representation vectors to quantify each trajectory point in the game trajectory. The representation learning module is used to reduce the dimensionality of the extracted state representation vectors; The situation clustering module is used to cluster the dimensionality-reduced state representation vectors. The graph modeling module is used to construct a directed graph based on clustering results, wherein the directed graph includes nodes with interpretable text and weighted edges, the weights of which describe the transition probability from one state category to another. The situation recognition module is used to obtain the current game state, map the game state to the nodes of the directed graph, and infer the next state and probability based on the directed graph.

Images