A Construction Supervision Decision-Making Method and System Based on Game Optimization and Multi-Agent Reinforcement Learning

Abstract

This invention discloses a construction supervision decision-making method and system based on game optimization and multi-agent reinforcement learning, relating to the field of construction supervision decision-making. This invention constructs a digital twin environment based on a Stackelberg master-slave game architecture, combines graph attention networks to extract features from the dynamic spatiotemporal interaction graph of the construction site, applies continuously differentiable hard safety constraints to the game process using a safety risk potential energy field based on physical field theory, and employs a multi-agent reinforcement learning algorithm based on Lagrange duality optimization to solve the Nash equilibrium strategy. This enables the supervision agent and the construction / resource agent to achieve an automatic balance between risk avoidance and schedule optimization in complex dynamic games, realizing intelligent collaborative decision-making that balances safety compliance and construction efficiency.

Description

Technical Field

[0001] This invention relates to the field of construction supervision decision-making, specifically to a construction supervision decision-making method and system based on game optimization and multi-agent reinforcement learning. Background Technology

[0002] With the rapid development of large-scale infrastructure construction and smart construction site technologies, modern construction sites have evolved into complex mega-systems deeply coupled with personnel, machinery, materials, and the environment. Existing construction supervision and decision-making technologies primarily rely on the combination of Building Information Modeling (BIM) and Internet of Things (IoT) sensing technologies. This involves deploying cameras, location tags, and environmental sensors on the construction site to achieve real-time data collection and visual monitoring of the site's status. In terms of decision support, traditional methods typically employ the Critical Path Method (CPM) or Program Evaluation and Review Technique (PERT) to develop static construction schedules, combined with expert systems or rule-based inference engines for safety warnings. For example, when sensors detect equipment being too close or the working environment exceeding limits, the system triggers an alarm based on preset fixed thresholds. Furthermore, some advanced solutions are beginning to utilize single-agent reinforcement learning or genetic algorithms to optimize construction resource scheduling, attempting to find the shortest or lowest-cost scheduling scheme while meeting constraints, thereby achieving digital management of the construction process.

[0003] However, the aforementioned existing technologies have significant shortcomings when facing the multi-objective dynamic game scenario of "safety risk assessment - schedule control - resource conflict". First, rule-based or static planning methods are difficult to adapt to the rapidly changing dynamic environment of construction sites and cannot handle the complex spatiotemporal interference problems when multiple trades are working simultaneously, leading to decision-making delays. Second, existing technologies often treat the construction unit (pursuing schedule and cost) and the supervision unit (pursuing safety and compliance) as a unified entity with aligned interests for centralized optimization, ignoring the inherently non-cooperative game relationship between the two, making it difficult to implement the generated decision instructions in actual conflicting interests. More importantly, traditional safety constraint handling methods are mostly hard blocking with "trigger-based work stoppage", lacking a continuous risk quantification mechanism based on physical field theory, and unable to guide the agent to make flexible proactive avoidance when approaching the risk boundary, resulting in a "zero-sum game" state between safety and schedule, making it difficult to achieve the optimal solution for efficiency while ensuring absolute safety. Summary of the Invention

[0004] This invention proposes a construction supervision decision-making method and system based on game optimization and multi-agent reinforcement learning, aiming to solve the technical problem that existing construction supervision decision-making technologies cannot effectively balance safety risk control, project progress assurance and resource conflict resolution under the circumstances of multiple stakeholders' conflicting interests and dynamic spatiotemporal interference.

[0005] The construction supervision decision-making method based on game optimization and multi-agent reinforcement learning includes the following steps:

[0006] S1. Based on BIM model data and on-site sensor data, define a set of decision-making intelligent agents including construction intelligent agents, supervision intelligent agents and resource intelligent agents. Construct a dynamic spatiotemporal interaction diagram based on the physical proximity and operational dependencies between intelligent agents, and initialize the state space and action space of intelligent agents.

[0007] Specifically, this step performs semantic recognition and instantiation on multi-source heterogeneous entities, including wall structures, large mechanical equipment, and construction personnel, to define a set of intelligent agents with independent decision-making capabilities. Then, the Euclidean distance calculation algorithm is used to quantify the spatial proximity between each intelligent agent in real time and dynamically generate connection edges between nodes by combining the logical pre-dependencies in the construction schedule. This rule-based and data-driven graph construction method transforms discrete site elements into a dynamic spatiotemporal interaction graph with graph theory characteristics. In turn, a local observation state space containing its own position, velocity, and load information, as well as an action space containing movement, turning, and work intensity adjustment instructions, are constructed for each intelligent agent. This allows for a structured representation of the complex spatiotemporal constraints of the construction site and establishes the basic environment and boundary conditions for the operation of the multi-agent game algorithm.

[0008] S2. Assign the supervisory agent the leader role according to the Stackelberg game logic, and assign the construction agent and resource agent the follower role. Define the game utility function to quantify the operation benefits and risk costs, and set the safety risk potential energy field based on physical field theory as the hard constraint of the game.

[0009] Specifically, this step, based on the actual command chain at the construction site, sets the supervisory agent with the highest safety veto power as the leader in the Stackelberg game and grants it the priority decision-making power to set global risk thresholds and issue rectification orders. At the same time, the construction and resource agents, which pursue economic benefits and operational efficiency, are set as followers, enabling them to make suboptimal decision optimizations after observing the leader's strategic constraints. In order to solve the mathematical problem of the difficulty in quantifying safety constraints in the game, this step introduces the theory of artificial potential energy field to simulate the dangerous areas and mobile machinery at the construction site as physical field sources that generate repulsive forces. By calculating the gradient change of potential energy values, a continuously differentiable safety risk penalty function is constructed and used as the core negative feedback term in the game utility function.

[0010] S3. Input the dynamic spatiotemporal interaction graph into the graph attention policy network, extract neighborhood feature representations through the graph attention layer, map the master-slave game architecture into a partially observable Markov decision process, and transform the game utility function into an immediate reward value for reinforcement learning.

[0011] Specifically, this step inputs the constructed dynamic spatiotemporal interaction graph into the graph attention policy network and uses a multi-head attention mechanism to automatically calculate the correlation coefficient between the central node and its neighboring nodes. This coefficient can dynamically measure the importance of different neighbors to the current decision-making agent, thereby achieving weighted aggregation and high-dimensional embedding of neighborhood features. Subsequently, the system concatenates this embedding vector, which integrates the topological structure, with the agent's own private observation state to form a joint state representation in the partially observable Markov decision process. At the same time, the game utility function, which includes positive incentives for task progress and negative penalties for safety potential, is directly mapped to the immediate reward signal in the reinforcement learning environment. This allows the agent to directly perceive the merits of the current strategy under the game model through interaction with the environment and obtain the gradient direction for parameter updates.

[0012] S4. Place the set of decision-making agents into a digital twin environment for interaction, construct a dual objective function based on the security risk potential field and the instantaneous reward value, and use a restricted multi-agent reinforcement learning algorithm to alternately update the policy network parameters and Lagrange multipliers until the security constraints are satisfied.

[0013] Specifically, this step, within a centralized training and distributed execution architecture, transforms the safety risk potential field constraint, originally a hard constraint, into a Lagrange multiplier penalty term in the objective function, thus constructing a dual objective function that includes maximizing the original profit and minimizing constraint violations. During training, an adversarial iterative process is employed using the original dual update algorithm. Specifically, with the Lagrange multipliers fixed, gradient descent is used to update the policy network parameters to find the optimal policy under the current penalty level. Simultaneously, with the policy network fixed, gradient ascent is used to update the Lagrange multipliers to increase the penalty weight for violations. This dynamic adjustment mechanism forces the policy network to continuously converge towards the feasible region satisfying the safety constraints during training, ultimately finding a Nash equilibrium point within the safety boundary.

[0014] S5. Acquire real-time field data, process it through the trained graph attention strategy network, output joint action vectors, and decode to generate construction supervision collaborative decision-making instructions;

[0015] Specifically, this step involves real-time access to the field sensor data stream and inputting it into a trained graph attention strategy network for forward propagation calculation to output the action probability distribution of each agent in the current state. To ensure the physical compliance of the decision, the system uses a masking mechanism to filter out invalid actions that violate mechanical physical limits or operating specifications, and generates joint discrete action vectors based on the corrected probability distribution. Subsequently, a pre-set semantic mapping dictionary is used to parse the abstract numerical action vectors into natural language instructions that can be understood by field personnel, such as specific deceleration values ​​or avoidance directions. These decision instructions, which contain spatiotemporal collaborative logic, are then accurately pushed to the corresponding mobile terminals or equipment controllers through a low-latency communication network to achieve closed-loop intelligent control of the construction site.

[0016] The beneficial effects of the invention are:

[0017] This invention constructs a digital twin environment based on a Stackelberg master-slave game architecture, extracts features from the dynamic spatiotemporal interaction graph of the construction site using a graph attention network, applies continuously differentiable hard safety constraints to the game process using a safety risk potential energy field based on physical field theory, and employs a multi-agent reinforcement learning algorithm based on Lagrange dual optimization to solve the Nash equilibrium strategy. This enables the supervision agent and the construction / resource agent to achieve an automatic balance between risk avoidance and schedule optimization in complex dynamic games, realizing intelligent collaborative decision-making that balances safety compliance and construction efficiency. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the construction supervision decision-making method based on game optimization and multi-agent reinforcement learning proposed in Embodiment 1 of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, and not all of them. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention. It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0022] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or machine that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or machine. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or machine that includes said element.

[0023] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0024] Example 1

[0025] Among them, such as Figure 1 A construction supervision decision-making method based on game optimization and multi-agent reinforcement learning includes the following steps:

[0026] S1. Based on BIM model data and on-site sensor data, define a set of decision-making intelligent agents including construction intelligent agents, supervision intelligent agents and resource intelligent agents. Construct a dynamic spatiotemporal interaction diagram based on the physical proximity and operational dependencies between intelligent agents, and initialize the state space and action space of intelligent agents.

[0027] S2. Assign the supervisory agent the leader role according to the Stackelberg game logic, and assign the construction agent and resource agent the follower role. Define the game utility function to quantify the operation benefits and risk costs, and set the safety risk potential energy field based on physical field theory as the hard constraint of the game.

[0028] S3. Input the dynamic spatiotemporal interaction graph into the graph attention policy network, extract neighborhood feature representations through the graph attention layer, map the master-slave game architecture into a partially observable Markov decision process, and transform the game utility function into an immediate reward value for reinforcement learning.

[0029] S4. Place the set of decision-making agents into a digital twin environment for interaction, construct a dual objective function based on the security risk potential field and the instantaneous reward value, and use a restricted multi-agent reinforcement learning algorithm to alternately update the policy network parameters and Lagrange multipliers until the security constraints are satisfied.

[0030] S5. Acquire real-time on-site data, process it through the trained graph attention strategy network, output a joint action vector, and decode it to generate collaborative decision-making instructions for construction supervision.

[0031] It should be noted that this embodiment abstracts the supervisory agent as a rulemaker in a game theory system rather than a mere participant. By assigning it the leader position in a Stackelberg game, it simulates the information advantage and initiative advantage possessed by supervisory units in real-world management. That is, the supervisory agent's decision-making is no longer a passive response to a single risk point, but rather constructs a dynamic compliance solution space by adjusting the global risk tolerance threshold or issuing mandatory rectification orders. The essence of its action space is the dynamic reshaping of the system's safety boundary. For example, lowering the risk threshold under severe weather conditions effectively shrinks the set of feasible strategies for all followers, thereby macroscopically forcibly reducing the overall entropy of the system and ensuring that the construction system always operates within a controllable safety envelope. Furthermore, by constructing a follower decision-making model based on bounded rationality, the construction agent and resource agent are abstracted as "economic men" pursuing their own profit maximization, but their decision-making behavior is strongly constrained by the leader's strategy. In the implementation logic, followers must constantly observe the safety boundary defined by the leader and, under this premise, find the optimal solution for the progress. This mechanism simulates how construction site teams and equipment must balance schedule pressure and cost control by finely adjusting their operating rates or resource bidding strategies while adhering to safety regulations. Furthermore, by using the concept of potential energy from field theory, abstract safety risks are visualized as a repulsive field in a virtual physical space; discrete safety regulations are transformed into continuously differentiable mathematical constraints, addressing the difficulty in quantifying safety indicators in traditional game theory. By constructing a game utility function, the system mathematically creates a multi-objective trade-off plane. On this plane, the positive benefits generated by task progress interact with the negative penalties generated by the safety potential field, forcing the agent to conduct a "cost-benefit analysis" during decision-making. This encourages efficient operation in low-risk areas, while rapidly generating significant virtual resistance (penalty) when high risks approach, thus achieving a paradigm shift from simple "post-event violation punishment" to "pre-event risk game" management at the algorithmic level.

[0032] Furthermore, in this embodiment, the digital twin environment establishes the static physical boundaries and work area constraints of the construction site by parsing BIM model data. It combines a set of decision-making agents defined based on on-site sensor data and divided into leader and follower roles according to Stackelberg game logic. The dynamic spatiotemporal interaction graph constructed based on the physical proximity and work dependency relationships between agents is used as the interaction topology basis, thereby mapping the set of decision-making agents to the static physical boundaries and work area constraints for assembly and construction. In step S1, by parsing the BIM model data, the coordinate set of insurmountable obstacles at the construction site (including walls, columns, and foundation pit edges) and the topology of functional areas (including material storage areas, fire lanes, and high-voltage line restricted areas) are extracted to establish a static environmental constraint layer for the digital twin environment. Based on on-site sensor data, supervision agents, construction agents, and resource agents are instantiated. According to Stackelberg game logic, the supervision agent is assigned the leader role, and the construction agent and resource agent are assigned the follower role to establish a dynamic decision layer for the digital twin environment. Furthermore, the Euclidean distance and process dependency relationship between each agent are calculated, and directed connection edges are established to form a dynamic spatiotemporal interaction graph. The dynamic decision layer is placed into the static environmental constraint layer, and the dynamic spatiotemporal interaction graph is used as an information flow channel to assemble and form the digital twin environment of the construction site, and the state space and action space of each agent are initialized.

[0033] Furthermore, step S1 specifically includes the following sub-steps:

[0034] S101. Map each agent in the set of decision agents to a graph node, and encode the agent's own attributes into a node feature vector;

[0035] S102. Calculate the Euclidean distance between any two intelligent agents. When the Euclidean distance is less than the preset communication radius or there is an upstream and downstream process dependency, establish a directed edge between the nodes to form an adjacency matrix.

[0036] S103. Combine the node feature vectors with the adjacency matrix to form a dynamic spatiotemporal interaction graph at time t. , wherein Represents a set of nodes, the Represents the set of edges.

[0037] Specifically, the above implementation transforms unstructured physical information of the construction site into computer-processable graph data, thus providing a standardized environmental observation foundation for multi-agent algorithms. In practice, the system first extracts the geometric dimensions and attribute information of entities such as walls, machinery, and personnel by parsing the IFC standard data in the BIM file. Combined with real-time status data transmitted from on-site sensors, these entities are instantiated as nodes in a graph network. For each node, embedding technology is used to encode heterogeneous attributes such as the agent's type, current speed, and load status into a high-dimensional feature vector, establishing the agent's own state representation.

[0038] Furthermore, the system constructs connection edges between nodes based on physical spatial relationships and operational logic relationships. On the one hand, the system calculates the Euclidean distance between all nodes in real time. Once the distance between two agents (such as a tower crane and a nearby building) is less than the preset safety communication or collision warning radius, a connection edge is established to characterize potential physical interaction risks. On the other hand, the system identifies the dependencies between work processes based on the construction schedule (such as a PERT chart). If the work of one construction unit is a prerequisite for another unit, a logical connection edge is established between the two.

[0039] Furthermore, by aligning and encapsulating the generated node feature matrix and adjacency matrix in the time dimension, a dynamic spatiotemporal interaction graph representing the overall situation at time t is constructed. This graph structure can map the dynamic topology of the construction site over time in real time, not only quantifying the local spatial relationships between agents, but also explicitly expressing complex operational coupling constraints through edge connections.

[0040] Furthermore, in step S2, the specific process of establishing a master-slave game architecture with the supervisory agent as the leader and other agents as followers includes the following sub-steps:

[0041] S201. Define the supervisory agent as the leader of the Stackelberg game, wherein the action space of the supervisory agent includes issuing safety rectification instructions and adjusting regional risk thresholds;

[0042] S202. Define the construction agent and the resource agent as followers in a Stackelberg game. The action space of the construction agent and the resource agent includes adjusting the construction rate and requesting resource bids. The followers make decisions after observing the leader's actions.

[0043] S203. Calculate the safety risk penalty term using the safety risk potential energy field of physical field theory, and construct a game utility function that includes the task progress benefit term and the safety risk penalty term.

[0044] Specifically, based on management authority in actual business operations, the system designates the supervisory agent as the leader in the game. The leader possesses priority decision-making power, and its decision-making logic focuses on macro-level risk control and compliance supervision. Its action space is designed to issue global safety rectification instructions or dynamically adjust risk tolerance thresholds within a region, thereby guiding the overall operation of the system. Correspondingly, the construction agent and resource agent are designated as followers. As latecomers, they can only optimize their strategies after observing the rules set by the leader (such as risk thresholds). The followers' action space focuses on specific execution levels, such as adjusting the rotation rate of the robotic arm or submitting resource bidding applications. Their goal is to maximize their operational efficiency and economic benefits as much as possible without violating the safety bottom line set by the leader. To quantify this game process, a composite game utility function is used, incorporating task progress benefits and safety risk penalties. The safety risk penalty term is key to the game constraint; the system introduces field theory concepts to calculate the current safety risk potential energy value of each agent. Once an agent's behavior leads it into a high-potential region (i.e., a high-risk region), the penalty term in the utility function will increase sharply, thus forming a strong negative feedback mechanism at the mathematical level. This forces followers to actively avoid high-risk behaviors while pursuing progress benefits, thereby achieving a dynamic balance between safety and progress.

[0045] For example, the security risk penalty item is specifically represented as follows:

[0046] ;

[0047] Among them, the Indicates the agent at time t. The security risk penalty item, this value is directly substituted into the game utility function; the aforementioned This represents the base penalty coefficient, used to balance the order of magnitude of the penalty term and the task benefit term; the... This represents an exponential function, used to construct nonlinear high-gradient penalty boundaries; This represents the risk sensitivity factor; the higher the value, the more severe the increase in penalty after exceeding the threshold. This indicates a modified linear unit operation that ensures a penalty is only incurred when the potential energy value exceeds a threshold; the... This represents the preset safety potential energy threshold, corresponding to the safety warning distance in physical space.

[0048] Furthermore, in step S203, the game utility function is specifically expressed as follows:

[0049] ;

[0050] Among them, the Indicates the first An intelligent agent at time The game utility value; the The weighting coefficients representing the task progress objectives; This represents the progress-based reward function for a regular task, where For a moment The state of union, For a moment The combined action; the The weighting coefficients representing security risk objectives; Indicates the first An intelligent agent at time The safety risk potential energy value is determined by a mathematical weighting method to quantify the incentives and risk constraints faced by the agent during the decision-making process. The formula consists of a positive progress benefit component and a negative risk penalty component. The task progress benefit component is directly related to the agent's output, such as the amount of work completed or the shortened construction period, incentivizing the agent to take proactive actions to advance the project. However, to prevent the agent from neglecting safety in pursuit of progress, a risk penalty component based on an exponential function is introduced. This component utilizes the non-linear growth characteristic of the exponential function; when the agent's safety risk potential energy value is low, the penalty value increases gradually, allowing the agent to perform normal construction work; however, once the risk potential energy value approaches or exceeds a critical point, the exponential operation causes the penalty value to increase explosively, significantly reducing the total utility value. This design constructs a soft safety barrier during algorithm training, enabling the agent to keenly perceive risk boundaries and automatically learn the optimal strategy to maximize benefits within a safe range. The introduction of a risk sensitivity factor allows the system to flexibly adjust its tolerance and penalty intensity based on the severity of different construction stages.

[0051] Furthermore, for the reward function of a regular task As an example, a specific implementation scheme is given, wherein the conventional task reward function A distance-guided dense reward mechanism is employed to address the training convergence difficulties caused by sparse rewards. The specific calculation logic is as follows: The first Euclidean distance between the agent's position at the previous moment before executing the action and the preset task target point in the BIM model is acquired in real time. Simultaneously, the second Euclidean distance between the agent's current position after executing the action and the task target point is acquired. The effective displacement of the agent within that time step is quantified by calculating the difference between the first and second Euclidean distances. If the difference is positive, it indicates that the agent is moving closer to the target, thus granting a positive progress reward; if the difference is negative, it indicates that the agent is moving away from the target, thus granting a negative progress penalty. A larger absolute value of the difference represents a faster movement speed and a correspondingly larger reward / penalty value. Furthermore, a milestone event judgment logic is introduced: when the second Euclidean distance is less than a preset judgment threshold, the task is judged as completed, and an additional fixed high completion reward value is added. This guides the agent to complete the construction task with the highest efficiency along the shortest path.

[0052] Furthermore, the specific calculation process for the aforementioned safety risk potential energy value is as follows:

[0053] ;

[0054] Among them, the Indicates the first An intelligent agent at time The safety risk potential energy value, the Indicates index Belongs to intelligent agents The set of neighboring nodes The Represents the Coulomb risk propagation constant; the Indicates the first The dangerous load factor of each neighboring node; Indicates time intelligent agent With intelligent agents The Euclidean distance between them; Represents intelligent agents In position The static risk potential energy of the environment at that time. Specifically, the first part of the formula simulates dynamic interactive risks, treating neighboring agents (such as moving machinery) as dynamic risk sources. Their degree of danger (danger load coefficient) is proportional to their physical mass, speed, or danger level, while their impact on the current agent decreases inversely with the square of the Euclidean distance. This means that the closer the distance and the higher the danger level of the other party, the greater the repulsive force (risk potential energy) experienced by the current agent, thus simulating a Coulomb force-like obstacle avoidance mechanism. Furthermore, the above implementation also introduces environmental static risk potential energy to characterize fixed dangerous areas in the construction site, such as edge openings, high-voltage line areas, or flammable material storage points. This static element is assigned a fixed potential energy value and superimposed on the total potential energy. Through this dynamic-static combined calculation method, a continuous and differentiable risk field value is generated for each agent, thereby reflecting the immediate collision threat and the dangerous attributes of the environment itself.

[0055] Furthermore, step S3 specifically includes the following sub-steps:

[0056] S301. Input the dynamic spatiotemporal interaction graph into the multi-head graph attention layer and calculate the attention coefficient between the central node and its neighboring nodes;

[0057] S302. Based on the attention coefficient, the feature vectors of neighboring nodes are weighted and aggregated to generate a neighborhood feature embedding that integrates local game topology information;

[0058] S303. The neighborhood features are embedded into the agent's own observation state and spliced ​​together to construct a joint state representation for reinforcement learning, and the value of the game utility function is used as the immediate reward value for reinforcement learning.

[0059] Specifically, the above implementation leverages the powerful feature extraction capabilities of graph attention networks to address the problems of complex interactions and uneven weight distribution among multiple agents. When processing dynamic spatiotemporal interaction graphs, the system first performs pairwise analysis of the central node (the current decision-making agent) and its neighboring nodes using a multi-head attention mechanism. This mechanism calculates attention coefficients to dynamically measure the importance of different neighbors to the current node's decision. For example, for a tower crane undergoing hoisting operations, the influence of a nearby signalman on its decision should be far greater than that of a distant transport vehicle; the attention coefficient is used to quantify this difference. Based on the calculated coefficients, the feature vectors of all neighboring nodes are weighted and summed. This process is equivalent to information filtering and fusion, compressing and encoding the game dynamics of the surrounding environment into the feature representation of the current node, generating a neighborhood feature embedding that includes local topological information. Furthermore, the embedded vector is concatenated with the agent's own observed state (such as its own velocity and position) to form a complete joint state representation. This joint state not only contains individual information but also incorporates weighted environmental interaction information, which serves as the input to the reinforcement learning policy network. This enables the agent to make decisions based on its perception of the global game environment. At the same time, the utility function value calculated above is directly used as the reward signal to drive the parameter optimization of the policy network.

[0060] Furthermore, in step S303, the generated neighborhood feature embedding vector, which integrates environmental topology information and neighbor game situation, is dimensionally concatenated with the agent's own private observation data (its own velocity and position coordinates) to construct a local observation state vector for each agent in a partially observable environment. This vector can represent the agent's subjective perception of the game situation at the current moment. Then, the local observation state vectors of all agents are aggregated at the centralized level to form a global joint state space. At the same time, the system directly assigns the game utility function value, which includes positive rewards for task progress and negative penalties for security risks, calculated in the previous steps, as the immediate reward signal in reinforcement learning. Thus, a partially observable Markov decision process tuple with local observation, joint state, joint action, and immediate reward as its core elements is established.

[0061] Furthermore, step S4 specifically includes the following sub-steps:

[0062] S401. Construct a policy network and a value network based on a centralized training and distributed execution architecture;

[0063] S402. Introduce Lagrange multipliers to transform the hard constraints based on the security risk potential energy field into soft penalty terms in the dual problem, and combine them with the immediate reward value to construct a dual objective function that maximizes the expected return and minimizes constraint violations;

[0064] S403. The policy network parameters are updated by gradient descent using the primal-dual update algorithm, combined with Lagrange multipliers, and then by gradient ascent using the policy network parameters.

[0065] It should be noted that step S4 specifically employs a restricted multi-agent reinforcement learning algorithm based on a centralized training and distributed execution architecture. The policy network is responsible for generating joint action vectors based on the joint state representation output by the graph attention layer, while the value network is responsible for evaluating the global value of this joint state. The core mechanism involves constructing a dual objective function that includes the expected initial reward and the penalty for security constraints. This function uses the game utility function defined in the preceding steps as the basic reward signal. Simultaneously, Lagrange multipliers are introduced to transform the hard constraints based on the security risk potential field into dynamic penalty terms in the dual problem. During the specific training iterations, the original dual update algorithm is used for alternating optimization. On one hand, with the Lagrange multipliers fixed, gradient descent updates are performed on the policy network parameters to find the policy that maximizes the overall reward under the current constraint weights. On the other hand, gradient ascent updates are performed on the Lagrange multipliers to dynamically adjust the penalty intensity based on the actual degree of violation of the risk potential, thereby forcing the agent's policy to gradually converge to a Nash equilibrium point that satisfies the security hard constraints during training.

[0066] The reinforcement learning network architecture in this embodiment adopts a centralized training distributed execution (CTDE) mode, mainly composed of two core components: a distributed graph attention policy network and a centralized global value network. As an example, the policy network is constructed by cascading a front-end graph attention feature extraction layer, a mid-end multi-layer fully connected perceptron layer, and a back-end action mapping output layer. Specifically, the graph attention feature extraction layer receives dynamic spatiotemporal interaction graphs and outputs neighborhood feature embeddings; the multi-layer fully connected perceptron layer consists of several (e.g., 2-3) linear layers with ReLU activation functions stacked together, used to map the joint state representation after concatenating the neighborhood feature embeddings with the agent's private states into high-dimensional hidden layer features; the action mapping output layer, depending on the type of action space (continuous or discrete), uses Tanh or Softmax activation functions to transform the hidden layer features into specific joint action vectors.

[0067] The value network is constructed by cascading a front-end state-action fusion layer, a mid-end global feature extraction layer, and a back-end value regression layer. Specifically, the state-action fusion layer concatenates the joint state vector and joint action vector of all agents dimensionally; the global feature extraction layer, constructed from multiple fully connected layers, extracts the non-linear interaction features between states and actions; and the value regression layer, consisting of a linear layer without an activation function, outputs a scalar value representing the global expected return (Q-value) of the current joint policy.

[0068] Specifically, the above implementation adopts a centralized training distributed execution (CTDE) architecture, where the policy network is responsible for generating actions and the value network is responsible for evaluating state values. To handle safety constraints, the system introduces the Lagrange multiplier method, transforming the safety risk potential field constraint, which was originally a hard constraint, into a dynamic penalty term in the objective function, thus transforming the constrained optimization problem into an unconstrained dual problem. In specific training iterations, an alternating optimization algorithm is used: on the one hand, with the Lagrange multipliers fixed, gradient descent is used to update the policy network parameters, enabling it to find a policy that maximizes profit and minimizes violations under the current penalty level; on the other hand, with the policy network parameters fixed, gradient ascent is used to update the Lagrange multipliers. The logic is that if the current policy violates the safety constraints (i.e., the risk exceeds the limit), the multiplier value will automatically increase, thereby increasing the penalty weight for violations in the next update, forcing the policy network to converge to a safer region. This adversarial dynamic adjustment mechanism ensures that the finally trained policy can mathematically strictly satisfy the preset safety risk constraints.

[0069] Furthermore, the dual objective function is specifically expressed as:

[0070] ;

[0071] Among them, the Indicates the network parameters for the policy Minimize the sum of the parts of the Lagrange multiplier. The dual optimization process for maximization; Describe the dual objective function; the Indicates based on the current strategy The mathematical expectation; the Indicates the discount factor; the Indicates time The instant reward value; Indicates time The constrained cost function based on the potential energy value of security risks; This indicates the preset maximum allowed cumulative risk threshold; the This represents the total time. Specifically, the formula includes a standard reinforcement learning objective (i.e., maximizing the cumulative expected reward (including progress gains, etc.)) and a constraint term based on Lagrange multipliers (measuring whether the actual risk cost of the current policy exceeds a preset allowable threshold). In the Min-Max optimization game, the external maximization operation targets the Lagrange multipliers, aiming to find the most severe penalty coefficient, so that any small constraint violation will be amplified; the internal minimization operation (corresponding to the negative sign in the formula, actually maximizing the reward) targets the policy network, seeking the optimal action distribution that can simultaneously cope with high penalties and obtain task rewards. Through this mathematical construction, the formula guides the algorithm to continuously seek the Nash equilibrium point between aggressively pursuing efficiency and conservatively avoiding risk during training, ultimately obtaining a cooperative policy that is both efficient and compliant.

[0072] Furthermore, step S5 specifically includes the following sub-steps:

[0073] S501. Input the real-time data from the scene into the trained graph attention policy network, output the action probability distribution of each agent through forward propagation, and filter actions that violate physical constraints through a masking mechanism, and sample to generate a joint discrete action vector.

[0074] S502. Using a preset instruction decoding dictionary, the joint discrete action vector is parsed into construction rectification instructions, progress adjustment plans, and resource scheduling schemes described in natural language, and pushed to the corresponding decision-making body via a mobile terminal.

[0075] Specifically, the above implementation inputs real-time sensor data from the construction site into a trained and converged graph attention policy network. The network calculates through forward propagation, outputting the probability distribution of various actions each agent will take in the current state. To ensure the physical feasibility of the decision, the system introduces a masking mechanism, forcibly setting the probability of illegal actions that violate physical limitations (such as the robotic arm exceeding its maximum elevation angle or a vehicle driving in the wrong direction) to zero. Then, based on the corrected probability distribution, sampling is performed to generate a set of joint discrete action vectors. Further, through a pre-set standardized instruction decoding dictionary, the abstract discrete action indexes are mapped to specific, human-readable natural language instructions. For example, the action vector [0, 2, 5] is parsed as "Supervisor: Issue a Level 2 rectification notice," "Construction team: Reduce operating speed to 50%," and "Tower crane: Suspend slewing operation." These instructions are ultimately pushed with low latency to the control interfaces of the mobile terminals or devices of the relevant parties via the 5G network, realizing closed-loop control from data perception to intelligent decision-making to on-site execution.

[0076] Example 2

[0077] Furthermore, as a preferred embodiment of the above-described Example 1, a construction supervision decision-making system based on game optimization and multi-agent reinforcement learning is proposed. This system is implemented based on the construction supervision decision-making method based on game optimization and multi-agent reinforcement learning described in Example 1, and includes:

[0078] The digital twin environment construction module is used to define a set of decision-making intelligent agents, including construction intelligent agents, supervision intelligent agents and resource intelligent agents, based on BIM model data and on-site sensor data. It constructs a dynamic spatiotemporal interaction diagram based on the physical proximity and operational dependencies between intelligent agents, and initializes the intelligent agent state space and intelligent agent action space.

[0079] The master-slave security game model construction module is used to assign the supervisory agent the leader role according to the Stackelberg game logic, and assign the construction agent and resource agent the follower role. It defines the game utility function to quantify the operation benefits and risk costs, and sets the security risk potential energy field based on physical field theory as the hard constraint of the game.

[0080] The Markov game mapping module inputs the dynamic spatiotemporal interaction graph into the graph attention policy network, extracts neighborhood feature representations through the graph attention layer, and maps the master-slave game architecture into a partially observable Markov decision process, and transforms the game utility function into a reinforcement learning reward function.

[0081] The dual-optimized collaborative training module is used to place the set of decision agents into a digital twin environment for interaction, construct a dual objective function combining immediate reward value based on the security risk potential field, and alternately update the policy network parameters and Lagrange multipliers using a restricted multi-agent reinforcement learning algorithm until the security constraints are met.

[0082] The risk collaborative decision-making module is used to acquire real-time on-site data, process it through the trained graph attention strategy network, output joint action vectors, and decode and generate construction supervision collaborative decision-making instructions.

[0083] Example 3

[0084] Furthermore, as a preferred embodiment of the above embodiments, an application scenario for construction supervision decision-making based on game optimization and multi-agent reinforcement learning, as described in Embodiment 1, is proposed. This scenario involves a high-rise building construction site where tower crane rebar hoisting operations are underway. Specifically, it includes:

[0085] Decision-making body:

[0086] Resource agent (follower): Tower crane No. 1, responsible for hoisting steel bars.

[0087] Construction AI (Follower): The ground rebar tying team is working on the floor slab.

[0088] Supervision Intelligent Agent (Leader): On-site Intelligent Supervision System.

[0089] Its application process is as follows:

[0090] Tower crane No. 1 is currently lifting a 2.5-ton steel bar bundle at a rotation speed of 0.8 meters per second. Meanwhile, five workers from the ground-based steel reinforcement team are located within the swing radius of the tower crane's boom, and are only 8 meters horizontally away from the vertical projection point of the load. Based on this data, the system constructs a dynamic spatiotemporal interaction map, identifying a strong spatial proximity relationship between the tower crane and the ground team.

[0091] The system detected a distance of 8 meters between the tower crane (high-risk load) and the workers (vulnerable target), which is close to the preset safety warning distance of 10 meters. According to the logic of field physics, the "safety risk potential energy value" generated by the tower crane on the ground team rapidly increased, reaching 85 (assuming a full score of 100 and a safety threshold of 60). The supervising agent, as the leader, was the first to perceive that the potential energy value exceeded the safety threshold. To reduce the probability of violation, it decided to adjust its strategy, lowering the "real-time risk tolerance threshold" for the area from the default 0.8 to 0.5, and preparing to issue a "deceleration warning" command. Furthermore, in the internal deduction of the algorithm: the tower crane agent originally planned to maintain a speed of 0.8 m / s to save time (gain high progress benefits). However, it observed that the supervisor lowered the risk threshold, and the current potential energy value of 85 would result in a large "penalty value" (e.g., deducting 500 utility points). After calculation, if it continued to maintain a high speed, the total utility would be negative. The tower crane agent discovered that if the speed is reduced to 0.3 m / s, although the progress gain decreases, the risk potential value will decrease to 55 (below the threshold), thus avoiding huge penalties and maximizing the total utility.

[0092] Based on the above Nash equilibrium solution, the system decodes and outputs specific collaborative decision-making instructions:

[0093] To Tower Crane No. 1 (Resource Agent): "Personnel working below have been detected, posing a high risk. Please immediately reduce the slewing speed from 0.8 m / s to 0.3 m / s and maintain the current height."

[0094] To the ground team (construction AI): "There is hoisting work above. Please have the 5 workers currently working on the site suspend work and move 3 meters to the south side of the pit to a safe area."

[0095] To the supervision terminal (supervision intelligent agent): "A yellow risk warning has been triggered. The system has automatically issued a deceleration command. Please confirm the rectification status on-site."

[0096] Through the above implementation process, the system did not simply and rudely cut off the tower crane's power supply (leading to a work stoppage), nor did it allow the tower crane to operate in a risky manner (leading to an accident). Instead, it found an optimal solution of "reducing speed without stopping work and fine-tuning personnel to avoid obstacles" through game theory optimization, thus preserving the construction progress to the greatest extent possible while ensuring absolute safety.

[0097] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A construction supervision decision-making method based on game optimization and multi-agent reinforcement learning, characterized in that, Includes the following steps: S1. Based on BIM model data and on-site sensor data, define a set of decision-making intelligent agents including construction intelligent agents, supervision intelligent agents and resource intelligent agents. Construct a dynamic spatiotemporal interaction diagram based on the physical proximity and operational dependencies between intelligent agents, and initialize the state space and action space of intelligent agents. S2. Assign the supervisory agent the leader role according to the Stackelberg game logic, and assign the construction agent and resource agent the follower role. Define the game utility function to quantify the operation benefits and risk costs, and set the safety risk potential energy field based on physical field theory as the hard constraint of the game. S3. Input the dynamic spatiotemporal interaction graph into the graph attention policy network, extract neighborhood feature representations through the graph attention layer, map the Stackelberg game architecture into a partially observable Markov decision process, and transform the game utility function into a reinforcement learning reward function; S4. Place the set of decision-making agents into a digital twin environment for interaction, construct a dual objective function based on the security risk potential field, and use a restricted multi-agent reinforcement learning algorithm to alternately update the policy network parameters and Lagrange multipliers until the security constraints are met. S5. Acquire real-time field data, process it through the trained graph attention strategy network, output joint action vectors, and decode to generate construction supervision collaborative decision-making instructions; In step S2, the specific process of establishing a Stackelberg game architecture with the supervisory agent as the leader and other agents as followers includes the following sub-steps: S201. Define the supervisory agent as the leader in a Stackelberg game. The action space of the supervisory agent includes issuing safety rectification instructions and adjusting regional risk thresholds. S202. Define the construction agent and resource agent as followers in a Stackelberg game. The action space of the construction agent and resource agent includes adjusting construction speed and requesting resource bids. Followers make decisions after observing the leader's actions. S203. Calculate the safety risk penalty term using the safety risk potential energy field of physical field theory, and construct a game utility function that includes task progress benefit terms and safety risk penalty terms. In step S203, the game utility function is specifically expressed as follows: ； Among them, the Indicates the first An intelligent agent at time The game utility value; the The weighting coefficients representing the task progress objectives; This represents the progress-based reward function for a regular task, where For a moment The state of union, For a moment The combined action; the The weighting coefficients representing security risk objectives; Indicates the first An intelligent agent at time The potential energy value of safety risks; The specific security risk penalty items are as follows: ； Among them, the Indicates the agent at time t. The security risk penalty item, this value is directly substituted into the game utility function; the aforementioned This represents the base penalty coefficient, used to balance the order of magnitude of the penalty term and the task benefit term; the... This represents an exponential function, used to construct nonlinear high-gradient penalty boundaries; This represents the risk sensitivity factor; the higher the value, the more severe the increase in penalty after exceeding the threshold. This indicates a modified linear unit operation that ensures a penalty is only incurred when the potential energy value exceeds a threshold; the... This represents the preset safety potential energy threshold, corresponding to the safety warning distance in physical space; The specific calculation process for the aforementioned security risk potential energy value is as follows: ； Among them, the Indicates the first An intelligent agent at time The potential energy value of the safety risk, the Indicates index Belongs to intelligent agents The set of neighboring nodes The Represents the Coulomb risk propagation constant; the Indicates the first The dangerous load factor of each neighboring node; Indicates time intelligent agent With intelligent agents The Euclidean distance between them; Represents intelligent agents In position The static risk potential energy of the environment at that time; Step S4 specifically includes the following sub-steps: S401. Construct a policy network and a value network based on a centralized training and distributed execution architecture; S402. Introduce Lagrange multipliers to transform the hard constraints based on the security risk potential field into soft penalty terms in the dual problem, and construct a dual objective function that maximizes expected revenue and minimizes constraint violations; S403. The policy network parameters are updated by gradient descent using the primal-dual update algorithm, combined with Lagrange multipliers, and then by gradient ascent using the policy network parameters.

2. The construction supervision decision-making method based on game optimization and multi-agent reinforcement learning as described in claim 1, characterized in that, Step S1 specifically includes the following sub-steps: S101. Map each agent in the set of decision agents to a graph node, and encode the agent's own attributes into a node feature vector; S102. Calculate the Euclidean distance between any two intelligent agents. When the Euclidean distance is less than the preset communication radius or there is an upstream and downstream process dependency, establish a directed edge between the nodes to form an adjacency matrix. S103. Combine the node feature vectors with the adjacency matrix to form a dynamic spatiotemporal interaction graph at time t. , wherein Represents a set of nodes, the Represents the set of edges.

3. The construction supervision decision-making method based on game optimization and multi-agent reinforcement learning as described in claim 1, characterized in that, Step S3 specifically includes the following sub-steps: S301. Input the dynamic spatiotemporal interaction graph into the multi-head graph attention layer and calculate the attention coefficient between the central node and its neighboring nodes; S302. Based on the attention coefficient, the feature vectors of neighboring nodes are weighted and aggregated to generate a neighborhood feature embedding that integrates local game topology information; S303. The neighborhood feature embedding is concatenated with the agent's own observation state to construct a joint state representation for reinforcement learning, and the value of the game utility function is used as the immediate reward value for reinforcement learning.

4. The construction supervision decision-making method based on game optimization and multi-agent reinforcement learning as described in claim 1, characterized in that, The dual objective function is specifically expressed as: ； Among them, the Indicates the network parameters for the policy Minimize the sum of the parts of the Lagrange multiplier. The dual optimization process for maximization; Describe the dual objective function; the Indicates based on the current strategy The mathematical expectation; the Indicates the discount factor; the Indicates time The instant reward value; Indicates time The constrained cost function based on the potential energy value of security risks; This indicates the preset maximum allowed cumulative risk threshold; the Indicates the total time.

5. The construction supervision decision-making method based on game optimization and multi-agent reinforcement learning as described in claim 1, characterized in that, Step S5 specifically includes the following sub-steps: S501. Input the real-time on-site data into the trained graph attention policy network, output the action probability distribution of each agent through forward propagation, and filter actions that violate physical constraints through a masking mechanism, and sample to generate a joint discrete action vector; S502. Parse the joint discrete action vector into construction rectification instructions, progress adjustment plans and resource scheduling schemes described in natural language through a preset instruction decoding dictionary, and push them to the corresponding decision-making body through a mobile terminal.

6. A construction supervision decision-making system based on game optimization and multi-agent reinforcement learning, wherein the system is implemented based on the construction supervision decision-making method based on game optimization and multi-agent reinforcement learning as described in any one of claims 1-5, characterized in that, include: The digital twin environment construction module is used to define a set of decision-making intelligent agents, including construction intelligent agents, supervision intelligent agents and resource intelligent agents, based on BIM model data and on-site sensor data. It constructs a dynamic spatiotemporal interaction diagram based on the physical proximity and operational dependencies between intelligent agents, and initializes the intelligent agent state space and intelligent agent action space. The Stackelberg safety game model construction module is used to assign the supervisory agent the leader role according to the Stackelberg game logic, and assign the construction agent and resource agent the follower role. It defines the game utility function to quantify the operation benefits and risk costs, and sets the safety risk potential energy field based on physical field theory as the hard constraint of the game. The Markov game mapping module inputs the dynamic spatiotemporal interaction graph into the graph attention policy network, extracts neighborhood feature representations through the graph attention layer, and maps the Stackelberg game architecture into a partially observable Markov decision process, and transforms the game utility function into a reinforcement learning reward function. The dual-optimized collaborative training module is used to place the set of decision agents into the digital twin environment for interaction, construct a dual objective function based on the security risk potential field, and use a restricted multi-agent reinforcement learning algorithm to alternately update the policy network parameters and Lagrange multipliers until the security constraints are met. The risk collaborative decision-making module is used to acquire real-time on-site data, process it through the trained graph attention strategy network, output joint action vectors, and decode and generate construction supervision collaborative decision-making instructions.

Images