Automated architectural space combination method based on multi-agent deep reinforcement learning

Abstract

The present invention is applicable to the field of architectural design technology, and in particular to an automated architectural space combination method based on multi-agent deep reinforcement learning, the method comprising: constructing an agent according to the building renovation goal, and setting up an agent training environment; setting initial parameters of the agent and the agent training environment; the agent makes a decision based on the initial parameters, and dynamically simulates and diffuses the agent in the agent training environment according to the decision generated by the agent, and obtains the functional room layout result; scores are given according to the functional room layout result, and the decision is optimized based on the scoring result, and training is continued until the score of each agent reaches a preset value. The present invention realizes the movement and collaboration of the agent in the environment, thereby improving the flexible decision-making ability of the agent in the reinforcement training, and in this way, the trained agent can quickly and effectively complete the renovation design task.

Description

Technical Field

[0001] The present invention belongs to the technical field of architectural design, and in particular relates to an automated architectural space combination method based on multi-agent deep reinforcement learning. Background Art

[0002] Unlike newly built buildings, buildings to be renovated often have different current conditions and flexible renovation needs. How to implement low-carbon, environmentally friendly, sustainable renovation designs that meet new needs in a large number of building renovation practices is a common issue faced by architects.

[0003] Some existing automated spatial layout and automated generative design algorithms are mostly based on new building design. Their setting conditions are often relatively simple, the problems they can solve are relatively focused, and their adaptability and flexibility are relatively weak. Their applicability is reduced when faced with complex design problems. Summary of the invention

[0004] The purpose of the present invention is to provide an automated architectural space combination method based on multi-agent deep reinforcement learning, aiming to solve the problem that the existing technology is more focused, has weak adaptability and flexibility, and has reduced applicability when facing complex design problems.

[0005] The present invention is implemented as follows: an automated building space combination method based on multi-agent deep reinforcement learning, the method comprising: constructing an agent according to a building renovation goal, and setting up an agent training environment; setting initial parameters of the agent and the agent training environment; the agent makes a decision based on the initial parameters, and dynamically simulates and diffuses the agent in the agent training environment according to the decision made by the agent to obtain a functional room layout result; scoring is performed according to the functional room layout result, and decision optimization is performed based on the scoring result, and training continues until the score of each agent reaches a preset value.

[0006] Preferably, each agent comprises a set of main actor networks, a set of target actor networks, a main critic network and a set of target critic networks, wherein the main actor network is used to determine the best action under the current environment state, the target actor network is used to calculate the action of the future state, and the main critic network and a set of target critic networks are used to evaluate the expected return under a specific state and action combination.

[0007] Preferably, in the step of the agent making decisions based on the initial parameters, the number of agents and the maximum number of steps for each scenario are specified, the forces that each agent is allowed to be acted on in four orthogonal directions on the plane are defined as the action space, and the space containing the agent's own position, the position of the rectangular corner points of the functional room formed by itself, the position information relative to the positions of other agents and the corner points of the functional rooms, and the position information relative to immovable objects is defined as the observation space. All position information is standardized, and the agent selects the next action and the plane force of each agent according to the initial parameters through the strategy it has learned, and generates a corresponding decision.

[0008] Preferably, in the step of performing dynamic simulation and diffusion on the intelligent agent in the intelligent agent training environment according to the decision generated by the intelligent agent, initial state list information is received, and the initial state list information at least includes the initial position of the intelligent agent, force information and a flag of whether to reset the dynamic simulation, and data preprocessing is performed on the initial state list information. The initial state list information is data-separated by data preprocessing, and the position information contained therein is mapped to the actual size, and the diffusion rate parameter in the initial state list information is read. The intelligent agent generates circles that can collide with each other according to its position coordinates and diffusion rate parameters. The movement of the circle is limited to the area to be designed, and the surroundings are set as rigid bodies. The circle will bounce back after collision. The force on the circle comes from the force vector of each intelligent agent. The force point is located at the center of the circle. The circle moves under force and collides with each other. The diffusion mode is calculated according to the position information after movement and collision, and each ring is diffused into a functional room.

[0009] Preferably, the step of calculating the diffusion pattern based on the position information after movement and collision specifically includes: initialization: defining a set of seed points within the boundary and associating an initial square for each point; expansion process: each square expands uniformly from its center outward, and the expansion rate is determined by the assigned expansion speed. The expansion is limited by the boundary and the adjacent squares; speed mapping: the expansion speed of each square is mapped to the size of a specific attribute, allowing the distribution and influence of the attribute to be visualized; collision detection: in each iteration, checking the collision between the edges of the expanded square and the boundary and other squares. If a collision occurs, the affected side stops expanding, and the other sides continue to expand until they also stop due to the collision. This process is repeated until all sides of all squares stop expanding due to the collision; the expansion of the rectangle is controlled by the following equation:

[0010] S i (T+1) = S i (T)+r i ΔT

[0011] Where S i (T) is the time T is the size of square i, r iis the expansion rate of square i and ΔT is the time increment between iterations.

[0012] Preferably, the steps of scoring according to the functional room layout results, optimizing decisions based on the scoring results, and continuing training until the score of each agent reaches a preset value include calculating the diffusion pattern according to the position information after movement and collision, calculating the coordinates of each functional room corner point, and the generated functional room layout is input into the reward system for scoring, and the score of each agent is obtained to determine whether the completion mark has been reached.

[0013] Preferably, in the step where the generated functional room layout is input into the reward system for scoring, the coordinates of the functional rooms and the scores and task completion marks of each intelligent agent are merged into a list, and the current state and observation value are the coordinate values ​​of the corner points of each functional room calculated by the above data, the relative coordinate values ​​of each corner point, and the relative coordinate value of each corner point relative to the column, and the relative coordinate value is calculated by the difference between its own coordinates and the relative coordinates.

[0014] Preferably, the reward system is composed of a group of logic algorithm modules and a group of MLP neural networks.

[0015] The reward system is scored based on a preset reward calculation formula, which includes:

[0016] F(d p )=-α·(d p +0.5l)

[0017] Among them, F(d p ) is the reward formula for calculating the functional room that needs to be close to, which is used to calculate the reward value when a room needs to be close to a specific point, d p represents the distance from the room to the target point, l is the length of the long side of the entire transformation area, and α is the weight coefficient;

[0018] G(d ij )=-β·(d ij +0.5l)

[0019] Among them, G(d ij ) is used to calculate the rooms with the same function that need to be close to each other, d ij is the distance calculated between two rooms with the same function, and β is the weight coefficient;

[0020] T(d)=θ·(d-0.5l)

[0021] Where T(d) is the formula for calculating the distance between rooms, d is the distance between rooms, and θ is the weight coefficient;

[0022] H(d ij)=γ·(d ij -0.5l)

[0023] Among them, H(d ij ) is to calculate the same function rooms that need to be far away from each other, d ij is the distance calculated between two rooms with the same function, and γ is the weight coefficient;

[0024] E(L)=u·Lv·(1-L)

[0025] Among them, E(L) is used to score the lighting of the room. L represents the degree to which one side of the room is close to the building outline. L = 1 means that at least one side of the room is close to the building outline, and L = 0 means that the room is surrounded by other rooms on all four sides. u and v are positive weight coefficients;

[0026]

[0027] t=d c / d max

[0028] Among them, D(d c ) is used to calculate the reward based on the positional relationship between the corner point of the functional room and the original column, and B(t) is used to convert the distance d from a point to the nearest column c Mapped to a Bezier curve, the score is determined based on the value of the curve, and t is based on the distance d c The normalized Bezier curve parameters are d c / d max , mapped to the interval [0,1] by normalization, d max is the column spacing, n is the order of the curve, for a cubic Bezier curve, n = 3, C(n, i) is the number of combinations, which represents the number of ways to select i elements from n elements, P i is the ith control point, and δ is a positive weight coefficient.

[0029] Preferably, the number of agents is the same as the number of functional rooms.

[0030] Preferably, in the neural network reward model, 5 shadow layers are set, the number of iterations is 15, the training set is four groups of 20 functional room rectangles with length, width, area and corresponding function labels close to the transformation target, and all are standardized. The test set is a spatial combination generated by reinforcement learning, and the neural network outputs the label of each corresponding function. When the label is the same as the expected function label, the shape and area of ​​the functional room meet expectations, and a positive reward is given.

[0031] The automated building space combination method based on multi-agent deep reinforcement learning provided by the present invention is based on the framework of multi-agent deep reinforcement learning and copes with complex space combination problems in architectural design, especially in the scenario where functional updates and space optimization need to be achieved under existing building conditions. This method uses intelligent agents to control the movement of building functional modules, uses the existing status quo of immovable buildings as the environment, and realizes the movement and collaboration of intelligent agents in the environment through a series of reward mechanisms, thereby improving the flexible decision-making ability of intelligent agents in intensive training. In this way, the trained intelligent agents can quickly and effectively complete renovation design tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of an automated building space combination method based on multi-agent deep reinforcement learning provided by an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of a Markov decision process for a building renovation problem provided by an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of a code for building a Grasshopper reinforcement learning environment and implementing a data interaction process through UDP provided in an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of a multi-agent reinforcement learning workflow provided by an embodiment of the present invention;

[0036] Figure 5 A schematic diagram of a code for a diffusion algorithm provided by an embodiment of the present invention;

[0037] Figure 6 A schematic diagram of calculating the reward value of the relationship between the corner of a room and a pillar provided by an embodiment of the present invention;

[0038] Figure 7 A schematic diagram of the change in average rewards of multiple agents during the training process provided by an embodiment of the present invention;

[0039] Figure 8 A visualization view of the reward score of the agent after training provided by an embodiment of the present invention;

[0040] Fig. 9 Some results provided by the embodiments of the present invention are illustrated;

[0041] Fig.10 A schematic diagram of real-time interactive collaboration of intelligent agents provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] like Figure 1As shown, it is a flow chart of an automated building space combination method based on multi-agent deep reinforcement learning provided by an embodiment of the present invention, the method comprising: constructing an agent according to a building renovation goal, and setting up an agent training environment; setting initial parameters of the agent and the agent training environment; the agent makes a decision based on the initial parameters, and dynamically simulates and diffuses the agent in the agent training environment according to the decision made by the agent to obtain a functional room layout result; scoring is performed according to the functional room layout result, and decision optimization is performed based on the scoring result, and training is continued until the score of each agent reaches a preset value.

[0043] In this embodiment, the present invention takes the transformation of a frame structure building as an experimental case. The original building is a small industrial plant located in the mid-latitude region of the northern hemisphere. The goal of the transformation is to transform it into an office building without changing the column grid system. The original building's structural system is a reinforced concrete frame structure, with a column grid spacing of 8000mm. It faces south, has a 4-span column grid, a 2-span column grid depth, and a building net height of 8m. There are roads and squares on the south side of the building, and there are some trees on the square. With the process of urbanization, the original production function of the building has been moved to the suburbs of the city, and the buildings on the site around the existing building have been gradually transformed into urban creative office areas. The present invention takes the spatial layout of the first floor of the original building after transformation as the goal of automatic generation.

[0044] Fuzzy control is a control method based on fuzzy logic, which is developed from the fuzzy set theory proposed by LAZadeh in 1965. Fuzzy control does not rely on the precise mathematical model of the system, but designs the controller based on human experience and intuition. It handles uncertainty and ambiguity through fuzzy logic, enabling the control system to work without precise models or in highly complex system environments. The spatial layout of a building is not just a jigsaw puzzle. The architect's preliminary layout process of the functional rooms of the building can be regarded as a process similar to fuzzy control, although it does not necessarily directly adopt the technical framework of fuzzy control. In this process, architects make decisions and layout plans based on their understanding and experience of spatial functional requirements, user behavior, environmental conditions, etc. This decision-making process involves a lot of uncertainty and ambiguity, and requires the processing and weighing of multiple complex factors and constraints. Indeed, arranging the rooms of a building like a puzzle is not a simple matter. Figure 1 Too precise determination and placement on the site will limit the creativity and flexibility of the design. Excellent architectural design often requires exploring innovations in space, form, materials, etc. on the basis of meeting functional requirements and site conditions. This design process involves constant trial, evaluation and adjustment to find the best balance. Therefore, whether it is humans or intelligent entities, the flexibility of architectural room layout is very important when arranging spaces.

[0045] In order to prevent multi-agent reinforcement learning from falling into a state of local precise solution and being unable to control the global situation at the beginning of training, the present invention calculates a parameter "expansion rate" that can be used to replace the specific precise size in training based on the expected aspect ratio of the functional rooms of the building. When arranging the rooms, the present invention regards each agent as a function, and demarcates the circular functional area with the diffusion rate as the radius. Multiple circles representing different functions move and collide in the venue, and a rectangular room is generated through the diffusion algorithm introduced later. The scoring system will score the generated room shape with reference to the shape of the expected functional room, thereby achieving a transition from fuzzy to precise. As shown in the following table, the table shows the functional requirements for renovating the first floor of a building drafted by the present invention before multi-agent training.

[0046]

[0047] Table 1 Functional requirements for the first floor of the renovated building

[0048]

[0049]

[0050] Markov decision process for building renovation problem: Markov process is the core and basic framework of reinforcement learning theory. This invention verifies that the building renovation problem conforms to the properties of Markov process and transforms the building renovation problem into Markov process, thus laying the foundation for building and implementing multi-agent reinforcement learning from the perspective of building renovation. A basic Markov process in reinforcement learning is Markov decision process (MDP). MDP provides a mathematical framework for reinforcement learning, which is used to describe the state of the environment, the actions that the agent can take in each state, the probability of state transition after the action, and the immediate reward obtained after taking a specific action. MDP consists of four main elements: state set (S), action set (A), state transition probability function (P), and reward function (R). This framework allows the agent to learn strategies through interaction with the environment to maximize long-term cumulative rewards.

[0051] like Figure 2As shown, in the present invention, each state can represent a specific architectural space combination layout, including the position, size and relationship of different functional blocks, as well as their relationship with existing immovable structures (such as column grids). Actions represent decisions to transfer from the current state to another state, such as adjusting the position or size of a functional block. The reward function is defined based on the design task, reflecting the contribution of the spatial combination layout to meeting the design requirements and optimization goals after transferring from one state to another. Each decision step aims to optimize this reward through the behavior of the agent, thereby driving the entire design towards the target optimization direction. The present invention uses 9 agents to represent all functional rooms on one floor of the renovated building. During the initialization process, each agent is a circle with a radius equal to the diffusion rate, which is randomly scattered within the scope of one floor of the building to be renovated. At the beginning of training, the agent starts a round. The agent starts to move the circles representing the functional rooms according to the initial random state. The position of the circle changes, and the position and shape of the rectangular room generated by the diffusion of the circle change. After that, the reward system obtains the current state and scores it, and outputs feedback to the agent. In this process, the space composed of column grids and boundaries where agents move, collide and interact is used as the environment for reinforcement learning. The feedback output actions of agents interacting with the environment are used as actions. The score of the current state by the reward system is used as a reward. The reward is generated by the environment and fed back to the agent to perform a new round of actions. The above constitutes a basic Markov decision process for multi-agent reinforcement learning applied to building renovation cases.

[0052] Tool platform construction and reinforcement learning environment construction: The present invention establishes a communication mechanism between Visual Studio Code (VSCode) and Grasshopper through a UDP communication method to achieve a scalable, easily reproducible and user-friendly multi-agent deep reinforcement learning framework. This framework relies on the interaction between the powerful programming capabilities of VSCode and the highly visualized parametric design tools of Grasshopper. The main advantages of this integration strategy are reflected in the following aspects: First, as a widely used code editor, VSCode provides an efficient and flexible programming environment for the development and testing of reinforcement learning algorithms, allowing designers and developers to quickly implement and iterate algorithm models. Secondly, Grasshopper's intuitive parametric modeling interface provides designers with a platform for interacting with the algorithm, so that the design scheme generated by the algorithm can be instantly visualized, enhancing the interactivity and experimental nature of the design. Through UDP communication, dynamic data exchange between algorithm logic and architectural design parameters is realized. This method reduces the technical threshold of reinforcement learning in architectural design applications to a certain extent. The scalability of this framework is reflected in the use of Grasshopper as an intelligent agent training environment. Grasshopper has a wide range of plug-ins, such as acoustic, optical and thermal simulation, which further expands the dimension of design parameters and allows the algorithm to learn and optimize in a more complex and changing environment.

[0053] On the VSCode platform, the Python libraries that need to be imported based on Python 3.8 mainly include: parl == 2.2.1, paddlepaddle == 2.5.1, pandas == 2.1.3, numpy == 1.26.0, gym == 0.26.2, socket, threading. The environment interface code developed by the present invention first defines a GrasshopperEnv class, which inherits from gym.Env and standardizes the interface of the reinforcement learning environment. By specifying the number of agents and the maximum number of steps for each scenario, a basic setting is provided for simulating architectural space combinations. The action space is defined as the forces (Fx1, Fx2, Fy1, Fy2) that can be applied to each agent in four orthogonal directions of the plane. The magnitude of the force in a single direction ranges from 0 to 0.05, and the observation space includes the agent's own position, the position of the corner points of the functional room rectangle formed by itself, and the position information relative to the positions of other agents and the corner points of the functional room, as well as relative to immovable objects (such as columns). The position information is standardized and scaled to -1 to 1. The calculation of the resultant force and the normalized position information from the four orthogonal forces are all intended to help train the neural network. At the beginning of each training cycle, the environment is initialized and the agent is assigned an initial position. Subsequently, the environment state and the initial observation data of the agent are sent from the Grasshopper environment to the reinforcement learning algorithm through the established UDP connection. This step is completed by listening to a specific port and parsing the received data packets, ensuring that the agent can obtain accurate information about the current environment. Based on the received state information, the agent selects the next action and the plane force of each agent through its learned strategy, and sends these actions back to the Grasshopper environment via UDP for execution. This sending process also involves packaging the action data and sending it to the specified port over the network. After receiving these action data, the Grasshopper environment will update the state of the building model according to the action, and calculate the corresponding reward value and the new environment state, as well as a flag to indicate whether the current round (episode) has ended. The updated environment state and reward value are sent back to the reinforcement learning algorithm again via UDP, and the algorithm updates the learning model of the agent based on this new information. In this way, the agent continuously receives the state of the environment, makes decisions, performs actions, and learns based on the results, gradually optimizing its decision-making strategy to achieve the purpose of combining in the architectural space under given conditions.

[0054] Multi-agent reinforcement learning workflow: Figure 4As shown, each agent in the present invention has four sets of Multi-Layer Perception (MLP) neural networks, including two sets of actor networks and two sets of critic networks, to process continuous action space and optimize the interaction between multiple agents. These networks play different roles in the execution of strategies and training processes, and work together to improve the decision-making quality and efficiency of the overall system. When executing strategies, the agent uses its main actor network to determine the best action under the current environmental state, such as how the circle representing the functional room is moved by Kangaroo dynamics simulation in the Grasshopper environment to achieve the optimal functional relationship and streamline as well as the relationship with the existing immovable building structure. Kangaroo dynamics simulation performs collision detection and response, and the agent can adjust its position according to physical feedback to prevent the functional rooms from nesting and overlapping with each other, and then expand the circular space into an actual rectangular room layout through a diffusion mechanism. The circles shown in the figure are proportionally enlarged for easy display. The size of the circle in the actual training is 0.8 times the display size, and it is elastic for easy movement.

[0055] During the training process, in addition to the main actor network used for decision-making, each agent also has a target actor network, which is used to calculate the actions of future states and stabilize the training process together with the target critic network, while the two sets of critic networks evaluate the expected rewards under specific state and action combinations. The main critic network directly participates in the value evaluation of the current strategy, while the target critic network receives the next state and the action from the target actor, providing a stable estimate of future rewards, helping to reduce the variance in the training process and improve the stability of learning. In the update of the target network, the present invention adopts a soft update strategy to prevent the target value from changing drastically and becoming unstable. The present invention stores the experience data containing the state, action, reward and next state (st, at, r, st+1) in the experience replay pool, and learns by batch learning. This learning method helps to break the time correlation, improve data utilization and alleviate the non-stationary distribution of data.

[0056] The MADDPG algorithm adopts a centralized training and distributed execution strategy, that is, in the training phase, the actions and state information of all agents are processed centrally to fully consider the dynamic interactions in the multi-agent environment, while in the execution phase, each agent makes independent decisions based on its own observations and learned strategies. This design not only allows agents to learn how to cooperate and compete effectively in a complex multi-agent environment, but also ensures that they can flexibly adapt to environmental changes during execution, ultimately achieving the automated layout of functional rooms. Through this complex but efficient neural network structure and training execution mechanism, MADDPG can provide strong support and optimization capabilities in the field of architectural design automation, especially in the specific task of functional room layout under limited conditions.

[0057] In the Grasshopper platform, the plug-ins required by the present invention mainly include LunchBox for GrasshopperV2, GH_CPython v0.1-alpha, Kangaroo 2.42. The present invention includes a UDP receiving module, a data preprocessing module, a CSV input module, an initialization condition setting module, a dynamic simulation and diffusion module, an agent behavior scoring module, and a UDP sending module in Grasshopper, wherein the agent behavior scoring module is described in detail later. During the reinforcement learning training process, the UDP receiving module receives a list of 55 pieces of information sent from VSCode, which includes the initial positions, forces, and a flag for resetting the Kangaroo dynamic simulation of 9 agents. The initial positions of the 9 agents are composed of 18 bits of standardized x and y coordinates, the forces are 36 bits of forces in four orthogonal directions of the 9 agents, and the last flag is a Boolean value, which is used to reset the Kangaroo dynamic simulation in a new round. Data preprocessing mainly divides the above received data and remaps the position coordinates to the actual size. The CSV input module reads the functional requirements table of the first floor of the renovated building and inputs the corresponding diffusion rate to the dynamic simulation and diffusion module. The diffusion rate parameters are input into the diffusion algorithm after being standardized. In the dynamic simulation and diffusion module, the agents are set as circles that can collide with each other according to their position coordinates and diffusion rate parameters. The movement of the circles is limited to the area to be designed, and the surroundings are set as rigid bodies. The circles will rebound after collision. The force on the circles comes from the pre-processed force vectors of each agent, and the force point is located at the center of the circle. The circles move under the force and collide with each other. The position information after movement and collision is input into the diffusion algorithm.

[0058] The present invention introduces an algorithm for calculating the diffusion pattern of a square analogous to the Voronoi tessellation principle, which is similar to the formation of Thiessen polygons in natural phenomena. Thiessen polygons (also called Voronoi diagrams or Dirichlet tessellation) are a division on a plane, which divides the plane into a number of polygonal regions, each of which is determined by a generating point (seed point). These polygonal regions have an important characteristic: the distance from any point in the region to the generating point in the region is less than the distance to any other generating point on the plane. The algorithm simulates the growth of squares starting from a seed point within a defined boundary, where the diffusion speed is similar to the growth of bubbles, and each square forms rectangles of different sizes and shapes due to collision detection during the diffusion process. The size of the rectangle is related to the diffusion speed and distance of the surrounding diffused points, and the size and shape themselves represent the size of a certain attribute in the system (such as influence or resource allocation). The algorithm runs in an iterative manner and includes the following steps:

[0059] like Figure 5 As shown, Initialization: A set of seed points are defined within the boundary and an initial square is associated with each point. These squares are conceptualized as the influence area of ​​the corresponding point. Expansion process: Each square expands uniformly outward from its center, and the expansion rate is determined by the assigned expansion speed. The expansion is limited by the boundary and the presence of neighboring squares, similar to the growth of Thiessen polygons due to neighboring influences. Speed ​​mapping: The expansion speed of each square is mapped to the size of a specific attribute, allowing visualization of the distribution and influence of that attribute. Larger attributes lead to faster expansion, simulating the supercharged effect of larger bubbles in foam. Collision detection: At each iteration, the algorithm checks for collisions between the edges of the expanding squares and the boundary and other squares. Once a collision occurs, the affected edge stops expanding, while other edges continue to expand until they are also stopped by collision. This process is repeated until all edges of all squares stop expanding due to collisions. The expansion of the rectangle is governed by the following equation:

[0060] S i (T+1) = S i (T)+r i ΔT;

[0061] Where S i (T) is the time T is the size of square i, r iis the expansion rate of square i, and ΔT is the time increment between iterations. The algorithm terminates when all rectangles reach a state where they cannot be expanded further without collision, and the result is a stable configuration similar to a Voronoi diagram composed of rectangles rather than irregular polygons. The algorithm is innovative in theory and has also shown its unique value in practical applications, especially in the field of architectural planning and design. The algorithm can be used to simulate and optimize the functional layout of internal spaces in buildings, especially in terms of room size and purpose allocation. In this way, each square not only represents a physical space, but also reflects the functional importance or expected intensity of use of the space. For example, when designing an office building, the algorithm can be used to determine the optimal size and layout of different functional areas such as offices, meeting rooms, and rest areas. The diffusion speed of the rectangles can be adjusted according to the expected frequency of use or traffic of each room. Faster diffusion squares represent spaces with high frequency of use or high importance, such as major meeting rooms or core office areas, while slower diffusion squares may be auxiliary spaces, such as storage rooms or less used facilities. In addition, the algorithm shows particular utility in dealing with space utilization and building efficiency. By adjusting diffusion parameters, designers can maximize the efficiency of space while maintaining functionality and comfort. For example, in apartment building design, the ratio of public space to private residential units can be optimized through algorithms to ensure that both the private space needs of residents are met and the public areas are fully utilized.

[0062] The circle representing the functional room is diffused into a rectangular room through a diffusion algorithm, so that the coordinates of the corner points of each functional room are calculated in Grasshopper. The generated functional room layout is input into the reward system for scoring, and the score of each agent is obtained, as well as the sign of whether the completion is reached. In the present invention, the sign of the completion of the design task is that the score reaches the set higher score line. The coordinates of the functional room and the scores and task completion marks of each agent are merged into a list and sent back to the VSCode end via UDP for processing. The current state and observation values ​​are the coordinate values ​​of the corner points of each functional room calculated by the above data, the relative coordinate values ​​of each corner point, and the relative coordinate values ​​of each corner point relative to the column. The relative coordinate value is calculated by the difference between its own coordinates and the relative coordinates. In VSCode, the main action network of each agent outputs the next action after obtaining the current state and observation, so that the entire reinforcement learning enters the next step. The current series of reinforcement learning data (st, at, r, st+1) will be stored in the experience pool, and batch learning will be performed when the number of steps reaches a certain number.

[0063] Reward system: In reinforcement learning, rewarding and punishing the results of the interaction between the agent and the environment is the core of guiding the agent to learn and achieve its goals. The reward system evaluates the results obtained by the agent's actions, thereby guiding the agent to learn how to maximize the long-term cumulative rewards through a series of decisions in a given environment. This process not only significantly affects the learning speed and efficiency of the agent, but also plays an important role in whether the agent can accurately understand and adapt to environmental changes, discover and utilize the reward rules in the environment. Due to the complexity of the building renovation task, the reward system of the present invention is deployed in Grasshopper, which consists of two main modules: logic algorithm and MLP neural network. The logic algorithm mainly scores the position relationship and orientation of the functional rooms of the building, and the MLP neural network scores the shape conformity of the functional rooms of the building. The present invention provides continuous rewards for the agent through the above two scoring modules, that is, a reward signal is provided after each step of behavior. Providing continuous rewards to the agent can effectively improve the learning efficiency and enable the agent to maintain consistency and goal orientation in the complex decision-making process.

[0064] Table 2 Reward calculation formula based on logical algorithm

[0065]

[0066] In the formulas listed in the table, F(d p ) is the reward formula for calculating the functional room that needs to be close. This formula is used to calculate the reward value when a room needs to be close to a specific point. Among them, d p Indicates the distance from the room to the target point, l is the length of the long side of the entire renovation area, and α is the weight coefficient. In the present invention, it is set that the lobby is close to the external crowd gathering point, one of the two stairwells is close to the lobby, the bathroom is close to the other stairwell, one of the offices is close to the lobby, and the conference room is close to the lobby. The 0.5l in the formula is to make the scoring transition from negative penalty to positive reward. G(d ij ) is used to calculate the rooms with the same function that need to be close to each other, d ij The distance between two rooms with the same function is calculated, and β is the weight coefficient. The four offices in this invention use this formula to calculate the reward. T(d) is the formula for calculating the distance between each other, d is the distance between rooms, and θ is the weight coefficient. In this invention, the toilet is set to be far away from the hall. H(d ij ) is to calculate the same function rooms that need to be far away from each other, d ijThe distance calculated for the same functional rooms is two by two, and γ is the weight coefficient. The two stairwells in the present invention are set to be far away from each other to ensure the maximum use efficiency of the stairs. The E(L) formula adds points for one side of the room close to the building outline (the daylighting side), and deducts points for the room surrounded by other rooms on all four sides (the non-daylighting side). L represents the degree to which one side of the room is close to the building outline. L=1 means that at least one side of the room is close to the building outline, and L=0 means that the room is surrounded by other rooms on all four sides. u and v are positive weight coefficients, which respectively represent the added points for being close to the building outline and the deduction points for being surrounded by other rooms on all four sides. When at least one side of the room is close to the building outline, L=1, and the function E(L) will give a positive score; when the room is surrounded by other rooms on all four sides, L=0, and the function E(L) will give a negative score. In the present invention, in order to avoid the projection of the conference room being affected by direct light, the conference room is set to actively approach the north side.

[0067] like Figure 6 As shown, D(d c ) is the position relationship between the corner point of the functional room and the original column. The formula for calculating its reward is as shown in the figure below. Figure 4 The black squares at the corner points are columns, and the cross section of the columns is a square with a side length of 800mm. Since the center of the cross section of the column is located on the axis, a quarter of the corner of the column cross section is shown in the figure. In the figure, red represents the area with added points, and blue represents the area with deducted points. The present invention uses a Bezier curve to evaluate the relative relationship between the corner points of the functional room and the columns. The square area shown in the figure is an area of ​​a unit column grid span taken from the renovated area, with a side length of 8 meters. B(t) maps the distance from a point to the nearest column to a Bezier curve, and determines the score based on the value of the curve. t is the Bezier curve parameter obtained after normalization based on the distance d, and the normalization equation is d c / d max , d max The column spacing is 8000mm, which is mapped to the interval [0,1] through normalization. n is the order of the curve. For a cubic Bezier curve, n = 3. C(n,i) is the number of combinations, which represents the number of ways to select i elements from n elements. P i is the i-th control point, and δ is a positive weight coefficient. Through this equation, the agent can be encouraged to bring the corners of the remodeled functional room closer to and align with the original column grid in reinforcement learning, and the agent that wraps the column in the middle of the functional room will be penalized by deducting points.

[0068] In the neural network reward module, the present invention uses the neural network in the Grasshopper plug-in Lunchbox, and constructs a simple MLP neural network in Grasshopper to score the shape and area of ​​the functional room and the shape and area of ​​the corresponding target function. The neural network sets 5 hidden layers and the number of iterations is 15. The training set is four groups of 20 functional room rectangles with length, width, area and corresponding function labels that are close to the transformation target, and all are standardized. The test set is a spatial combination generated by reinforcement learning, and the neural network outputs the label of each corresponding function. When the label is the same as the expected function label, and the shape and area of ​​the functional room meet expectations, a positive reward is given. On the contrary, when the label does not correspond, a negative penalty is given. The overall score is also multiplied by a positive weight coefficient to balance the overall reward system.

[0069] Multi-agent reinforcement learning training: The present invention configures a series of hyperparameters in detail to ensure that the algorithm can effectively adapt to the complexity and variability of architectural design. The learning rate (Learning Rate) of the Critic network and the Actor network is set to 0.01 respectively. This setting is intended to balance the stability and convergence speed of learning, ensure that the agent can effectively learn from environmental feedback, and avoid training instability caused by excessive learning rate. In the reinforcement learning framework, the learning rate is a key factor in adjusting the weight update amplitude. An appropriate learning rate can speed up the learning process without causing excessive policy fluctuations. Secondly, the reward discount factor (Gamma) is set to 0.95, which means that the agent will consider 95% of future rewards as the value of the current decision when making a decision. This setting helps the agent to fully consider long-term interests while pursuing immediate rewards, prompting it to find more comprehensive and lasting solutions in architectural design tasks. The soft update coefficient (Tau) is set to 0.01 to control the speed at which the target network parameters converge to the main network parameters. The soft update mechanism is one of the key technologies in the MADDPG algorithm. It improves the stability of the learning process by slowly updating the parameters of the target network, thereby avoiding large fluctuations during training. In addition, the batch size is set to 1024, which determines the number of experiences extracted from the experience replay cache for training at one time. A larger batch size can reduce sample noise and improve the accuracy of gradient estimation, but it also increases the computational burden. The maximum number of steps per training cycle (Max Step Per Episode) is set to 15. This parameter limits the maximum number of steps that the agent interacts with the environment in each training cycle to ensure the efficiency of the training process and prevent the agent from falling into an endless loop in some cases.

[0070]

[0071]

[0072] Results and discussion: Figure 7 After 43,400 rounds and a total of 651,000 steps of reinforcement learning training, the average total reward of multiple agents reached 300 points per round. The figure shows the trend of the average round reward and its variance for each agent during the training process. The figure calculates the average value every 200 rounds and trains it into a broken line after sliding average. The x-axis in the figure represents the number of training rounds in thousands, and the y-axis represents the average reward value. Each broken line corresponds to the change in the average reward of an agent, which also includes the sum of the average rewards of all agents. The shaded part represents the variation range of the reward value around its average value, showing the volatility or dispersion of the reward distribution. The area with a larger shaded width indicates that the reward obtained by the agent has a greater uncertainty, while the smaller shaded width indicates that the reward distribution is more concentrated. For the convenience of display, the shaded range of the broken line representing the total reward value is the variance value multiplied by 1 / 2, and the shaded range of the broken line representing the reward value of each agent is the variance value multiplied by 1 / 4, which means that for each agent, the range above and below the shaded area is the average reward value plus or minus one quarter of the standard deviation. Agent1 to Agent9 correspond to the functions "hall", "office1", "office2", "office3", "office4", "conference room", "stair1", "stair2", "washroom" respectively. It can be observed from the figure that as the training progresses, the average rewards of different agents change, reflecting the adaptability and efficiency of their learning strategies. Some agents may show faster learning progress and more stable performance, while others may experience more fluctuations. Overall, the global total reward value shows an intuitive upward fluctuation trend, which fully reflects the characteristics of MADDPG's distributed execution and centralized training, and reflects the multi-agent system's trade-offs on the individual scores of a few agents in order to improve the global score, as well as the transformation process of multi-agents from relative independence to mutual cooperation.

[0073] like Figure 8As shown in the figure, multiple agents show different scores and trends at each stage of training. The figure below shows the reward scores of 9 agents in the last 500 rounds of training. The numbers on the x-axis represent Agent1 to Agent9, and correspond to the functional rooms of the building mentioned above. The y-axis is the round reward score of each agent. The reward score data of each agent in the figure is composed of a jitter scatter plot, a line box plot, and a kernel density estimation curve. The horizontal line inside the box represents the median, and the upper and lower boundaries of the box represent the third quartile and the first quartile. The protrusion of the kernel density estimation curve represents a dense distribution of scatter points, and the concave represents a sparse distribution of scatter points. It can be seen from the figure that after training, Agent1, which controls the hall, and Agent6, which controls the conference, have higher scores, while Agent2-Agent5, which controls four offices, Agent7 and Agent8, which control two stairs, and Agent9, which controls the washroom, have lower scores than Agent1 and Agent6. This shows that the tasks of the lobby and conference room are relatively fixed and simple, both close to fixed points and fixed sides, while the tasks of the office, stairwell, and bathroom are relatively complex due to their dynamic nature. The scores of Agent5, who controls one of the offices, and Agent7, who controls one of the stairs, are higher among the same type of functions, which verifies that the pre-training setting encourages one of the offices and stairwells to be close to the lobby, while the lobby has a simpler task and a more uniform and fixed position during training, making it easier for Agent5 and Agent7 to get rewards compared to other agents with the same function.

[0074] like Fig. 9As shown, 5 solutions with a single-step score of more than 15 points in the late stage of training are shown, and each line is the same solution. From left to right, they are the layout diagram calculated by multi-agent reinforcement learning, the relationship diagram of each function, and the diagram after algorithm processing and deepening. The relative position and rectangular diffusion results of each agent can be seen in the first column of figures. The second column of figures shows the relationship between each function in the set reward system, where the functions connected by solid lines are the functional relationships of rewarding close to each other, and the functions connected by dotted lines are the functional relationships of rewarding far away from each other. The present invention found that the solutions with higher scores showed certain similarities. The large space conference room was arranged in the northwest corner, and the hall was arranged in the southwest corner where people gathered. Some solutions arranged the stairwell to the west of the hall so that the distance between the two stairwells became larger to increase the reward. The toilet and another stairwell were arranged in the northeast corner, and the location of the office was relatively concentrated. Overall, the results of multi-agent reinforcement learning have achieved the goals set by the reward system relatively well. The room layout of the agent effectively echoes the original column grid system. However, some solutions still have some places that do not look good from a design perspective. For example, in solution e, the bathroom is arranged in the south direction. In design, designers often arrange some important functions that require lighting in the south direction, and arrange the equipment room or bathroom in other directions. In this case, the office obviously needs to face south more than the bathroom. In addition, even if the reward system is set up to use neural networks to judge the shape of the room and score it, the room shape of some solutions is still not good. For example, in solution a in the figure below, the length-to-width ratio of the conference room obviously exceeds the normal use requirements, resulting in a waste of space.

[0075] The present invention uses Leafvein in the Grasshopper plug-in to calculate the shortest path so as to offset the functional room to one side to generate a traffic space. The present invention focuses on the spatial organization of multi-agent reinforcement learning in the given environment, so the calculation and generation of the traffic space are not discussed much. The third column of the figure is a plan view after further deepening the spatial layout model using a logical algorithm. Rhino.Inside and Revit are used in the deepening process. The deepening algorithm defines a series of window and door openings and their numbers and directions, generates walls through baselines, and places furniture blocks, etc. Some functions such as bathrooms are defined as two fixed layout modes according to different areas during the deepening, as shown in Figure c.

[0076] This invention innovatively introduces multi-agent reinforcement learning and multi-software collaborative interaction to complete the renovation task of existing buildings. In the process of exploration, many failures and experiences have been generated. These experiences can help other scholars reduce certain exploration work in similar research. The following summarizes some of the more important points to note in the construction and training of the reinforcement learning framework of this invention.

[0077] Trade-off between operability and training efficiency: The present invention builds a reinforcement learning training environment based on Rhino and Grasshopper through UDP communication, which makes multi-agent reinforcement learning more friendly to the field of architecture and more operable. In future research, a rich set of parametric plug-ins related to architectural design can be used to conduct more complex reinforcement learning. However, this approach abandons the efficiency of reinforcement learning training to a certain extent. A completely closed training environment will be more efficient in training, but it will also greatly increase the cost and difficulty of environmental development for various different construction tasks. The present invention uses the PARL reinforcement learning module, which has a simple and efficient parallel training framework. The present invention team will try to use supercomputing servers for parallel training in further exploration to improve training efficiency.

[0078] Battery operation characteristics and UDP transmission in Grasshopper: Combining reinforcement learning with Grasshopper to communicate via UDP presents a series of unique technical challenges. These challenges mainly stem from the execution mode of Grasshopper and the characteristics of the UDP protocol. Grasshopper is a graphical algorithm editor widely used in the design field, and its operations are usually based on direct user input or preset automation script triggers. However, when introducing reinforcement learning, an algorithm that requires continuous interaction with the environment, Grasshopper needs to be able to continuously receive new instructions from the algorithm and feedback the results, which is not completely consistent with the operation mechanism of Grasshopper itself. Because Grasshopper was not designed to consider the scenario of continuously listening to external instructions, in order to achieve this, some mechanism, such as a timer, must be used to periodically refresh the Grasshopper state so that it can process new data or commands. In this process, UDP, as a connectionless network protocol, can provide low-latency communication, but it does not guarantee the order or reliable arrival of data packets, which brings challenges to ensuring the integrity and accuracy of data transmission. Therefore, the instructions and feedback sent from VSCode (or any other running environment of reinforcement learning algorithms) to Grasshopper need to ensure that the algorithm can receive the correct information and make decisions accordingly. This requires efficient thread management and data synchronization mechanisms on the VSCode side to ensure that the previous instruction has been correctly processed and responded to before sending new data. At the same time, effective data buffering and error handling mechanisms need to be implemented on the Grasshopper side to deal with possible packet loss or disordered order. Since Grasshopper triggers execution through the Timer component, it actually "wait" for new data from the VSCode side in each Timer cycle. If Grasshopper does not receive any data within the predetermined listening cycle, it may not only cause the iterative process of the reinforcement learning algorithm to be interrupted, but more seriously, this timeout waiting state may cause the response of the Grasshopper application itself to freeze the software. Therefore, when there are no new instructions or data to be sent, the VSCode side should also maintain communication with Grasshopper by sending repeated signals that do not change variables, even if these data packets do not introduce any new instructions or variable changes. The core purpose of this method is to ensure that the listening loop on the Grasshopper side can continue.

[0079] Markov decision process and the curse of dimensionality: When applying reinforcement learning to spatial combination tasks, the present invention initially attempted to generate a list of locations for each room as an action step in reinforcement learning. This approach seems to be able to provide the greatest flexibility and freedom for room layout in theory, but in fact introduces several key problems, leading to the failure of the training process. First, the core of reinforcement learning is based on the Markov decision process (MDP), in which the transition of each state depends only on the current state and the action taken, and is independent of the previous state or action. This feature requires that each state contains enough information to determine the best action for the next step. However, when each step is to generate a list of locations for a room, the action itself may not contain enough information to guide how to place the next room. The correlation of each step in a round is weak, breaking the premise of MDP, making it difficult for the reinforcement learning algorithm to learn effective strategies from the sequence of actions. Therefore, the present invention adopts the method of moving the agent to strengthen the correlation between each step, so that the final state of each round is generated by the superposition of each step. Secondly, in previous studies, the experiment controlled two functional rooms by a single agent and achieved a certain training effect, but when the number of agents increased, the difficulty of training increased greatly. The present invention finds that when a single agent needs to represent all rooms and decide the position for each room, the action space will grow exponentially as the number of rooms increases. This is because the agent needs to consider all possible room position combinations at each step, and the number of these combinations is an exponential function of the number of rooms. This explosive growth of the action space leads to the so-called "curse of dimensionality", that is, as the scale of the problem increases, the required computing resources and training time increase exponentially, making training difficult. Therefore, the present invention introduces a multi-agent system, so that the action space of each agent is greatly reduced, and the overall layout can be optimized through a collaborative mechanism.

[0080] Introduction of dynamic model: Collision in dynamic system seems to be a method that takes up more computing resources. The circle collision introduced in the present invention increases the calculation speed of each round to a certain extent. However, in fact, in previous studies, whether it is a room layout with completely determined dimensions or a point movement and diffusion rectangle without collision, the effects of these methods in reinforcement learning training are not satisfactory. The room layout with determined dimensions always contains a large number of room overlaps and intersections in training, even if a large number of penalty measures are set. The method of moving points and diffusing rectangles without collision is prone to the possibility that the distance between points is too close, and the solution space is enlarged, which greatly increases the difficulty of training. Collision in dynamic system effectively solves the disadvantages of the above two modes. Although the amount of calculation in a single round is increased, it greatly reduces the difficulty of reinforcement learning training.

[0081] like Fig.10As shown, the present invention explores a method for spatial layout in building renovation scenarios that can give full play to the flexibility of multi-agent systems. The present invention also affirms the excellent performance of a series of classic logic algorithms such as KD tree, annealing algorithm, etc. in specific building layout tasks. The application of reinforcement learning in the present invention is also in the exploratory stage, and there are still many parts that need to be improved and expanded. The task of building renovation is not just a completely closed environment similar to a game. The final review and decision-making of the plan still cannot be separated from human collaboration. To this end, the present invention uses the "DotDisplay" in the Kangaroo plug-in in Grasshopper to achieve that while the agent generates the plan, people can change the plan by dragging the control point, and this control can be fuzzy control from one direction to another, and the intelligent body will make further changes according to the changes, thereby realizing real-time interactive collaboration between people and agents.

[0082] This invention proposes and implements an innovative method, which aims to address complex spatial combination problems in architectural design through a framework based on multi-agent deep reinforcement learning (MADDPG), especially in the face of scenarios where functional updates and spatial optimization need to be achieved under existing building conditions. This method uses intelligent agents to control the movement of building functional modules, uses the existing conditions of immovable buildings as the environment, and realizes the movement and collaboration of intelligent agents in the environment through a series of reward mechanisms, thereby improving the flexible decision-making ability of intelligent agents in intensive training. In this way, the trained intelligent agents can quickly and effectively complete renovation design tasks. The achievements and contributions of this invention are mainly reflected in the following aspects:

[0083] First, the present invention successfully established a method framework for automated spatial layout of building renovation based on multi-agent deep reinforcement learning. This framework not only provides a new research direction for the field of architectural design in theory, but also provides an effective technical means for solving practical problems in building renovation in practice, especially showing significant advantages in improving design efficiency and spatial layout quality.

[0084] Secondly, the present invention has developed a tool platform with good operability. The platform is mainly developed based on Vscode and Grasshopper. It has strong scalability and is easy to reproduce, laying the foundation for applying multi-agent reinforcement learning to more complex building environment simulation and optimization scenarios in the future.

[0085] Finally, by conducting reinforcement learning training in a specific building renovation case, the feasibility of the proposed method was verified and its effectiveness in practical applications was successfully demonstrated. This result not only confirms the practical value of the multi-agent reinforcement learning algorithm (MADDPG) in improving the efficiency and quality of building renovation design tasks, but also provides valuable experience and data support for subsequent research.

[0086] Although the present invention has achieved initial results, there are still many challenges to be solved in the field of building design automation. Future research can further explore how to apply multi-agent deep reinforcement learning technology in more complex building renovation scenarios and irregular plots, especially in the optimization of design constraint processing and human-computer interaction interface research, so as to further enhance the design flexibility and human-computer interaction design experience. The building functional modules are raised from two-dimensional planes to spatial blocks to achieve automatic generation of multiple floors at the same time. In addition, standardizing the observation space format of the agent in the construction task will facilitate the sharing of building agents. Using more efficient training methods and continuously developing and improving the tool platform to adapt to a wider range of application scenarios and functional requirements are also important directions for future research.

[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An automated architectural space combination method based on multi-agent deep reinforcement learning, characterized in that: The method comprises: Build an agent based on the building renovation goals and set up an agent training environment; Set the initial parameters of the agent and the agent training environment; The agent makes decisions based on the initial parameters, and the agent is dynamically simulated and diffused in the agent training environment according to the decision made by the agent to obtain the functional room layout result; Scoring is performed based on the functional room layout results, decision optimization is performed based on the scoring results, and training is continued until the score of each agent reaches the preset value; In the step where the agent makes a decision based on the initial parameters, the number of agents and the maximum number of steps for each scenario are specified, the forces that each agent is allowed to be acted on in four orthogonal directions on the plane are defined as the action space, and the space containing the agent's own position, the position of the rectangular corners of the functional room formed by itself, the position relative to other agents and the corners of the functional room, and the position information relative to the immovable object is defined as the observation space. All position information is standardized. The agent selects the next action and the plane force of each agent according to the initial parameters through the strategy it has learned, and generates the corresponding decision; In the step of dynamically simulating and diffusing the intelligent agent in the intelligent agent training environment according to the decision generated by the intelligent agent, the initial state list information is received, and the initial state list information at least includes the initial position of the intelligent agent, force information and a flag of whether to reset the dynamic simulation, and the initial state list information is data preprocessed. The initial state list information is data separated by data preprocessing, and the position information contained therein is mapped to the actual size, and the diffusion rate parameter in the initial state list information is read. The intelligent agent generates circles that can collide with each other according to its position coordinates and diffusion rate parameters. The movement of the circle is limited to the area to be designed, and the surroundings are set as rigid bodies. The circle will bounce back after collision. The force on the circle comes from the force vector of each intelligent agent. The force point is located at the center of the circle. The circle moves under the force and collides with each other. The diffusion mode is calculated according to the position information after movement and collision, and each ring is diffused into a functional room.

2. The method for automatic building space combination based on multi-agent deep reinforcement learning according to claim 1 is characterized in that: Each agent contains a set of main actor networks, a set of target actor networks, a main critic network, and a set of target critic networks. The main actor network is used to determine the best action in the current environment state, the target actor network is used to calculate the action in the future state, and the main critic network and a set of target critic networks are used to evaluate the expected return under a specific state and action combination.

3. The method for automatic building space combination based on multi-agent deep reinforcement learning according to claim 1, characterized in that: The steps of calculating the diffusion pattern based on the position information after the movement and collision include: Initialization: Define a set of seed points within the boundary and associate an initial square with each point; Expansion process: Each square expands uniformly from its center outward, the expansion rate is determined by the assigned expansion speed, and the expansion is limited by the boundaries and the constraints of neighboring squares; Speed ​​Mapping: The expansion speed of each square is mapped to the magnitude of a specific attribute, allowing visualization of the distribution and impact of that attribute; Collision detection: In each iteration, check for collisions between the edges of the expanded square and the border and other squares. If a collision occurs, the affected edge stops expanding, and the other edges continue to expand until they are also stopped by the collision. This process is repeated until all edges of all squares are stopped from expanding due to collision. The expansion of the rectangle is governed by the following equation: S i (T+1)=S i (T)+r i ·ΔT Where S i (T) is the size of square i at time T, r i is the expansion rate of square i and ΔT is the time increment between iterations.

4. The method for automatic building space combination based on multi-agent deep reinforcement learning according to claim 1, characterized in that: The steps of scoring according to the functional room layout results, optimizing decisions based on the scoring results, and continuing training until the score of each agent reaches a preset value include calculating the diffusion pattern according to the position information after movement and collision, calculating the coordinates of each functional room corner point, and the generated functional room layout is input into the reward system for scoring, obtaining the score of each agent, and judging whether the completion mark is reached.

5. The method for automatic building space combination based on multi-agent deep reinforcement learning according to claim 4 is characterized in that: The generated functional room layout is input into the reward system for scoring. The coordinates of the functional rooms and the scores and task completion marks of each intelligent agent are merged into a list. The current state and observation value are the coordinate values ​​of the corner points of each functional room calculated by the above data, the relative coordinate values ​​of each corner point, and the relative coordinate value of each corner point relative to the column. The relative coordinate value is calculated by the difference between its own coordinates and the relative coordinates.

6. The method for automatic building space combination based on multi-agent deep reinforcement learning according to claim 4, characterized in that: The reward system consists of a set of logic algorithm modules and a set of MLP neural networks. The reward system scores based on a preset reward calculation formula, which includes: F(d p )=-α·(d p +0.5l) Among them, F(d p ) is the reward formula for calculating the functional room that needs to be close to, which is used to calculate the reward value when a room needs to be close to a specific point, d p represents the distance from the room to the target point, l is the length of the long side of the entire transformation area, and α is the weight coefficient; G(d ij )=-β·(d ij +0.5l) Among them, G(d ij ) is used to calculate the rooms with the same function that need to be close to each other, d ij is the distance calculated between two rooms with the same function, and β is the weight coefficient; T(d)=θ·(d-0.5l) Where T(d) is the formula for calculating the distance between rooms, d is the distance between rooms, and θ is the weight coefficient; H(d ij )=γ·(d ij -0.5l) Among them, H(d ij ) is to calculate the same function rooms that need to be far away from each other, d ij is the distance calculated between two rooms with the same function, and γ is the weight coefficient; E(L)=u·Lv·(1-L) Among them, E(L) is used to score the lighting of the room. L represents the degree to which one side of the room is close to the building outline. L = 1 means that at least one side of the room is close to the building outline, and L = 0 means that the room is surrounded by other rooms on all four sides. u and v are positive weight coefficients; t=d c / d max Among them, D(d c ) is used to calculate the reward based on the positional relationship between the corner point of the functional room and the original column, and B(t) is used to convert the distance d from a point to the nearest column c Mapped to a Bezier curve, the score is determined based on the value of the curve, and t is based on the distance d c The normalized Bezier curve parameters are as follows: c / d max , mapped to the interval [0,1] by normalization, d max is the column spacing, n is the order of the curve, for a cubic Bezier curve, n = 3, C(n, i) is the number of combinations, which represents the number of ways to select i elements from n elements, P i is the ith control point, and δ is a positive weight coefficient.

7. The method for automatic building space combination based on multi-agent deep reinforcement learning according to claim 1, characterized in that: The number of agents is the same as the number of functional rooms.

8. The method for automatic building space combination based on multi-agent deep reinforcement learning according to claim 6, characterized in that: In the neural network reward model, 5 hidden layers are set, the number of iterations is 15, and the training set is four groups of 20 functional room rectangles with length, width, area and corresponding function labels that are close to the transformation target, and all are standardized. The test set is a spatial combination generated by reinforcement learning, and the neural network outputs the label of each corresponding function. When the label is the same as the expected function label, the shape and area of ​​the functional room meet expectations, and a positive reward is given.

Images