Building two-dimensional bidirectional module layout generation system based on artificial intelligence reinforcement learning

Abstract

The invention relates to a building two-dimensional bidirectional module layout generation system based on artificial intelligence reinforcement learning, and belongs to the technical field of buildings. The system automatically processes input information, including a to-be-arranged area, a module type and module parameters, and constructs a reinforcement learning framework by using priori knowledge of a modular building arrangement rule. Through training, the model outputs a plurality of module arrangement and combination results obtained through calculation of the reinforcement learning model. According to the system, a two-dimensional layout mode is adopted, whether arrangement is completed or not is judged through the remaining area, and the width of the module is not limited any more, and whether arrangement is completed or not is judged only depending on the length. The length and width of the modules are only determined by input parameters of the algorithm, and the modules are allowed to be flexibly arranged in the transverse direction and the longitudinal direction. Manual operation is simplified, and the method has the advantages of automation, quick response, multi-scheme generation and the like. The practicability of the method is further enhanced by the multi-dimensional evaluation analysis and visualization function, so that the method shows a wide application prospect in the field of buildings.

Description

Technical Field

[0001] The invention belongs to the technical field of building, and relates to a building two-dimensional bidirectional module layout generation system based on artificial intelligence reinforcement learning. Background Art

[0002] In recent years, with the rapid growth of the world's population and the rapid advancement of urbanization, the construction industry is facing unprecedented challenges. Traditional construction methods can no longer meet the comprehensive needs of modern society for construction speed, quality, cost and environmental sustainability. In this context, prefabricated modular buildings, as an innovative construction method, have gradually attracted widespread attention and favor.

[0003] Modular construction transfers most of the construction work to the factory for prefabrication, significantly improving the efficiency and quality of the construction process. This construction method not only shortens the construction period, but also reduces on-site waste, while bringing the advantages of better quality control, disassembly and reusability. In addition, modular construction can effectively reduce the accident rate and improve the safety of construction workers in a factory-produced environment. These advantages have made modular construction widely used in a variety of scenarios, such as schools, hotels, dormitories and offices.

[0004] However, modular construction also faces some challenges. For example, in terms of apartment generation, traditional construction methods have high flexibility in apartment design, while modular construction, due to the introduction of the concept of prefabricated modules, requires more factors to be considered at the early stage of design, such as the size, shape, and connection method of the modules. This increases the complexity of the design stage, making it easy to make mistakes, which in turn increases the cost of the project.

[0005] On the other hand, modular building unit generation also needs to consider the issue of diversity. Since the prefabricated modules of modular buildings have certain standardization and repetitiveness, how to achieve diversity and personalization of unit types while ensuring the quality of the building is a difficult problem in the promotion of modular buildings.

[0006] To solve these problems, researchers began to explore the use of intelligent algorithms and artificial intelligence to generate content to automate the process of modular building unit generation. Reinforcement learning, as an advanced machine learning method, can continuously learn and optimize decision-making strategies in interaction with the environment. Applying reinforcement learning to modular building unit generation can automatically explore various feasible unit design solutions based on different needs and constraints, and through continuous learning and optimization, find the best solution that meets multiple requirements such as function, structure, cost, user personalization and environmental sustainability.

[0007] At present, some methods for generating building module layouts based on reinforcement learning have been proposed in the prior art. However, when setting the module width, the existing module arrangement method usually sets it to be the same as the area to be arranged. Whether the arrangement is completed is judged only by considering the relationship between the module length and the length of the area to be arranged. This method is a one-dimensional and unidirectional arrangement with obvious limitations. Summary of the invention

[0008] In view of this, the purpose of the present invention is to provide a building two-dimensional bidirectional module layout generation system based on artificial intelligence reinforcement learning, which automatically processes input information, including the area to be arranged, the type of module and the module parameters, and uses the prior knowledge of modular building arrangement rules to construct a reinforcement learning framework. Through training, the model can output a variety of module arrangement combination results calculated by the reinforcement learning model for designers to choose. The system adopts a two-dimensional layout method, and judges whether the arrangement is completed by the remaining area, and no longer limits the width of the module to only rely on the length to judge whether the arrangement is completed. The length and width of the module are determined only by the input parameters of the algorithm, and the module is allowed to be flexibly arranged in the horizontal and vertical directions. This innovation not only simplifies manual operation, but also has the advantages of automation, rapid response and multi-scheme generation. In addition, the multi-dimensional evaluation analysis and visualization functions further enhance the practicality of the method, making it show a broad application prospect in the field of architecture. Through this advanced method, designers can carry out architectural design more efficiently, improve work efficiency and optimize design quality.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] A building two-dimensional bidirectional module layout generation system based on artificial intelligence reinforcement learning, the system comprising: an algorithm input unit, an algorithm framework building unit, an algorithm initialization unit, an algorithm training unit, a scheme evaluation unit and an algorithm output unit;

[0011] The algorithm input unit is used to input the area to be arranged, the module size, and the algorithm parameters; the area to be arranged includes the building outline and the room boundary; the module size includes the minimum length, width, and maximum length and width of the input module; the algorithm parameters include different generation targets, namely, the total number of modules, the type of modules, and the weights α1, α2, and α3 of the degree of module alignment, which must satisfy α1+α2+α3=1;

[0012] The algorithm framework building unit is used to build a reinforcement learning framework;

[0013] The algorithm initialization unit is used to initialize the reinforcement learning environment and the intelligent agent;

[0014] The algorithm training unit is used to perform agent training in a cyclic iteration and output a final solution;

[0015] The scheme evaluation unit is used to perform quantitative evaluation of the results and visual description of indicators, including quantitative statistical evaluation of the number of module types, the total number of modules, and the degree of room alignment of the generated scheme;

[0016] The algorithm output unit is used to output module arrangement scheme comparison analysis, scheme floor plan and scheme data table.

[0017] Furthermore, the algorithm input unit receives three input conditions: 1) a JSON description file of the area to be arranged; 2) a set of optional modules; 3) algorithm parameters, including the weights of the three arrangement goals of total number of modules, module types, and module alignment on the results, and the exploration decay rate parameter of reinforcement learning; specifically, the following steps are included:

[0018] S1. Processing the JSON description file of the area to be arranged: reading the JSON description file of the area to be arranged, obtaining the coordinates of the boundary of each area, and calculating the coordinates of each area through the coordinates of the boundary to form a dot matrix of the entire area to be arranged;

[0019] S2, optional module set processing: Generate different modules by inputting the maximum and minimum length and width constraints and generation strategies, and access the specific module size by module serial number;

[0020] S3, algorithm parameter processing: obtain the weights α1, α2, α3 of the total number of modules, module types, and module alignment degree, which must satisfy α1+α2+α3=1; obtain the exploration rate parameter ε decay .

[0021] Furthermore, the algorithm framework building unit builds a reinforcement learning algorithm framework, including the definition and setting of the environment, state, action, and reward mechanism; the environment is defined as a layout area, which is composed of the boundary of the layout area and the boundary of the internal functional area; a two-dimensional layout method is adopted, and the state setting cannot only set the distance to the boundary in a certain direction; the intelligent body state is the dot matrix information of the entire layout area, in which the arranged areas are marked by the modules as their type numbers, and the unarranged areas are marked as -1; whether the area has been arranged with modules is judged by whether the coordinate value in the dot matrix is ​​-1, and whether the area to be arranged is completed is judged by whether there are still areas with -1; the intelligent body action obtains a list of optional module size types by the length and width range parameters of the module in the input parameters and the generation strategy, and then accesses and selects different actions through the type number; the reward mechanism provides feedback on the actions performed by the intelligent body from three angles: the total number of modules, the type of modules, and the degree of module alignment.

[0022] Furthermore, the algorithm framework construction specifically includes the following steps:

[0023] S1. Build environment: Count the boundaries of each room and functional area, and store each boundary in the form of starting point and end point in a Json file;

[0024] S2. Construct state space: Construct the state space of the agent to obtain the state of the agent in the environment at any time; the modules are arranged from the upper left corner of the area, and the position of the next arrangement is randomly selected from the 4 vertices of the module arranged last time; the dot matrix of the entire room is used as the state space, and the coordinate value of the area is -1 to indicate which areas are not arranged and which areas are arranged, and whether the arrangement is completed is determined by whether all the areas to be arranged are not -1;

[0025] S3, build action space: the agent action space is determined by the set of module sizes;

[0026] S4. Construct a reward mechanism: Construct a reward function to obtain the reward that the environment gives back to the agent after executing the action, that is, selecting a module of a certain size and placing it into the environment.

[0027] Further, in step S4, the action execution is evaluated from three aspects according to three objectives: 1) the total number of modules; 2) the type of modules; 3) the degree of module alignment; the specific evaluation methods include:

[0028] S41. After placing the module, calculate the positional relationship between the placed module and the layout area. There are three possible positional relationships: ① The placed module overlaps with the previously placed module; ② The size of the placed module exceeds the room boundary; ③ The placed module neither overlaps with the layout area nor exceeds the room boundary. By checking the relationship, a reward r describing the total number of modules and the type of modules is obtained from a qualitative and quantitative perspective. 1、 r2; Because the total number of modules is described by the number of times the action is executed, executing any action indicates an increase in the total number of modules, and judging whether the module type appears indicates an increase in the type;

[0029] S42, after the module is successfully placed, the alignment degree between the modules in the entire room is determined, that is, whether there are many misaligned module vertices. The more misaligned module vertices there are, the lower the module alignment degree is considered to be, and a reward r3 for the module alignment degree is obtained;

[0030] S43, the sign of training termination, the remaining area of ​​the room cannot accommodate any module, and feedback r4 is given by the stock of the remaining space size; r1, r2, r3 are adjusted according to the input weights α1, α2, α3, and a batch of results that meet the input targets are obtained through training through different feedback signals.

[0031] Further, the algorithm initialization unit initialization step specifically includes:

[0032] S1. Initialize the environment, including: initializing each coordinate value of the dot matrix to -1; returning the initial position of the agent to the upper left corner of the room; clearing the historical action list;

[0033] S2. Initialize the neural network and use the deep reinforcement learning method. The agent makes decisions and evaluates actions through the neural network. The neural network includes the action evaluation network N eval and the target network N target , the initialization of the neural network includes: randomly initializing the action evaluation network N eval The weight of N target The weights are set to be equal to N eval same;

[0034] S3. Initialize the experience buffer: Using the deep reinforcement learning method, the agent trains the neural network by sampling the existing data in the experience buffer; positive and negative samples are stored in different experience replay buffers; initialization of the experience replay buffer includes clearing the two buffers;

[0035] S4. Initialize the parameters of the deep reinforcement learning agent, including:

[0036] S41, initialization exploration rate ε, exploration decay rate ε decay ; The agent determines whether to randomly select an action or obtain an action through a neural network by generating a random number; if the random number is less than ε, the action is randomly selected, otherwise the current state is input into the neural network to obtain the output action; the initial ε is set to a large value, and the agent mainly performs actions randomly; each time an action is determined, ε is adjusted according to ε decay Decay, subsequent agents will tend to use neural networks to get actions;

[0037] S42, initializing a discount factor γ, where the discount factor is used to calculate a time difference target value during training;

[0038] S43, initialize the training batch size batch_size;

[0039] S44. Initialize the target network parameter update frequency update_freq.

[0040] Further, the algorithm training unit carries out training by adopting the following steps:

[0041] S1. Agent selection action: At any time in state S, the agent decides to randomly select a module or select the action A with the largest Q value based on the current state S according to the environment exploration rate ε; at the same time, the exploration rate ε is based on the exploration decay rate ε decay reduce;

[0042] S2. Feedback from the environment: Based on the actions performed by the agent, i.e. the selected module, the environment gives the agent corresponding reward values ​​based on the reward mechanism, including:

[0043] S21, record the current position of the agent, find the corresponding module size according to the selected action, and update the position of the agent;

[0044] S22. Give a reward R according to the reward mechanism;

[0045] S23, determine the placement position of the current module. If the length or width of the module plus the length or width of the agent exceeds the range of the room, the action is invalid, the module sequence is not updated, the current state Q value is updated, and the training end flag T of this round is set to true; if there are other modules at the position where the current module is placed, the action is invalid, the module sequence is not updated, the current state Q value is updated, and the training end flag T of this round is set to true; if the position where the current module is placed does not exceed the range of the room and there are no other modules, the action is valid, the module sequence is updated, the Q value is updated, and the position of the agent is updated;

[0046] S24, obtain a new state S' according to the updated pointer;

[0047] S25. Store the five-tuple (S, A, R, S', T) into the positive sample or negative sample experience buffer according to whether the action is valid;

[0048] S3, Update neural network: This step samples data from the experience buffer, trains the network, and completes weight update; first check whether the size of the positive and negative sample experience buffers is not less than batch_size / 2. If so, do not update the network; otherwise, update the network. The main process is as follows:

[0049] S31. Sample batch_size / 2 from positive and negative samples respectively, combine them into a batch of data, and input N eval , get the estimated output yhat;

[0050] S32, using the target network output to calculate the time difference target value y;

[0051] S33, calculate the mean square error loss of yhat and y, and update N by gradient descent eval ;

[0052] S34. If the number of training times meets update_freq, N eval The weight value is copied to N target ;

[0053] S4. Terminate training and output results: Perform training according to the set number of training rounds. During the training process, store the results of successful arrangement and record the result information. After the training, output the Pareto images of all results, and select a series of corresponding Pareto frontier solutions as the result output according to multiple objectives, and output a variety of arrangement methods for designers to screen.

[0054] Furthermore, the scheme evaluation unit performs the scheme quantitative evaluation including the following steps:

[0055] S1. Count the total number of modules and calculate the length of the execution action sequence of each area to be arranged to obtain the total number of modules;

[0056] S2, counting the number of module types, calculating the number of modules, and storing them in a dictionary form;

[0057] S3, counting the module alignment degree, calculating the number of isolated vertices of adjacent modules in each arrangement area, and determining whether the length and width of adjacent modules are the same;

[0058] S4. Output the evaluation results.

[0059] Furthermore, the algorithm output unit algorithm output includes: 1) a Pareto diagram of all collected arrangement schemes; 2) a display of Pareto frontier solutions among multiple targets; 3) a data table of arrangement schemes for each module.

[0060] The beneficial effects of the present invention are:

[0061] The present invention adopts a two-dimensional layout method, and determines whether the arrangement is completed by the remaining area, instead of limiting the width of the module and only relying on the length to determine whether the arrangement is completed. The length and width of the module are determined only by the input parameters of the algorithm, and the modules are allowed to be flexibly arranged in the horizontal and vertical directions. This innovation not only simplifies manual operation, but also has the advantages of automation, rapid response and multi-scheme generation. In addition, the multi-dimensional evaluation analysis and visualization functions further enhance the practicality of this method, making it show broad application prospects in the field of architecture. Through this advanced method, designers can carry out architectural design more efficiently, improve work efficiency and optimize design quality.

[0062] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0064] Figure 1 It is the overall operation flow chart of the present invention;

[0065] Figure 2 It is a schematic diagram of the area to be arranged;

[0066] Figure 3 It is the module size range or type and fixed size diagram;

[0067] Figure 4 This is a diagram of the agent training framework;

[0068] Figure 5 Figure 1 is a diagram of the agent training process;

[0069] Figure 6 Provide quantitative evaluation diagram for the scheme;

[0070] Figure 7 This is the floor plan of the module layout. DETAILED DESCRIPTION

[0071] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0072] The present invention provides a building two-dimensional bidirectional module layout generation system based on artificial intelligence reinforcement learning, which automatically processes input information, including the area to be arranged, module types and module parameters, and uses the prior knowledge of modular building arrangement rules to build a reinforcement learning framework. Through training, the model can output a variety of module arrangement combination results calculated by the reinforcement learning model for designers to choose. Figure 1 This is an overall operation flow chart of the present invention, and the system includes: an algorithm input unit, an algorithm framework building unit, an algorithm initialization unit, an algorithm training unit, a solution evaluation unit and an algorithm output unit; the algorithm input unit is used to input the area to be arranged, the module size, and the algorithm parameters; the area to be arranged includes the building outline and the room boundary; the module size includes the minimum length, width and the maximum length and width of the input module; the algorithm parameters include different generation targets, namely the total number of modules, the type of modules, and the weights of the degree of module alignment; the algorithm framework building unit is used to construct a reinforcement learning framework; the algorithm initialization unit is used to initialize the reinforcement learning environment and the agent; the algorithm training unit is used to iterate the agent training and output the final solution; the solution evaluation unit is used to quantitatively evaluate the results and visualize the indicators, including quantitative statistical evaluation of the number of module types, the total number of modules, and the degree of room alignment of the generated solution; the algorithm output unit is used to output the module arrangement solution comparison analysis, solution floor plan and solution data table.

[0073] like Figure 2 and Figure 3 As shown, the algorithm input unit inputs the area to be arranged, module size, and algorithm parameters; the area to be arranged includes the building outline and room boundary; the module size includes the minimum length, width and maximum length and width of the input module, such as length 500-1000mm and width 400-800mm.

[0074] The input conditions of the algorithm input unit include: 1) JSON description file of the area to be arranged; 2) length and width parameters of the module and generation strategy; 3) algorithm parameters, including the weights of the three arrangement goals of total number of modules, module types, and module alignment on the results, and the exploration decay rate parameters of reinforcement learning. After processing, the output includes: 1) dot map of the area to be arranged; 2) list of optional module size types; 3) weights of each arrangement goal and algorithm exploration decay rate, which are used as the input of the next unit.

[0075] The algorithm framework building unit is used to build the reinforcement learning algorithm framework, including the definition and setting of the environment, state, action, and reward mechanism. The environment is defined as a layout area, which consists of the boundary of the layout area and the boundary of the internal functional area. Unlike the existing definition of the state of the agent, our method is oriented to a two-dimensional layout, so the state setting cannot only set the distance to the boundary in a certain direction. In our method, the state of the agent is the dot matrix information of the entire layout area, in which the arranged areas are marked by the modules as their type numbers, and the unarranged areas are marked as -1. Whether the area has been arranged by the coordinate value in the dot matrix is ​​determined by whether it is -1, and whether the area to be arranged is completed by whether there is still an area with a value of -1. The agent action is obtained by the length and width range parameters of the module in the input parameters and the generation strategy to obtain a list of optional module size types, and then access and select different actions through the type number. The reward mechanism provides feedback on the actions performed by the agent from three perspectives: the total number of modules, the type of modules, and the degree of module alignment. Figure 4 Figure 2 is a diagram of the agent training framework.

[0076] Specifically, the algorithm input unit receives three input conditions: 1) JSON description file of the area to be arranged; 2) optional module set; 3) algorithm parameters, including the influence weights of the three arrangement goals of total number of modules, module types, and module alignment on the results, as well as the exploration decay rate parameter of reinforcement learning. The specific processing process is:

[0077] Processing of the JSON description file of the area to be arranged: read the JSON description file of the input area to be arranged, obtain the coordinates of the boundary of each area, calculate the coordinates of each area through the coordinates of the boundary, and form a dot matrix of the entire area to be arranged.

[0078] Optional module set processing: Generate different modules by inputting the maximum and minimum length and width constraints and generation strategies, and access the specific module size through the module serial number.

[0079] Algorithm parameter processing: Get the weights α1, α2, α3 of the total number of modules, module types, and module alignment, which must satisfy α1+α2+α3=1; get the exploration rate parameter ε decay .

[0080] The algorithm framework is constructed, including the following steps:

[0081] Step 1. Build the environment

[0082] Count the boundaries of each room and functional area, and store each boundary in the form of starting point and end point in a Json file.

[0083] Step 2. Construct the state space

[0084] Construct the state space of the agent to obtain the state of the agent in the environment at any time. The modules are arranged from the upper left corner of the area, and the position of the next arrangement is randomly selected from the 4 vertices of the module arranged last time. The dot matrix of the entire room is used as the state space. Whether the coordinate value of the area is -1 indicates which areas are not arranged and which areas are arranged. Whether the arrangement is completed is determined by whether all the areas to be arranged are not -1.

[0085] Step 3. Construct the action space

[0086] The agent action space is determined by a set of module sizes.

[0087] Step 4. Build a reward mechanism

[0088] Construct a reward function to obtain the reward that the environment gives back to the agent after executing an action, that is, selecting a module of a certain size and placing it in the environment. Based on the three objectives, the action execution is evaluated from three aspects: (1) the total number of modules; (2) the type of modules; and (3) the degree of module alignment. The specific evaluation ideas are:

[0089] (1) After placing the module, calculate the positional relationship between the placed module and the layout area. There are three possible positional relationships: ① The placed module overlaps with the previously placed module; ② The size of the placed module exceeds the room boundary; ③ The placed module neither overlaps with the layout area nor exceeds the room boundary. By checking the relationship, a reward r describing the total number of modules and the type of modules is obtained from a qualitative and quantitative perspective. 1、 Since the total number of modules is described by the number of times the action is executed, executing any action indicates an increase in the total number of modules, and judging whether the module type appears indicates an increase in the type.

[0090] (2) After the module is successfully placed, the degree of alignment between the modules in the entire room is determined, that is, whether there are many misaligned module vertices. The more misaligned module vertices there are, the lower the module alignment degree is considered, and the module alignment degree reward r3 is obtained.

[0091] (3) The sign of training termination is that the remaining area of ​​the room cannot accommodate any module. Feedback r4 is given on the remaining space size.

[0092] According to the input weights α1, α2, α3, r1, r2, and r3 are adjusted. Through different feedback signals and training, a batch of results that meet the input targets are obtained.

[0093] The steps performed by the algorithm initialization unit include:

[0094] Step 1. Initialize the environment

[0095] The initialization of the environment includes:

[0096] (1) Initialize each coordinate value of the dot map to -1.

[0097] (2) The initial position of the agent returns to the upper left corner of the room.

[0098] (3) Clear the history action list.

[0099] Step 2. Initialize the neural network

[0100] Using deep reinforcement learning methods, the agent makes decisions and evaluates actions through a neural network. The neural network includes an action evaluation network N eval and the target network N target . The initialization of the neural network includes:

[0101] (1) Randomly initialize the action evaluation network N eval The weight of .

[0102] (2) N target The weights are set to be equal to N eval same.

[0103] Step 3. Initialize the experience buffer

[0104] Using deep reinforcement learning, the agent trains the neural network by sampling the existing data in the experience buffer. Positive and negative samples are stored in different experience replay buffers. Experience replay buffer initialization includes clearing the two buffers.

[0105] Step 4. Initialize agent parameters

[0106] Initialize the deep reinforcement learning agent parameters, including:

[0107] (1) Initialize the exploration rate ε and the exploration decay rate ε decay The agent determines whether to randomly select an action or obtain an action through a neural network by generating a random number. If the random number is less than ε, the action is randomly selected, otherwise the current state is input into the neural network to obtain the output action. The initial ε is set to a large value, and the agent mainly performs actions randomly. After each action is determined, ε is adjusted according to ε decay Attenuation, subsequent agents will tend to use neural networks to get actions.

[0108] (2) Initialize the discount factor γ. The discount factor is used to calculate the temporal difference target value during training.

[0109] (3) Initialize the training batch size batch_size.

[0110] Initialize the target network parameter update frequency update_freq.

[0111] Figure 5 The figure is a diagram of the agent training process. The execution steps of the algorithm training unit include:

[0112] Step 1: The agent chooses an action

[0113] At any time in state S, the agent decides to randomly select a module or select action A with the largest Q value based on the current state S according to the environment exploration rate ε. At the same time, the exploration rate ε is based on the exploration decay rate ε decay Reduce.

[0114] Step 2: Feedback from the environment

[0115] According to the action performed by the agent, that is, the module selected, the environment will give the agent the corresponding reward value according to the reward mechanism. The specific process is:

[0116] (1) Record the current position of the agent, find the corresponding module size according to the selected action, and update the position of the agent.

[0117] (2) Give a reward R according to the reward mechanism.

[0118] (3) Determine the placement position of the current module. If the length or width of the module plus the length or width of the agent exceeds the range of the room, the action is invalid, the module sequence is not updated, the current state Q value is updated, and the training end flag T for this round is set to true. If there are other modules at the location where the current module is placed, the action is invalid, the module sequence is not updated, the current state Q value is updated, and the training end flag T for this round is set to true. If the location where the current module is placed does not exceed the range of the room and there are no other modules, the action is valid, the module sequence is updated, the Q value is updated, and the position of the agent is updated.

[0119] (4) Obtain the new state S' according to the updated pointer.

[0120] (5) Depending on whether the action is valid, store the five-tuple (S, A, R, S', T) into the positive sample or negative sample experience buffer.

[0121] Step 3: Update the Neural Network

[0122] This step samples data from the experience buffer, trains the network, and completes the weight update. First, check whether the size of the positive and negative sample experience buffers is not less than batch_size / 2. If it is less than, the network is not updated; otherwise, the network is updated. The main process is as follows:

[0123] (1) Sample batch_size / 2 from positive and negative samples respectively, combine them into a batch of data, and input N eval , and obtain the estimated output yhat.

[0124] (2) Use the target network output to calculate the time-difference target value y.

[0125] (3) Calculate the mean square error loss between yhat and y, and update N by gradient descent eval .

[0126] (4) If the number of training times meets update_freq, N eval The weight value is copied to N target .

[0127] Step 4: Terminate training and output results

[0128] The training is performed according to the set number of training rounds. During the training process, the successful arrangement results are stored and the result information is recorded. After the training is completed, the Pareto images of all results are output, and a series of corresponding Pareto frontier solutions are selected as the result output according to multiple goals. A variety of arrangement methods are output for designers to screen.

[0129] Figure 6 For the scheme quantitative evaluation diagram, the execution steps of the scheme evaluation unit include: Step 1: Count the total number of modules: calculate the length of the execution action sequence of each area to be arranged, and get the total number of modules. Step 2: Count the number of module types: calculate the number of modules and store them in the form of a dictionary. Step 3: Count the degree of module alignment: calculate the number of isolated vertices of adjacent modules in each arrangement area, and determine whether the length and width of adjacent modules are the same. Step 4: Output the evaluation result.

[0130] The algorithm outputs include: (1) a Pareto diagram of all collected layout plans; (2) a display of the Pareto frontier solutions for multiple objectives; and (3) a data table of the layout plans for each module. Figure 7 This is the floor plan of the module layout.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A building two-dimensional bidirectional module layout generation system based on artificial intelligence reinforcement learning, characterized by: The system includes: an algorithm input unit, an algorithm framework building unit, an algorithm initialization unit, an algorithm training unit, a solution evaluation unit and an algorithm output unit; The algorithm input unit is used to input the area to be arranged, the module size, and the algorithm parameters; the area to be arranged includes the building outline and the room boundary; the module size includes the minimum length, width, and maximum length and width of the input module; the algorithm parameters include different generation targets, namely, the total number of modules, the type of modules, and the weights α1, α2, and α3 of the degree of module alignment, which must satisfy α1+α2+α3=1; The algorithm framework building unit is used to build a reinforcement learning framework; The algorithm initialization unit is used to initialize the reinforcement learning environment and the intelligent agent; The algorithm training unit is used to perform agent training in a cyclic iteration and output a final solution; The scheme evaluation unit is used to perform quantitative evaluation of the results and visual description of indicators, including quantitative statistical evaluation of the number of module types, the total number of modules, and the degree of room alignment of the generated scheme; The algorithm output unit is used to output module arrangement scheme comparison analysis, scheme floor plan and scheme data table.

2. The system for generating a two-dimensional bidirectional module layout of a building based on artificial intelligence reinforcement learning according to claim 1, characterized in that: The algorithm input unit receives three input conditions: 1) a JSON description file of the area to be arranged; 2) A set of optional modules; 3) Algorithm parameters, including the weights of the three arrangement goals of total number of modules, module types, and module alignment on the results, as well as the exploration decay rate parameter of reinforcement learning; specifically, the following steps are included: S1. Processing the JSON description file of the area to be arranged: reading the JSON description file of the area to be arranged, obtaining the coordinates of the boundary of each area, and calculating the coordinates of each area through the coordinates of the boundary to form a dot matrix of the entire area to be arranged; S2, optional module set processing: Generate different modules by inputting the maximum and minimum length and width constraints and generation strategies, and access the specific module size by module serial number; S3, algorithm parameter processing: obtain the weights α1, α2, α3 of the total number of modules, module types, and module alignment degree, which must satisfy α1+α2+α3=1; obtain the exploration rate parameter ε decay .

3. The system for generating a two-dimensional bidirectional module layout of a building based on artificial intelligence reinforcement learning according to claim 2, characterized in that: The algorithm framework building unit builds a reinforcement learning algorithm framework, including the definition and setting of the environment, state, action, and reward mechanism; the environment is defined as an arrangement area, which is composed of the boundary of the arrangement area and the boundary of the internal functional area; a two-dimensional arrangement method is adopted, and the state setting cannot only set the distance to the boundary in a certain direction; The state of the agent is the dot matrix information of the entire arrangement area, where the arranged area is marked by the module with its type number, and the unarranged area is marked with -1; Whether the coordinate value in the dot map is -1 is used to determine whether the area has been arranged with modules, and whether there is still an area with -1 is used to determine whether the area to be arranged has been arranged; the agent's actions are based on the length and width range parameters of the module in the input parameters and the generation strategy to obtain a list of optional module size types, and then access and select different actions through the type number; the reward mechanism provides feedback on the actions performed by the agent from three perspectives: the total number of modules, the type of modules, and the degree of module alignment.

4. The system for generating a two-dimensional bidirectional module layout of a building based on artificial intelligence reinforcement learning according to claim 3 is characterized in that: The algorithm framework construction specifically includes the following steps: S1. Build environment: Count the boundaries of each room and functional area, and store each boundary in the form of starting point and end point in a Json file; S2. Construct state space: Construct the state space of the agent to obtain the state of the agent in the environment at any time; the modules are arranged from the upper left corner of the area, and the position of the next arrangement is randomly selected from the 4 vertices of the module arranged last time; the dot matrix of the entire room is used as the state space, and the coordinate value of the area is -1 to indicate which areas are not arranged and which areas are arranged, and whether the arrangement is completed is determined by whether all the areas to be arranged are not -1; S3, build action space: the agent action space is determined by the set of module sizes; S4. Construct a reward mechanism: Construct a reward function to obtain the reward that the environment gives back to the agent after executing the action, that is, selecting a module of a certain size and placing it into the environment.

5. The system for generating a two-dimensional bidirectional module layout of a building based on artificial intelligence reinforcement learning according to claim 4, characterized in that: In step S4, the action execution is evaluated from three aspects according to three objectives: 1) the total number of modules; 2) the types of modules; 3) Module alignment; specific evaluation methods include: S41. After placing the module, calculate the positional relationship between the placed module and the layout area. There are three possible positional relationships: ① The placed module overlaps with the previously placed module; ② The size of the placed module exceeds the room boundary; ③ The placed module neither overlaps with the layout area nor exceeds the room boundary. By checking the relationship, a reward r describing the total number of modules and the type of modules is obtained from a qualitative and quantitative perspective. 1、 r2; Because the total number of modules is described by the number of times the action is executed, executing any action indicates an increase in the total number of modules, and judging whether the module type appears indicates an increase in the type; S42, after the module is successfully placed, the alignment degree between the modules in the entire room is determined, that is, whether there are many misaligned module vertices. The more misaligned module vertices there are, the lower the module alignment degree is considered to be, and a reward r3 for the module alignment degree is obtained; S43, the sign of training termination, the remaining area of ​​the room cannot accommodate any module, and feedback r4 is given by the stock of the remaining space size; r1, r2, r3 are adjusted according to the input weights α1, α2, α3, and a batch of results that meet the input targets are obtained through training through different feedback signals.

6. The system for generating a two-dimensional bidirectional module layout of a building based on artificial intelligence reinforcement learning according to claim 5, characterized in that: The algorithm initialization unit initialization step specifically includes: S1. Initialize the environment, including: initializing each coordinate value of the dot matrix to -1; returning the initial position of the agent to the upper left corner of the room; clearing the historical action list; S2. Initialize the neural network and use the deep reinforcement learning method. The agent makes decisions and evaluates actions through the neural network. The neural network includes the action evaluation network N eval and the target network N target , the initialization of the neural network includes: randomly initializing the action evaluation network N eval The weight of N target The weights are set to be equal to N eval same; S3. Initialize the experience buffer: Using the deep reinforcement learning method, the agent trains the neural network by sampling the existing data in the experience buffer; positive and negative samples are stored in different experience replay buffers; initialization of the experience replay buffer includes clearing the two buffers; S4. Initialize the parameters of the deep reinforcement learning agent, including: S41, initialization exploration rate ε, exploration decay rate ε decay ; The agent determines whether to randomly select an action or obtain an action through a neural network by generating a random number; if the random number is less than ε, the action is randomly selected, otherwise the current state is input into the neural network to obtain the output action; the initial ε is set to a large value, and the agent mainly performs actions randomly; each time an action is determined, ε is adjusted according to ε decay Decay, subsequent agents will tend to use neural networks to get actions; S42, initializing a discount factor γ, where the discount factor is used to calculate a time difference target value during training; S43, initialize the training batch size batch_size; S44. Initialize the target network parameter update frequency update_freq.

7. The system for generating a two-dimensional bidirectional module layout of a building based on artificial intelligence reinforcement learning according to claim 6, characterized in that: The algorithm training unit carries out training by adopting the following steps: S1. Agent selection action: At any time in state S, the agent decides to randomly select a module or select the action A with the largest Q value based on the current state S according to the environment exploration rate ε; at the same time, the exploration rate ε is based on the exploration decay rate ε decay reduce; S2. Feedback from the environment: Based on the actions performed by the agent, i.e. the selected module, the environment gives the agent corresponding reward values ​​based on the reward mechanism, including: S21, record the current position of the agent, find the corresponding module size according to the selected action, and update the position of the agent; S22. Give a reward R according to the reward mechanism; S23, determine the placement position of the current module. If the length or width of the module plus the length or width of the agent exceeds the range of the room, the action is invalid, the module sequence is not updated, the current state Q value is updated, and the training end flag T of this round is set to true; if there are other modules at the position where the current module is placed, the action is invalid, the module sequence is not updated, the current state Q value is updated, and the training end flag T of this round is set to true; if the position where the current module is placed does not exceed the range of the room and there are no other modules, the action is valid, the module sequence is updated, the Q value is updated, and the position of the agent is updated; S24, obtain a new state S' according to the updated pointer; S25. Store the five-tuple (S, A, R, S', T) into the positive sample or negative sample experience buffer according to whether the action is valid; S3, Update neural network: This step samples data from the experience buffer, trains the network, and completes weight update; first check whether the size of the positive and negative sample experience buffers is not less than batch_size / 2. If so, do not update the network; otherwise, update the network. The main process is as follows: S31. Sample batch_size / 2 from positive and negative samples respectively, combine them into a batch of data, and input N eval , get the estimated output yhat; S32, using the target network output to calculate the time difference target value y; S33, calculate the mean square error loss of yhat and y, and update N by gradient descent eval ; S34. If the number of training times meets update_freq, N eval The weight value is copied to N target ; S4. Terminate training and output results: Perform training according to the set number of training rounds. During the training process, store the results of successful arrangement and record the result information. After the training, output the Pareto images of all results, and select a series of corresponding Pareto frontier solutions as the result output according to multiple objectives, and output a variety of arrangement methods for designers to screen.

8. The system for generating a two-dimensional bidirectional building module layout based on artificial intelligence reinforcement learning according to claim 7, characterized in that: The scheme evaluation unit performs the scheme quantitative evaluation including the following steps: S1. Count the total number of modules and calculate the length of the execution action sequence of each area to be arranged to obtain the total number of modules; S2, counting the number of module types, calculating the number of modules, and storing them in the form of a dictionary; S3, counting the module alignment degree, calculating the number of isolated vertices of adjacent modules in each arrangement area, and determining whether the length and width of adjacent modules are the same; S4. Output the evaluation results.

9. The system for generating a two-dimensional bidirectional building module layout based on artificial intelligence reinforcement learning according to claim 8, characterized in that: The algorithm output unit algorithm output includes: 1) a Pareto diagram of all collected arrangement schemes; 2) a display of the Pareto frontier solutions among multiple targets; 3) a data table of arrangement schemes for each module.

Images