Construction progress automatic control method and system based on generative model and reinforcement learning

Abstract

The invention belongs to the technical field of constructional engineering management, and discloses a construction progress automatic control method and system based on a generative model and reinforcement learning, and the control method comprises the steps: S1, training the generative model; s2, generating a construction progress plan; s3, reinforcement learning model optimization; and S4, cooperative work of the generative model and the reinforcement learning model. The control system comprises five core modules: a data acquisition module, a generative model module, a reinforcement learning model module, a progress adjustment module and a feedback module. According to the system, through cooperative work of the modules, the construction progress control system can effectively realize dynamic optimization and real-time feedback control in the process from initial plan generation to construction, information exchange among the modules ensures that the system can quickly respond to field changes, an optimal construction progress plan is provided, and the construction progress control efficiency is improved. And efficient and intelligent construction management is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of construction project management, and specifically relates to a construction progress automatic control method and system based on a generative model and reinforcement learning. Background Art

[0002] Construction project construction progress management is an important part of project management, which directly affects the economic benefits and resource utilization efficiency of the project. At present, traditional construction progress management methods usually rely on the experience of project managers or network planning techniques (such as critical path method and plan review technology) to formulate construction plans. However, various uncontrollable factors often occur in the actual construction process, such as weather changes, resource shortages or sudden safety accidents, which leads to frequent adjustments during the execution of the plan, bringing great challenges to management.

[0003] With the rapid development of artificial intelligence technology, its application in construction management has gradually emerged. Among them, the generative model, with its powerful data processing and plan generation capabilities, can generate a reasonable construction schedule based on historical data, while the reinforcement learning model can continuously optimize and adjust the schedule during the construction process to adapt to the real-time changing construction conditions and external environment, thereby effectively addressing the shortcomings of traditional methods.

[0004] Most existing construction progress control systems lack the organic combination of automatic generation and dynamic adjustment. Especially in complex construction environments, the schedule generation and adjustment processes are often relatively independent and lack linkage, resulting in low efficiency in construction progress management. Therefore, the study of automatic construction progress control systems based on generative models and reinforcement learning can effectively improve the level of intelligence and automation of construction management and reduce the risk of project delays, which has important practical significance. Summary of the invention

[0005] In order to overcome the above technical problems, the present invention provides a construction progress automatic control method and system based on generative models and reinforcement learning, so as to solve the problem of lack of combination of automatic generation and dynamic adjustment in existing construction progress management, improve the intelligence and automation level of construction management, and reduce the risk of construction progress delays.

[0006] The present invention adopts the following technical solutions:

[0007] A construction progress automatic control method based on generative model and reinforcement learning includes the following steps:

[0008] Generative model training

[0009] The present invention first trains a generative model based on a large amount of historical construction project data, including project schedules, resource allocation, dependencies between tasks, and actual feedback data recorded during the construction process. The data from different projects comes from a wide range of sources, covering a variety of construction environments and project sizes, and can provide rich and diverse reference information for the training of the generative model.

[0010] Before model training, cleaning and preprocessing the data is a necessary step. This includes removing outliers, filling in missing data, and standardizing data to ensure the consistency and reliability of input data. The cleaned data will be converted into feature inputs that the model can process, including task categories, resource types, task start and end times, etc. The preprocessed data provides a high-quality training foundation for the model, ensuring that the model can refer to past successful experiences when generating a schedule.

[0011] After data preprocessing, the present invention uses a generative model based on the Transformer structure to train these data. Due to its advantages in processing complex sequence data and multi-dimensional task relationships, the Transformer model can effectively learn the dependencies between construction tasks and resource scheduling patterns. The model is gradually optimized through multiple iterative training to generate an initial schedule that meets actual construction conditions.

[0012] The model training process continuously improves its performance by adjusting parameters (such as learning rate and number of network layers). By learning the regularity of historical data, the generative model can generate a high-precision initial construction schedule for new projects, laying the foundation for subsequent dynamic adjustment and optimization.

[0013] Construction schedule generation

[0014] After the generative model is trained, the present invention can automatically generate a construction schedule according to the specific requirements of the project to be constructed. The generated construction schedule is presented in the form of a task list, which contains the detailed arrangement of each task, including the start time, end time, type and quantity of required resources, and the dependencies between tasks. This information is derived by the generative model based on historical project data and project requirements to ensure that the plan is feasible and logical.

[0015] In the process of generating the construction schedule, the model not only considers the resource allocation of the project itself, but also makes reasonable plans for key factors such as the project's scale, complexity, and expected duration. The generated plan is flexible and can adapt to the requirements of projects of different sizes, and can generate highly customized construction schedules based on specific project characteristics. This intelligent generation capability effectively reduces the reliance on manual participation in the formulation of construction plans and improves the efficiency and accuracy of the overall construction plan.

[0016] The generation of construction schedule is not limited to traditional linear time planning, but can also reflect the dynamic dependencies between tasks. For example, when a task must be started after another task is completed (when the time sequence of tasks is fixed), the model will automatically identify and arrange these dependencies to ensure the rationality of the plan. By reasonably arranging resources and time, the system can maximize resource utilization and reduce the possibility of construction delays.

[0017] The generated construction schedule can be further fine-tuned according to the project characteristics. The present invention supports adjusting the plan according to actual needs. For example, when a project has specific construction difficulties or resource shortages, the model can give priority to these constraints and generate corresponding optimization plans. This flexible adjustment function greatly improves the applicability of the system, enabling it to generate reliable and efficient construction schedules when dealing with projects of different types and sizes.

[0018] Reinforcement Learning Model Optimization

[0019] During the construction process, due to the constant changes in environmental conditions, resource allocation, and task progress, the originally generated construction schedule may encounter unpredictability in the actual situation. The present invention dynamically adjusts and optimizes the initially generated construction schedule by introducing a reinforcement learning model. The reinforcement learning model can adaptively respond to changes that occur during the construction process, and by continuously optimizing and adjusting the order of construction tasks and resource allocation, the overall construction schedule can better adapt to the actual construction environment.

[0020] The core of the reinforcement learning model lies in its efficient use of feedback information. The model learns through real-time data collected from the construction site and makes decisions based on this data to optimize the current construction plan. This real-time data includes the completion progress of tasks, the use of resources, changes in the external environment (such as weather conditions or supply chain delays), etc. The model continuously adjusts the construction schedule based on this information to ensure the accuracy and efficiency of task execution and reduce construction delays caused by external factors.

[0021] During the optimization process, the reinforcement learning model is able to set the corresponding reward mechanism according to the priority of the construction goal. For example, in some cases, the control of the construction period may be the priority goal, while in other cases, resource utilization efficiency or construction quality may be the key considerations. By setting different reward functions, the model can flexibly adjust its optimization strategy to achieve a balance between different goals, ensuring the smooth progress of the construction task and achieving the expected results.

[0022] The collaboration between generative and reinforcement learning models

[0023] A notable feature of the present invention is the organic combination of the generative model and the reinforcement learning model. The generative model is used to automatically generate the initial construction schedule at the beginning of the project, while the reinforcement learning model dynamically adjusts and optimizes the plan during the construction process. Through the collaborative work of these two models, the present invention can realize the full process automation management from the generation of the construction schedule to the real-time optimization, ensuring the adaptability and execution efficiency of the plan.

[0024] The main task of the generative model is to quickly generate a reasonable initial plan based on historical data and project requirements before the construction project starts. This plan is a guiding document for the start of the project, defining the order of tasks, the allocation of resources, and the expected construction period. However, unforeseen events often occur during the construction process, resulting in the original plan not being able to proceed strictly as expected. At this time, the reinforcement learning model will come into play. Through learning and feedback of real-time data, the initial plan is gradually optimized and adjusted to make it more suitable for the current construction environment and needs.

[0025] The collaboration between the two models is highly flexible and adaptable. The generative model provides a stable base plan that lays a good foundation for the start of the project, while the reinforcement learning model ensures the continuous optimization of the plan. As the construction progresses, the reinforcement learning model continuously fine-tunes and optimizes the plan based on field feedback data (task completion, resource usage, weather conditions). Through this process of continuous feedback and adjustment, the system is able to maintain a high degree of consistency between planning and actual execution, avoiding the problem of disconnection between planning and actual execution in traditional construction management.

[0026] The collaborative work of the generative model and the reinforcement learning model can not only provide a stable initial plan for the construction project, but also adjust and optimize it according to the real-time situation during the actual construction process to ensure the feasibility and flexibility of the plan. This synergy greatly improves the level of intelligent construction progress management, allowing the project to proceed smoothly in a complex and changing environment.

[0027] The invention also discloses a construction progress automatic control system based on a generative model and reinforcement learning.

[0028] The construction progress control system of the present invention is composed of five core modules, all of which work together to achieve automatic generation, dynamic optimization and real-time control of the construction progress. The entire system includes a data acquisition module, a generative model module, a reinforcement learning model module, a progress adjustment module and a feedback module. Each module exchanges information through a data interface to ensure efficient management during the construction process. Specific system modules include:

[0029] Data acquisition module

[0030] As the basic module of the construction progress control system of the present invention, the data acquisition module collects engineering data from the construction site in real time. Specific data include task completion progress, resource usage, material inventory, mechanical equipment operating status, and environmental data (such as weather conditions, temperature, humidity, etc.). Data collection relies on intelligent sensor equipment, RFID sensors for monitoring materials and resources, Internet of Things sensors for monitoring the status of mechanical equipment, and weather stations and temperature and humidity sensors for collecting environmental information. All collected data are transmitted to the central control system in real time via a wireless network for analysis and optimization by generative models and reinforcement learning models.

[0031] The data acquisition module ensures that all key data related to the construction progress can be timely and accurately summarized into the system to form a comprehensive field data set. These data provide precise feedback to the system and support the subsequent dynamic optimization process.

[0032] Generative Model Module

[0033] The function of the generative model module is to generate a construction schedule based on the input project parameters at the beginning of the project. The input data includes a detailed task list, resource constraints (such as the number and type of available resources), project scale, duration requirements, etc. Based on the trained Transformer architecture generative model, the present invention can generate a preliminary schedule that meets the project requirements by learning from historical construction data.

[0034] The plan generated by the generative model of the present invention lists in detail the start and end time of each construction task, the types and quantities of resources required, the dependencies between tasks, etc. This preliminary plan provides clear guidance for construction and lays the foundation for subsequent dynamic adjustments. The generative model efficiently handles the task dependencies and resource allocation issues of the project, ensuring that the generated schedule is both executable and flexible enough to cope with the changing needs of the project.

[0035] Reinforcement Learning Model Module

[0036] The reinforcement learning model module is used in the present invention to optimize the construction schedule in real time. The module continuously receives real-time data from the data acquisition module, including the completion status of on-site tasks, resource utilization efficiency, equipment operation status, and changes in the external environment (such as weather changes, equipment failures, etc.). Based on these data, the reinforcement learning model continuously optimizes the current construction progress through a reward mechanism to ensure that the project can be carried out efficiently.

[0037] The reinforcement learning model in the present invention adopts a deep Q network (DQN) structure and sets reward functions related to construction period control, resource optimization and construction quality. Whenever changes or plan deviations are detected on site, the model immediately makes adjustments to rearrange the task sequence and resource allocation to minimize construction interruptions and resource waste. The reinforcement learning model module ensures that the construction plan can continuously adapt to real-time changes on site and optimize the execution effect of construction tasks.

[0038] Schedule Adjustment Module

[0039] The schedule adjustment module updates the construction plan in real time based on the optimization suggestions provided by the reinforcement learning model. The module directly connects to the management system of the construction site and transmits the optimized construction schedule to the on-site construction personnel and management system to ensure that the on-site construction can proceed according to the latest adjustment plan.

[0040] The schedule adjustment module of the present invention can quickly respond to the optimization output of the system and reallocate construction tasks and resources. For example, if a task is delayed due to equipment failure, the system will adjust the order of other related tasks according to the optimized plan and immediately notify the construction personnel and resource management department to ensure that the overall construction progress is not significantly affected. The schedule adjustment module ensures that the plan adjustment generated by the system can be converted into actual operation in a timely manner by connecting with the on-site equipment.

[0041] Feedback module

[0042] The feedback module plays a role of closed-loop control in the present invention. It continuously monitors the execution of the construction progress and feeds back the task completion status, resource usage, equipment operation efficiency, etc. on site to the generative model and reinforcement learning model. Through these real-time feedbacks, the system can further optimize the output of the model and continuously improve the accuracy and execution efficiency of the plan.

[0043] The data collected in real time by the feedback module is not only used to optimize the current project, but also provides historical reference data for future projects, thereby improving the prediction ability and optimization efficiency of the generative model and reinforcement learning model. The feedback module ensures that the system can continuously accumulate experience in long-term project execution, achieve self-learning and improvement, and further improve the intelligent level of construction progress management.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The method of the present invention can effectively improve the adaptability of the construction schedule through continuous iterative optimization of the reinforcement learning model. After each task is completed, the model will update its strategy according to the latest task progress, thereby gradually optimizing the entire construction process. This dynamic adjustment capability significantly improves the flexibility of construction management, allowing the project to respond to changes in a timely manner, ensuring that the construction progress always remains on the expected track, and avoiding major delays caused by emergencies;

[0046] 2. The collaborative work of the generative model and the reinforcement learning model can effectively reduce the reliance on human experience. Traditional construction management often relies on the experience of the project manager to adjust the plan, while the present invention greatly reduces the necessity of human intervention through automated model adjustment. The synergy of the generative model and the reinforcement learning model makes the entire construction progress management process more standardized and automated, thereby improving the efficiency and reliability of project management.

[0047] 3. Through the collaborative work of various modules, the construction progress control system of the present invention can effectively realize dynamic optimization and real-time feedback control from initial plan generation to the construction process. The information exchange between modules ensures that the system can quickly respond to changes on site, provide the best construction progress plan, and realize efficient and intelligent construction management. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the construction progress control method of the present invention;

[0049] Figure 2 is a flow chart for training and generating a generative model of the present invention;

[0050] Figure 3 is an optimization flow chart of the reinforcement learning model of the present invention;

[0051] Figure 4 It is a system structure diagram of the construction progress control system of the present invention. DETAILED DESCRIPTION

[0052] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. Unless otherwise specified, the raw materials and equipment used can be purchased from the market or are commonly used in the art. The methods in the embodiments are conventional methods in the art unless otherwise specified. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0053] like Figure 1As shown in the figure, the flowchart of the construction progress control method of the present invention shows the whole process of the whole system from information collection, data aggregation, plan generation to dynamic optimization and re-verification. The whole process is coordinated by various modules to ensure that the construction project is automatically controlled and optimized during the planning and implementation process. The specific steps are as follows:

[0054] Before starting a construction project, you first need to design an initial information collection form. This collection form is used to define various key information required for the construction project, including a list of construction tasks, time requirements for each task, resource requirements (manpower, materials, equipment) and their constraints, and the impact of environmental factors (weather, temperature and humidity, etc.) on construction.

[0055] The initial information collection of the project is an important step before the start of the construction project, which can ensure that all required data items are correctly collected and used in the subsequent construction plan generation and optimization process. The collected content has been discussed and adjusted many times to reflect the actual needs and conditions of the construction site to the greatest extent.

[0056] The data acquisition module is an important part of the system. It relies on external sensors to monitor the construction site and collect data, including:

[0057] (1) Monitor the progress of each task through sensors, including the task start time, end time and completion percentage.

[0058] (2) Real-time monitoring of the use and efficiency of resources such as labor, equipment, and materials, such as the number of workers and the utilization rate of machinery.

[0059] (3) Collect environmental data that may affect the construction progress, such as weather changes, temperature, humidity, etc., through equipment such as temperature and humidity sensors and anemometers.

[0060] All collected data is transmitted to the system's central database via wireless network for aggregation and storage. These data will provide the basis for subsequent processing of generative models and reinforcement learning models.

[0061] The generative model module generates a preliminary construction schedule based on the aggregated construction site data. The schedule includes the start and end time of each construction task, the type and quantity of resources, the dependencies of tasks, etc. The generative model identifies the optimal sequence of tasks and resource allocation by learning from historical construction projects.

[0062] The generative model is characterized by the model using the Transformer architecture to generate new task schedules by encoding the characteristics of historical construction tasks. The generation of the preliminary plan is based on the learning results of similar projects in historical data, so the plan not only contains the time schedule of tasks, but also reflects the efficient use of resources.

[0063] Optimize the generated plan based on reinforcement learning model

[0064] During the execution of the construction project, the reinforcement learning model module optimizes the generated preliminary plan in real time. The reinforcement learning model receives real-time field data from the data acquisition module and adjusts the construction schedule based on this data. The specific steps are as follows:

[0065] (1) The reinforcement learning model takes real-time data from the construction site as input, including the progress status of the task, resource usage, and environmental changes.

[0066] (2) The model calculates rewards based on these data. The basis for setting positive and negative rewards is factors such as task completion on time, resource utilization efficiency, and construction safety. The model optimizes strategies through a deep Q network (DQN) and gradually adjusts the task sequence and resource allocation to improve overall construction efficiency and quality.

[0067] The optimized construction plan will be continuously fed back and adjusted according to the real-time situation at the construction site, so that the construction plan can dynamically adapt to the ever-changing construction environment and form a closed-loop control system.

[0068] At the same time, the optimized construction schedule needs to be verified at the actual construction site. During the verification process, the system will monitor the execution of each task in real time to ensure that the construction tasks are executed according to the plan. If there are differences between the plan and the actual situation during the execution process, the system will feed back these differences to the reinforcement learning model for further optimization and adjustment.

[0069] The verification content mainly includes:

[0070] (1) Progress tracking: Track the completion progress of each construction task in real time to verify whether the optimization plan is feasible.

[0071] (2) Resource effectiveness: Check whether the use of each resource achieves the expected effect and whether there is any resource waste or resource shortage.

[0072] (3) External environmental impact: Verify the impact of environmental data on the construction progress and ensure that the system can make corresponding adjustments when the environment changes.

[0073] The optimized construction schedule in the present invention can be used for actual construction guidance, helping the construction team to carry out construction according to the optimized task sequence and resource allocation. The system continuously collects real-time data to ensure that the progress of each task is in line with the optimized plan.

[0074] The system controls the progress of the construction site by constantly tracking the progress of tasks to ensure that each task is completed on time.

[0075] The optimized plan can guide the actual construction and improve the efficiency of resource utilization, avoid waste of resources, and ensure that the overall production efficiency is improved while ensuring the construction quality.

[0076] Through the initial information collection, data collection and aggregation, preliminary plan generation, reinforcement learning optimization, on-site verification and feedback re-optimization, the construction progress control method of the present invention forms a complete closed-loop control system. The system can dynamically adjust and optimize according to the real-time data of the construction site to ensure the smooth and efficient completion of the construction task and reduce project delays and resource waste.

[0077] The closed-loop process of the construction progress control method of the present invention realizes the full-process automatic control of the construction progress through the collaborative work of various modules. The system can not only generate a construction schedule through a generative model before the project starts, but also use a reinforcement learning model for dynamic optimization during the construction process, ultimately ensuring the best match between the construction plan and the site conditions.

[0078] like Figure 2 As shown, the present invention trains historical construction data through a generative model to generate a preliminary construction schedule. The generative model is built on the Transformer architecture and uses an encoder-decoder approach to model the dependencies and resource allocation of construction tasks. The following is a specific implementation method for generative model training and generation:

[0079] The data collection module collects a large amount of historical construction data from different completed construction projects, including the start and end time of tasks, resource usage (labor, materials, equipment, etc.), dependencies between tasks, and feedback data recorded during the construction process. The collected data covers different types of construction projects to ensure the generalization ability of the model.

[0080] In order to ensure the quality and consistency of the data, the collected data is preprocessed. The preprocessing process includes:

[0081] (1) Detect and remove outliers in construction data by setting reasonable thresholds to ensure the accuracy and representativeness of the data.

[0082] (2) Use interpolation, averaging, or other data interpolation methods to fill in missing data to ensure the integrity of the model training data.

[0083] (3) Standardize and normalize numerical data, scaling the data values ​​to a relatively consistent range (between 0 and 1) to facilitate convergence during model training.

[0084] (4) For categorical data (task type, resource type, etc.), embedding vectors are used for encoding. Embedding vectors are used to represent category information with intrinsic relationships as low-dimensional vectors, thereby improving the model's ability to understand category features.

[0085] The model training process adopts the supervised learning method, using the actual construction plan in the historical construction data as the supervision signal. The specific training steps include:

[0086] (1) The mean squared error (MSE) loss function is used to measure the difference between the construction schedule predicted by the model and the actual construction data.

[0087] (2) Use the gradient descent method to optimize the model parameters. Update the weights in the model through backpropagation, gradually reducing the loss value of the training data until the model converges.

[0088] During the model training process, the cross-validation method is used to evaluate the model performance to ensure that the model is not overfitting. The generalization ability of the model is obtained by dividing the training data into multiple subsets and alternating between training and validation. For the selection of hyperparameters (learning rate, number of hidden layers, number of attention heads), the grid search and random search methods are used for optimization to achieve the best parameter combination of the model.

[0089] After sufficient training, the generative model can generate a preliminary plan of the construction schedule based on the initial information of a new project. The input includes information such as the task list, resource constraints, and project size. The output plan includes the start and end time of each task, the type and quantity of resources required, and the dependencies between tasks.

[0090] The initial construction schedule generated by the generative model provides the basis for the management of the entire construction project. During the construction process, the generated plan will be further optimized in real time by the reinforcement learning model to cope with the constant changes at the construction site.

[0091] like Figure 3 As shown, during the construction process, the present invention uses a reinforcement learning model to dynamically optimize the generated preliminary construction plan to cope with various dynamic changes in the construction site. The reinforcement learning model adjusts the construction schedule in real time by learning and iterating feedback from real-time data on the construction site to ensure that the construction task can be completed efficiently. The following is the optimization process and implementation details of the reinforcement learning model:

[0092] The data acquisition module collects data from the construction site in real time, including:

[0093] (1) Completion progress of current construction tasks.

[0094] (2) Utilization rate and consumption of human resources, materials and equipment.

[0095] (3) Weather changes, temperature and humidity, etc. These factors may affect the construction progress and resource allocation.

[0096] All these data are collected by IoT devices (RFID sensors, temperature and humidity sensors, etc.) and transmitted to the reinforcement learning model module through wireless networks. The model uses these data to perceive the status of the construction site in real time.

[0097] The reinforcement learning model of the present invention is constructed based on the Deep Q-Network (DQN). The model maximizes long-term returns (rewards) by learning the mapping relationship between the state and action of the construction site. Specifically, the optimization process of the reinforcement learning model includes the following steps:

[0098] Define state, state S t It is composed of real-time data from the construction site, including the completion progress of the task, resource usage, dependencies of the current task, etc.

[0099] Define action, action A t Indicates the scheduling strategies that can be adopted in the current state, such as adjusting the order of tasks, reallocating resources, etc.

[0100] Design reward function, reward function R t The design is defined according to the construction objectives, mainly considering the following factors:

[0101] (1) Task completion on time: When a task is completed on time or ahead of schedule, a positive reward will be given.

[0102] (2) Resource utilization efficiency: When the resource utilization rate is high (reaching the average resource utilization rate), positive rewards are given.

[0103] (3) Construction quality: If the construction quality meets the standards, positive rewards will be given; otherwise, negative rewards will be given.

[0104] (4) Construction safety: Positive rewards will be given if the construction is safe and accident-free, and negative rewards will be given if the construction is safe.

[0105] (4) Delay and waste: If the task is delayed or resources are wasted, a negative reward is given.

[0106] Q-value function: The model uses the Q-value function Q(S,A) to estimate the expected cumulative reward that can be obtained after taking action A in state S. The Q-value is updated using the following formula:

[0107]

[0108] in:

[0109] α is the learning rate, which is used to control the step size of each update.

[0110] γ is a discount factor, which indicates the importance of future rewards.

[0111] R t is the immediate reward at the current time step.

[0112] Indicates the maximum Q value in the next state.

[0113] The deep Q network training process mainly includes the following steps:

[0114] Initialize the parameters θ of the Q network and the randomly initialized experience replay pool to store the four-tuple of state, action, reward and next state (S t ,A t ,R t ,S t+1 ).

[0115] At each time step, the model randomly extracts a batch of samples from the experience replay pool for training to reduce the correlation between samples and improve the stability of the model.

[0116] Adopt ∈-greedy strategy to select actions, randomly select actions with probability ∈, A t = random action, select the action that maximizes the Q value with a probability of 1-∈,

[0117] Execute action A t , observe the immediate reward R t and the new state S t+1 , the quaternion (S t ,A t ,R t ,S t+1 ) is stored in the experience replay pool.

[0118] Randomly draw samples from the experience replay pool and update the parameters θ of the Q network through backpropagation, using the above Q value update formula.

[0119] Repeat the above steps until the loss function of the Q network converges.

[0120] The reinforcement learning model calculates instant rewards based on the input real-time data. The design of rewards directly affects the degree to which the model optimizes the construction schedule. The specific reward calculation includes:

[0121] If a task is completed within the specified time, the reward value is +r1, otherwise it is -r2.

[0122] For construction arrangements that use resources efficiently (reaching the company's historical average resource utilization rate or above), a reward of +r3 will be given. If it is less than that, a penalty of -r4 will be given.

[0123] If there are no safety accidents during the construction process, a reward of +r5 will be given, otherwise a penalty of -r6 will be given.

[0124] If the quality of the staged inspection during the construction process meets the standard, a reward of +r7 will be given, otherwise a penalty of -r8 will be given.

[0125] Based on the reward calculation results, the model makes strategic adjustments to the execution order and resource allocation of construction tasks. After each adjustment, the reinforcement learning model attempts to optimize the arrangement of construction tasks to maximize the future cumulative rewards and ensure that the construction process is carried out efficiently and safely.

[0126] After multiple iterations of training and optimization, the reinforcement learning model outputs an optimized construction schedule. The optimized plan adjusts the order of tasks in the preliminary construction plan to cope with unforeseen events encountered during the construction process. It can also adjust the resource allocation of each task according to the on-site conditions to improve resource efficiency and reduce costs.

[0127] The reinforcement learning model continuously receives new real-time data during the construction process and continuously optimizes the construction plan through a closed-loop feedback mechanism. Whenever a new task is completed or the environment changes, the data acquisition module feeds real-time information back to the reinforcement learning model, which then makes adjustments accordingly to ensure that the construction plan is always consistent with the actual construction site.

[0128] like Figure 4 As shown, the construction progress control system of the present invention includes a data acquisition module, a generative model module, a reinforcement learning model module, a progress adjustment module and a feedback module. The functions of each module in the system are clear and they work together to achieve automatic generation, optimization and dynamic control of the construction progress. The following is a specific implementation description of the system modules and their interrelationships:

[0129] The data acquisition module is responsible for acquiring real-time data from the construction site, including task progress, resource usage, and environmental data. The collected data is transmitted to the central database via a wireless network, providing a data basis for each module in the system.

[0130] The generative model module generates a preliminary construction schedule based on the initial conditions of the project, including task sequence, resource allocation and time arrangement. This module generates a reasonable initial plan for the new project by learning from historical construction data as the basis for starting the construction task.

[0131] The reinforcement learning model module dynamically optimizes the preliminary plan generated by the generative model. Through the real-time collected field data, the model adjusts the task sequence and resource allocation based on the reward mechanism to ensure that the construction plan can flexibly respond to changes in construction.

[0132] The schedule adjustment module adjusts the task arrangement and resource allocation of the construction site according to the optimization results of the reinforcement learning model. The optimized plan is communicated to the construction team through the management system to ensure that the construction activities are carried out according to the optimal plan of the system.

[0133] The feedback module continuously monitors the execution status of the construction tasks and feeds back real-time data to the reinforcement learning model module to achieve further optimization of the plan and closed-loop control.

[0134] Module collaboration and system advantages

[0135] Each module works together to form a closed-loop control system from data collection, plan generation to real-time optimization and feedback adjustment. The data collection module provides real-time data of the construction site, the generative model module generates a preliminary plan, the reinforcement learning model module optimizes the plan, the schedule adjustment module is responsible for on-site implementation, and the feedback module forms a closed-loop monitoring. Through this structure, the system can realize automated plan generation, optimization and dynamic adjustment during the construction process, improve construction efficiency and reduce project delays.

[0136] The integrated design of the system ensures efficient collaboration among modules, thereby realizing intelligent control and dynamic adjustment of construction progress, significantly improving the automation and accuracy of construction management.

[0137] Although the embodiments of the present invention have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the above embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. The construction progress automatic control method based on generative model and reinforcement learning is characterized by: Includes the following: S1.Generative model training; Generative models are trained based on a large amount of historical construction project data, including project schedules, resource allocation, dependencies between tasks, and actual feedback data recorded during the construction process; S2. Construction schedule generation; Automatically generate a construction schedule based on the specific needs of the project to be constructed. The generated construction schedule is presented in the form of a task list, which contains the detailed arrangement of each task, including the start time, end time, type and quantity of required resources, and the dependencies between tasks; S3. Reinforcement learning model optimization; The reinforcement learning model is introduced to dynamically adjust and optimize the initially generated construction schedule. The reinforcement learning model adaptively responds to changes in the construction process and continuously optimizes and adjusts the sequence of construction tasks and resource allocation. S4. Collaboration between generative models and reinforcement learning models.

2. The construction progress automatic control method based on generative model and reinforcement learning according to claim 1 is characterized in that: Before model training, the data is cleaned and preprocessed, including removing outliers, filling in missing data, and standardizing data to ensure the consistency and reliability of the input data.

3. The construction progress automatic control method based on generative model and reinforcement learning according to claim 1 is characterized in that: The generated construction schedule can be further adjusted according to the project characteristics.

4. The construction progress automatic control method based on generative model and reinforcement learning according to claim 1 is characterized in that: When optimizing the reinforcement learning model, a reward mechanism is set up to give different rewards to matters of different urgency, and its optimization strategy is flexibly adjusted to achieve a balance between different goals.

5. The construction progress automatic control system based on generative model and reinforcement learning is characterized by: It includes five core modules: data acquisition module, generative model module, reinforcement learning model module, progress adjustment module and feedback module.

6. The construction progress automatic control system based on generative model and reinforcement learning according to claim 5 is characterized in that: The data acquisition module, as the basic module of the construction progress control system, collects engineering data from the construction site in real time. Specific data include task completion progress, resource usage, material inventory, mechanical equipment operating status, and environmental data.

7. The construction progress automatic control system based on generative model and reinforcement learning according to claim 5 is characterized in that: The generative model module generates a construction schedule based on the input project parameters at the beginning of the project.

8. The construction progress automatic control system based on generative model and reinforcement learning according to claim 7 is characterized in that: Reinforcement learning model module for optimizing construction schedules in real time.

9. The construction progress automatic control system based on generative model and reinforcement learning according to claim 8 is characterized in that: The progress adjustment module updates the construction plan in real time based on the optimization suggestions provided by the reinforcement learning model.

10. The construction progress automatic control system based on generative model and reinforcement learning according to claim 9 is characterized in that: The feedback module plays the role of closed-loop control, continuously monitoring the execution of the construction progress and feeding back the on-site situation to the generative model and reinforcement learning model.

Images