Intelligent scheduling method based on fabricated bridge storage area

Abstract

The invention provides an intelligent scheduling method based on a fabricated bridge storage area, and relates to the technical field of intelligent construction, and the method comprises the following steps: screening out a plurality of candidate scheduling schemes with the highest score through calculating the reward scores of different scheduling schemes based on a scheduling DDQN algorithm; the method comprises the following steps: converting a factory into a graph structure based on a library field scheduling MAPF algorithm, accurately mapping a physical space through a gridding environment, taking each piece of transportation equipment as an independent agent, and capturing path crossing and resource preemption in real time by utilizing a conflict detection function; and by adopting an MAPF algorithm, aiming at each candidate plan scheme, calculating and obtaining a corresponding library field use scheme. According to the invention, intelligent scheduling of warehouse-in and warehouse-out links of the prefabricated bridge prefabricated parts can be realized, the storage position is optimized, and the reshipment frequency is reduced.

Description

Technical Field

[0001] The present invention relates to the field of smart construction technology, and in particular to a smart scheduling method based on a prefabricated bridge storage area. Background Art

[0002] Although some workshops have begun adopting automated production lines and digital management systems, overall, processes remain inadequately refined and information technology levels vary. Many workshops suffer from inaccurate production scheduling and unscientific inventory management. Manual management leads to delayed and inaccurate production scheduling, resulting in low production efficiency. Low inventory management accuracy results in material waste or insufficient inventory, hindering production progress.

[0003] Considering the multi-objective optimization problem of beam transportation scheduling, we studied intelligent management and control technology for beam storage areas based on artificial intelligence scheduling algorithms. Based on existing intelligent scheduling algorithms related to ports, warehouses, and the actual needs of the project's beam storage areas, we constructed a beam yard scheduling knowledge base, introduced expert systems, and studied how to simulate and replace human brain decision-making mechanisms. We further deployed production scheduling and inventory management solutions for beam storage areas to minimize beam transfers, optimize transportation routes, and maximize construction efficiency. Summary of the Invention

[0004] In response to the above-mentioned existing technologies, the present invention proposes an intelligent scheduling method based on the storage area of ​​prefabricated bridges. By combining the unique coding of prefabricated components (box girders) and their subsequent beam erection and use plan, the optimal storage position of the box girders is calculated to reduce the number of subsequent transportation or shorten the transportation path to the shortest possible time. Before the beams are moved into storage, the platform completes the layout plan through its built-in algorithm, determines the optimal location, and sends a signal at the signal terminal to determine the optimal travel path, connecting to the intelligent operation system for moving beams on site.

[0005] The present invention provides an intelligent scheduling method based on a prefabricated bridge storage area, comprising the following steps:

[0006] Based on the DDQN scheduling algorithm, the reward scores of different scheduling plans are calculated to select multiple candidate scheduling plans with the highest scores;

[0007] Based on the MAPF algorithm for warehouse scheduling, the factory area is converted into a graph structure, and the physical space is accurately mapped through a grid environment. Each transport device acts as an independent intelligent agent, and conflict detection functions are used to capture path intersections and resource preemption in real time.

[0008] The MAPF algorithm is used to calculate and obtain the corresponding warehouse utilization plan for each candidate plan.

[0009] Preferably, in the scheduling and dispatching DDQN algorithm, the scheduling problem is modeled by Markov decision process, the state space integrates the real-time state of the warehouse, the completion degree of the plan, the load of the equipment and personnel, and the temporary change instruction, and the action space synchronously optimizes the four-dimensional decisions of the warehouse location allocation, path planning, team scheduling and equipment assignment; the prior knowledge driven reinforcement learning is adopted, the order data is preprocessed by spectral clustering to accelerate the training, the double Q network architecture and the epsilon-greedy strategy are combined to solve the overestimation problem and balance the exploration-exploitation contradiction; the engineering training mechanism is designed, the sample correlation is broken based on the five-tuple experience replay, and the periodic target network parameter synchronization and the mean square error back propagation are adopted to ensure the robust convergence of the algorithm in the dynamic industrial environment.

[0010] Preferably, in the warehouse scheduling MAPF algorithm, based on the dynamic priority adjustment mechanism, the three-dimensional weights of task urgency, equipment load rate and path time cost are introduced into the CBS algorithm kernel to real-time allocate the priority of the agent and replace the static strategy; the time-space decoupling collaborative optimization is adopted to fuse the time window offset and the A* local replanning dual-mode strategy; the high-priority task is fine-tuned in time sequence, and the ordinary task is dynamically obstacle-avoiding, so that the Pareto balance of zero-collision path and global time consumption minimization is realized, and the scheduling efficiency and safety of the fabricated bridge storage area are significantly improved.

[0011] Preferably, in the MAPF algorithm, after obtaining multiple candidate schemes, the warehouse path scheme corresponding to each scheme is calculated by using the MAPF algorithm; for each candidate scheme, the MAPF algorithm simulates multiple agents to ensure that they can efficiently and safely complete their respective tasks under the constraints of limited space and resources; by comparing the performance of each candidate scheme in the warehouse path and its related cost, and combining the reverse analysis method, the advantages and disadvantages of each scheme are evaluated; by comprehensively considering the efficiency, resource consumption and other factors of each scheme, the scheme with the lowest cost and the highest benefit is finally determined.

[0012] Compared with the prior art, the beneficial effects of the present application are as follows: the present application proposes a smart scheduling method based on the fabricated bridge storage area, which realizes the intelligent scheduling of the fabricated bridge prefabricated component in and out of the warehouse by developing the algorithm, optimizes the storage position, and reduces the number of rehandling. The present application can effectively solve the problems of current workshop process management not being fine, informationization level being uneven, etc., accurately optimize the beam transportation scheduling, reduce the number of rehandling beams, optimize the transportation path, and improve the construction efficiency. The present application can rely on high-speed infrastructure and base production system data to build a data resource pool and realize accurate data management. The intelligent scheduling algorithm is closely combined with the business process, breaks the traditional business mode, uses advanced technology to improve the production scheduling information integration level, strengthens the operation plan realization rate management and business prediction and early warning capability, and is expected to greatly improve the production efficiency in the related field, reduce the cost, and promote the intelligent development of the industry. DETAILED DESCRIPTION

[0013] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific embodiments.

[0014] Example: A smart scheduling method based on a prefabricated bridge storage area includes the following steps:

[0015] Based on the DDQN scheduling algorithm, a multi-constraint dynamic decision-making framework is used. That is, by calculating the reward scores of different scheduling plans, multiple candidate scheduling plans with the highest scores are screened out. Specifically, the scheduling problem is modeled through a Markov decision process (MDP). The state space integrates the real-time status of the warehouse, the completion rate of the plan, the load of equipment and personnel, and temporary change instructions. The action space simultaneously optimizes the four-dimensional decision-making of warehouse location allocation, path planning, team scheduling, and equipment assignment. Prior knowledge-driven reinforcement learning is adopted, and spectral clustering is introduced to pre-process order data to accelerate training. The dual Q network architecture is combined with the ∈-greedy strategy to solve the over-estimation problem and balance the exploration-exploitation contradiction. An engineered training mechanism is designed to break the sample correlation based on quintuple experience replay. Through periodic target network parameter synchronization and mean square error back propagation, the robust convergence of the algorithm in a dynamic industrial environment is ensured.

[0016] Based on the MAPF algorithm for warehouse scheduling, a conflict-driven multi-agent modeling method is used to convert the factory area into a graph structure (G=(V,E)). The physical space is accurately mapped through a grid environment. Each transportation equipment acts as an independent agent, and the conflict detection function is used to capture path intersections and resource preemption in real time. Furthermore, based on the dynamic priority adjustment mechanism, the three-dimensional weights of task urgency, equipment load rate and path time cost are introduced into the CBS algorithm kernel to allocate agent priorities in real time, replacing the static strategy. The spatiotemporal decoupling collaborative optimization is adopted, integrating the time window offset (±Δt) and A* local replanning dual-mode strategy. High-priority tasks are fine-tuned in timing, and ordinary tasks are dynamically obstacle-avoided to achieve a Pareto equilibrium of zero-collision paths and global time minimization, significantly improving the scheduling efficiency and safety of prefabricated bridge storage areas.

[0017] The MAPF algorithm is used to calculate and derive the corresponding depot utilization plan for each candidate plan. After obtaining multiple candidate plans, the MAPF algorithm is used to calculate the depot path plan corresponding to each plan. For each candidate plan, the MAPF algorithm simulates multiple agents to ensure that they can complete their respective tasks efficiently and safely within the constraints of limited space and resources. By comparing the performance of each candidate plan in the depot path and its associated costs, combined with a reverse analysis method, the advantages and disadvantages of each plan are evaluated. By comprehensively considering factors such as the efficiency and resource consumption of each plan, the lowest cost and highest benefit plan is ultimately determined.

[0018] For example, input:

[0019] number of iterations T,

[0020] dimension of state feature n,

[0021] ActionSpace,

[0022] step size a,

[0023] decay factor g,

[0024] exploration rate e,

[0025] current Q network reward Q,

[0026] target Q network reward Q',

[0027] number of samples m of batch gradient descent,

[0028] target Q network parameter update frequency C.

[0029] Output:

[0030] current Q network parameters.

[0031] Randomly initialize the value Q corresponding to all state-action pairs, randomly initialize all parameters w of the current Q network, initialize the parameters w' of the target Q network Q' = w. Empty the experience replay set D. Take the port long-term berthing plan and the daily day-night berthing plan as the single round calculation step size of the agent, and prefer to feed the prior order data after spectral clustering into the deep dual Q value network to prepare for iterative training.

[0032] Iteration process:

[0033] Initialize S as the first state of the current state sequence, get its feature vector φ(S),

[0034] Use φ(S) as input in the Q network to get the Q value output corresponding to all actions of the Q network. Use the e-greedy method to select the corresponding action A in the current Q value output,

[0035] Execute the current action A in state S to get the feature vector φ(S') and the reward R$ corresponding to the new state S', and whether the state is_end,

[0036] Store the five-tuple {φ(S), A, R, φ(S'), is_end} in the experience replay set D,

[0037] S = S',

[0038] Sample m from the experience replay set D,

[0039] {φ(S j ), Aj ,R j ,φ(S' j ),is_end j},j=1,2.,,,m,

[0040] Calculate the current target Q value y j ,

[0041]

[0042] Use the mean squared error loss function

[0043]

[0044] Update all parameters w of the Q network through the gradient back propagation of the neural network,

[0045] If T%C=1, then update the target Q network parameter w′=w,

[0046] If S′ is the terminal state, the current round of iteration is completed.

[0047] The DDQN algorithm evaluates multiple solutions. Based on the current state of the proposed solution and its interaction with the environment, the algorithm continuously updates the Q-value to predict the reward score for each solution. The reward score reflects the overall effectiveness of each solution in terms of resource allocation, timeliness, and space utilization. Ultimately, the DDQN algorithm selects the most promising solutions based on these reward scores.

[0048] This embodiment relies on the infrastructure of the Yanluo Expressway and obtains data through the production system of the base. This project (DMS part) will unify the data standards for data interaction and create a data resource pool that integrates data storage, development, governance, analysis and other functions; build a more accurate, real-time and efficient data collection system to uniformly acquire, manage and publish data, realize accurate management of data, ensure data security and controllability, and reduce the subsequent external system docking costs.

[0049] The intelligent scheduling algorithm is an algorithm related to specific business process rules (algorithm middle platform). This project (strategy middle platform) realizes the goals of plan formulation, task scheduling, and result optimization according to the actual business logic. Utilizing advanced technologies such as intelligent algorithms and big data analysis, we provide algorithm implementation for path planning, intelligent warehouse scheduling, smart access and other functions in the actual production process, and jointly play the role of an intelligent brain. By sorting out the current production scheduling processes and management models, breaking the traditional business model, and utilizing artificial intelligence technology, big data technology, and cloud computing technology, we build intelligent algorithm models to realize multi-module conjugate output plans. Improve the level of production scheduling information integration, carry out business scenario data modeling, strengthen the management of job plan fulfillment rates, and enhance business forecasting and early warning capabilities.

[0050] The above are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent solutions made using the contents of the present invention specification, directly or indirectly applied in other related technical fields, are also within the patent protection scope of the present invention.

Claims

1. A smart scheduling method based on the storage area of ​​​​prefabricated bridges, characterized in that: The steps include: Based on the DDQN scheduling algorithm, the reward scores of different scheduling plans are calculated to select multiple candidate scheduling plans with the highest scores; Based on the MAPF algorithm for warehouse scheduling, the factory area is converted into a graph structure, and the physical space is accurately mapped through a grid environment. Each transport device acts as an independent intelligent agent, and conflict detection functions are used to capture path intersections and resource preemption in real time. The MAPF algorithm is used to calculate and obtain the corresponding warehouse utilization plan for each candidate plan.

2. The intelligent scheduling method based on the storage area of ​​the prefabricated bridge according to claim 1 is characterized in that: Based on the scheduling DDQN algorithm, the scheduling problem is modeled through a Markov decision process. The state space integrates the real-time status of the warehouse, the completion rate of the plan, the load of equipment and personnel, and temporary change instructions. The action space simultaneously optimizes the four-dimensional decision-making of storage location allocation, path planning, team scheduling, and equipment assignment. It uses reinforcement learning driven by prior knowledge and introduces spectral clustering to pre-process order data to accelerate training. The dual-Q network architecture and the ϵ-greedy strategy are combined to solve the overestimation problem and balance the exploration-exploitation contradiction. An engineered training mechanism is designed to break sample correlation based on quintuple experience replay, and robust convergence of the algorithm in dynamic industrial environments is ensured through periodic target network parameter synchronization and mean square error back propagation.

3. The intelligent scheduling method based on the storage area of ​​the prefabricated bridge according to claim 1 or 2, characterized in that: Based on the dynamic priority adjustment mechanism in the MAPF algorithm for warehouse scheduling, the three-dimensional weights of task urgency, equipment load rate and path time cost are introduced into the core of the CBS algorithm to assign agent priorities in real time, replacing the static strategy; Adopting spatiotemporal decoupling collaborative optimization, integrating time window offset and A* local replanning dual-mode strategy; Fine-tune the timing of high-priority tasks and dynamically avoid obstacles for ordinary tasks to achieve a Pareto equilibrium of zero-collision paths and minimized global time consumption, significantly improving the scheduling efficiency and safety of prefabricated bridge storage areas.

4. The intelligent scheduling method based on the storage area of ​​the prefabricated bridge according to claim 1 or 2, characterized in that: In the MAPF algorithm, after obtaining multiple candidate solutions, the MAPF algorithm is used to calculate the warehouse path solution corresponding to each solution. For each candidate solution, the MAPF algorithm simulates multiple intelligent agents to ensure that they can complete their respective tasks efficiently and safely under limited space and resource constraints; by comparing the performance of each candidate solution in the warehouse path and its related costs, combined with the reverse analysis method, the advantages and disadvantages of each solution are evaluated; by comprehensively considering factors such as the efficiency and resource consumption of each solution, the solution with the lowest cost and the highest benefit is finally determined.

Images